Exploring Difficulties in Parsing Imperatives and Questions

What it is about

Parsers trained on the Wall Street Journal fail on imperatives and questions even when the words are known, because the word order and the missing subject change the structure. The authors extract 750 imperatives and about 1,240 questions from the fiction part of the Brown Corpus of the Penn Treebank, and compare them with 3,859 questions from QuestionBank. They test two dependency parsers (MST, Malt), a deep HPSG parser (Enju) and a POS tagger, each adapted by a simple supervised method. Adaptation helps: 2 to 8 points on the Brown sets and 25 to 35 points on QuestionBank. Error analysis gives three causes: sentence-initial verbs tagged as nouns, a name used as a vocative taking the ROOT role, and a fronted to-infinitive clause that makes the parser choose the wrong main verb. For the Malt parser, ROOT errors on imperatives fall from 33 to 9 per 100 sentences after adaptation.

Ideas

• I1. Tagger rule for the first word: in en/src/ptag.rs, a sentence-initial bare verb with no subject is tagged as a verb unless a determiner follows. The paper's most common tagger error is exactly this, so measure the error count per school-text lesson before and after.
• I2. Vocative rule in en/src/parse.rs: a name or address term set off by commas at the start or end of a sentence gets a vocative role and is never the root. Measure ROOT accuracy on imperative lessons, with the school questions as a held-out set.
• I3. Partition before parsing: a fronted "To X, do Y" or "If X, ..." is split off as its own unit, so the main imperative clause is parsed alone. Try it as a rule in the parser front end, and use a negative control in which the same clause is kept whole; the split should win only where it is marked.
• I4. Question-type table: label each question as wh, yes/no or tag in prag, and report accuracy per type. The paper shows yes/no questions are under-learned from QA-style data, so check the balance in the school question set.
• I5. Punctuation: the paper notes that quotation marks and "?" or "!" placed outside the quotes confuse the root choice. Add the two placement patterns to 07. Punctuation, capital letters, abbreviations and numbers and test them with the tokenizer en/src/tok.rs.

For Mova

Imperatives and questions: 26. Questions and 21. The simple clause. Vocatives and dialogue: 38. Dialogue, discourse markers and reported speech. Punctuation: 07. Punctuation, capital letters, abbreviations and numbers. Code: en/src/ptag.rs (first-word tagging), en/src/parse.rs (vocative, ROOT choice), en/src/tok.rs (quotes and punctuation), prag for the speech-act label. Measure: tagging accuracy and ROOT accuracy on a held-out set of imperatives and questions, split by lesson, run twice for determinism.

Topics: Part-of-speech tagging

Cite

The paper itself, not this card. BibTeX from the library's record of the work.

@inproceedings{hara2011exploring,
  title = {{Exploring Difficulties in Parsing Imperatives and Questions}},
  author = {Hara, Tadayoshi and Matsuzaki, Takuya and Miyao, Yusuke and Tsujii, Jun'ichi},
  booktitle = {Proceedings of 5th International Joint Conference on Natural Language Processing},
  year = {2011},
  url = {https://aclanthology.org/I11-1084/}
}

References

21 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. John Blitzer, Ryan Mcdonald, and Fernando Pereira. 2006. Domain adaptation with structural correspondence learning. In Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, pages 120–128, Sydney, Australia. find
2. Stephen Clark and James R. Curran. 2007. Wide-coverage efficient statistical parsing with CCG and log-linear models. Computational Linguistics, 33(4):493–552. find our card
3. A. B. Clegg and A. Shepherd. 2005. Evaluating and integrating treebank parsers on a biomedical corpus. In Proceedings of the ACL 2005 Workshop on Software, Ann Arbor, Michigan. find
4. Tadayoshi Hara, Yusuke Miyao, and Jun’ichi Tsujii. 2007. Evaluating impact of re-training a lexical disambiguation model on domain adaptation of an HPSG parser. In Proceedings of 10th International Conference on Parsing Technologies (IWPT 2007), pages 11–22. find
5. Frederick Jelinek. 1998. Statistical Methods for Speech Recognition. The MIT Press, Cambridge, MA, USA. find
6. Richard Johansson and Pierre Nugues. 2007. Extended constituent-to-dependency conversion for english. In Proceedings of NODALIDA 2007. find our card
7. John Judge, Aoife Cahill, and Josef van Genabith. 2006. Questionbank: Creating a Corpus of Parsing-Annotated Questions. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the ACL, pages 497–504. find
8. David McClosky, Eugene Charniak, and Mark Johnson. 2006. Reranking and self-training for parser adaptation. In Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics, pages 337–344, Sydney, Australia. find our card
9. David McClosky, Eugene Charniak, and Mark Johnson. 2010. Automatic Domain Adaptation for Parsing. In Proceedings of the 2010 Annual Conference of the North American Chapter of the ACL, pages 28–36, Los Angeles, California. find
10. Ryan McDonald, Koby Crammer, and Fernando Pereira. 2005a. Online large-margin training of dependency parsers. In Proceedings of ACL-2005. find our card
11. Ryan McDonald, Fernando Pereira, Kiril Ribarov, and Jan Hajič. 2005b. Non-projective dependency parsing using spanning tree algorithms. In Proceedings of HLT/EMNLP-2005. find our card
12. Yusuke Miyao and Jun’ichi Tsujii. 2005. Probabilistic disambiguation models for wide-coverage HPSG parsing. In Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics (ACL), pages 83–90. find our card
13. Takashi Ninomiya, Takuya Matsuzaki, Yusuke Miyao, and Jun’ichi Tsujii. 2007. A log-linear model with an n-gram reference distribution for accurate hpsg parsing. In Proceedings of IWPT 2007, June. Prague, Czech Republic. find
14. Joakim Nivre, Johan Hall, and Jens Nilsson. 2006. MaltParser: A data-driven parser-generator for dependency parsing. In Proceedings of LREC, pages 2216–2219. find
15. Joakim Nivre, Johan Hall, Sandra Kübler, Ryan Mc-Donald, Jens Nilsson, Sebastian Riedel, and Deniz Yuret. 2007. find
16. The CoNLL 2007 shared task on dependency parsing. In Proceedings of the CoNLL Shared Task Session of EMNLP-CoNLL 2007, pages 915–932, Prague, Czech Republic, June. Association for Computational Linguistics. find
17. Laura Rimell and Stephen Clark. 2008. Adapting a Lexicalized-Grammar Parser to Contrasting Domains. In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pages 475–584. find
18. Brian Roark and Michiel Bacchiani. 2003. Supervised and unsupervised PCFG adaptation to novel domains. In Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology, pages 126–133, Edmonton, Canada. find
19. Mark Steedman, Miles Osborne, Anoop Sarkar, Stephen Clark, Rebecca Hwa, Julia Hockenmaier, Paul Ruhlen, Steven Baker, and Jeremiah Crim. 2003. Bootstrapping statistical parsers from small datasets. In Proceedings of the tenth conference on European chapter of the Association for Computational Linguistics, pages 331–338, Budapest, Hungary. find
20. Ivan Titov and James Henderson. 2006. Porting statistical parsers with data-defined kernels. In Proceedings of the Tenth Conference on Computational Natural Language Learning, pages 6–13, New York City. find
21. Yoshimasa Tsuruoka, Yuka Tateishi, Jin-Dong Kim, Tomoko Ohta, John McNaught, Sophia Ananiadou, and Jun’ichi Tsujii. 2005. Developing a robust part-of-speech tagger for biomedical text. In Advances in Informatics - 10th Panhellenic Conference on Informatics, volume LNCS 3746, pages 382–392, Volos, Greece, November. ISSN 0302-9743. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/hara-2011-exploring-difficulties-parsing-imperatives-questions/