Robust VPE Detection using Automatically Parsed Text

What it is about

This short student paper is the robust version of Nielsen's detector. It uses machine learning on free text that has been parsed automatically, with no hand-written parses. On a mixed corpus the detector reaches 70% F1. Two parsers are compared. The Charniak parser gives full structure, and RASP gives full parses for 70% of sentences, partial parses for 28%, and tags only for 2%. Re-parsing the Penn Treebank gives 63% F1, rising to 68% with extra features. The BNC, which has tags but no parses, gives 76% F1 with lexical forms and tags alone. A parse-based feature, the auxiliary-final VP, works well with Charniak's structures but only half as well with RASP's. The empty-VP feature keeps a precision above 80%, but its recall drops to about 20% on parser output.

Ideas

• I1. en parse-eval: report detection F1 separately for sentences with a full parse, a partial parse and no parse, as the RASP split in the paper suggests. This shows where the pipeline loses sentences, on EWT test.
• I2. Feature reliability per parser: for each parse-based feature in en, measure its precision and recall on gold trees and on en trees. The feature that loses most on parser output is the first to be made more robust.
• I3. Empty-VP analogue: Mova has no empty nodes, so use the rule that a modal with no following verb marks a VPE site. Its precision and recall should be reported in the same way as the paper's empty-VP feature, so that the drop on parser output can be seen.

For Mova

The code is en/src/expert.rs and the en parse-eval command. The chapter is 29. Ellipsis, word order and information structure. The BNC is NC, so the BNC part is test-only, and the paper's parsers are not Mova parsers. Measure first on EWT test, with the full, partial and no-parse split.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.3115/1219079.1219084

@inproceedings{nielsen2004robust,
  title = {{Robust VPE Detection using Automatically Parsed Text}},
  author = {Nielsen, Leif Arda},
  booktitle = {Proceedings of the ACL Student Research Workshop},
  year = {2004},
  doi = {10.3115/1219079.1219084},
  url = {https://doi.org/10.3115/1219079.1219084}
}

References

23 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Eric Brill. 1995. Transformation-based error-driven learning and natural language processing: A case study in part-of-speech tagging. Computational Linguistics, 21(4):543–565. find
2. E. Briscoe and J. Carroll. 2002. Robust accurate statistical annotation of general text. In Proceedings of the 3rd International Conference on Language Resources and Evaluation, Las Palmas, Gran Canaria. find
3. Eugene Charniak. 2000. A maximum-entropy-inspired parser. In Meeting of the North American Chapter of the ACL, page 132. find
4. Walter Daelemans, Jakub Zavrel, Ko van der Sloot, and Antal van den Bosch. 2002. Tilburg memory based learner, version 4.3, reference guide. Downloadable from http://ilk.kub.nl/downloads/pub/papers/ilk0210.ps.gz. ↗
5. Mary Dalrymple, Stuart M. Shieber, and Fernando Pereira. 1991. Ellipsis and higher-order unification. Linguistics and Philosophy, 14:399–452. find
6. Robert Fiengo and Robert May. 1994. Indices and Identity. MIT Press, Cambridge, MA. find
7. Daniel Hardt. 1993. VP Ellipsis: Form, Meaning, and Processing. Ph.D. thesis, University of Pennsylvania. find
8. Daniel Hardt. 1997. An empirical approach to vp ellipsis. Computational Linguistics, 23(4). find our card
9. Mark Johnson. 2002. A simple pattern-matching algorithm for recovering empty nodes and their antecedents. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics. find
10. Andrew Kehler and Gregory Ward. 1999. On the semantics and pragmatics of ‘identifier so’. In Ken Turner, editor, The Semantics/Pragmatics Interface from Different Points of View (Current Research in the Semantics/Pragmatics Interface Series, Volume I). Amsterdam: Elsevier. find
11. MBL Prec 62.30 65.41 66.92 71.31 GIS-MaxEnt Rec Prec F1 38.66 79.45 52.01 50.66 73.78 60.07 43.42 56.89 49.25 61.84 72.30 66.66 L-BFGS-MaxEnt Rec Prec F1 56.66 71.42 63.19 65.33 72.05 68.53 51.63 79.00 62.45 62.74 73.84 67.84 F1 57.85 61.48 60.93 63.50 MBL Prec 63.63 67.16 63.21 73.29 GIS-MaxEnt Rec… find
12. Andrew Kehler. 1993. A discourse copying algorithm for ellipsis and anaphora resolution. In Proceedings of the Sixth Conference of the European Chapter of the Association for Computational Linguistics (EACL-93), Utrecht, the Netherlands. find
13. Torbjorn Lager. 1999. The mu-tbl system: Logic programming tools for transformation-based learning. In Third International Workshop on Computational Natural Language Learning (CoNLL’99). Downloadable from http://www.ling.gu.se/ lager/mutbl.html. ↗
14. Shalom Lappin. 1993. The syntactic basis of ellipsis resolution. In S. Berman and A. Hestvik, editors, Proceedings of the Stuttgart Ellipsis Workshop, Arbeitspapiere des Sonderforschungsbereichs 340, Bericht Nr. 29-1992. University of Stuttgart, Stuttgart. find
15. Shalom Lappin. 1996. The interpretation of ellipsis. In Shalom Lappin, editor, The Handbook of Contemporary Semantic Theory, pages 145–175. Oxford: Blackwell. find
16. G. Leech. 1992. 100 million words of english : The British National Corpus. Language Research, 28(1):1–13. find
17. Robert Malouf. 2002. A comparison of algorithms for maximum entropy parameter estimation. In Proceedings of the Sixth Conference on Natural Language Learning (CoNLL-2002), pages 49–55. find
18. M. Marcus, G. Kim, M. Marcinkiewicz, R. MacIntyre, M. Bies, M. Ferguson, K. Katz, and B. Schasberger. 1994. The Penn Treebank: Annotating predicate argument structure. In Proceedings of the Human Language Technology Workshop. Morgan Kaufmann, San Francisco. find our card
19. Leif Arda Nielsen. 2003a. A corpus-based study of verb phrase ellipsis. In Proceedings of the 6th Annual CLUK Research Colloquium. find
20. Leif Arda Nielsen. 2003b. Using machine learning techniques for VPE detection. In Proceedings of RANLP. find
21. R. Quinlan. 1993. C4.5: Programs for Machine Learning. San Mateo, CA: Morgan Kaufmann. find our card
22. Adwait Ratnaparkhi. 1998. Maximum Entropy Models for Natural Language Ambiguity Resolution. Ph.D. thesis, University of Pennsylvania. find
23. Stuart Shieber, Fernando Pereira, and Mary Dalrymple. 1996. Interactions of scope and ellipsis. Linguistics and Philosophy, 19(5):527–552. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2004-nielsen-robust-vpe-detection-automatically-parsed/