Mind the Gap: Data Enrichment in Dependency Parsing of Elliptical Constructions

What it is about

Gapping (for example "Mary won gold and Peter bronze") is rare in treebanks, so parsers learn it poorly: the orphan relation has almost no training examples. The authors enrich training data in two ways: they parse web-crawl sentences with two parsers and keep only the trees on which both agree, sampled so that tree length and relation diversity follow the treebank; and they add artificial gapping sentences made by deleting a coordinated verb. Five languages are tested with the CoNLL 2017 winning parser. For English, adding the artificial sentences and 15% of the filtered web data raises the orphan F-score from 3.4 to 36.2, while overall LAS stays at about 87 (86.94 to 87.07). Improvements in overall LAS for the other languages are small, and the Czech treebank does not benefit.

Ideas

• I1. en/src/parse.rs: measure orphan precision, recall and F on the English gapping test (the 466 sentences of Schuster et al. 2018 plus the two orphan relations of EWT test), and LAS on EWT test as the guard; test only, if the licence of that set does not allow training.
• I2. annot crate and gparse: label raw text with the en perceptron and gparse, and keep only sentences on which both agree, as a silver source for orphan examples; metric: orphan F on the test set above, LAS on EWT test, run twice to check determinism.
• I3. Stratified sampling of silver trees by (tokens, unique relation types per token): compare with random sampling at equal token count on EWT dev with en parse-eval; the paper finds the stratified sample better.
• I4. Artificial gapping by deleting the verb of a coordinated clause, filtered by the grammar rules in en/seeds/verbal rather than by hand; metric: share of artificial sentences a human judges grammatical on 50 samples.

For Mova

This is 29. Ellipsis, word order and information structure and 24-coordination-subordination.md. Code: en/src/parse.rs for the gate, annot for the silver trees, and en/src/ctx.rs if orphan frames are to enter world. The idea that sentences both parsers agree on are safe is an example of "examples are not truth": measure the error rate of the agreed set before using it.

Topics: Dependency parsing · Treebanks

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/w18-6006

@inproceedings{droganova2018mind,
  title = {{Mind the Gap: Data Enrichment in Dependency Parsing of Elliptical Constructions}},
  author = {Droganova, Kira and Ginter, Filip and Kanerva, Jenna and Zeman, Daniel},
  booktitle = {Proceedings of the Second Workshop on Universal Dependencies (UDW 2018)},
  year = {2018},
  doi = {10.18653/v1/w18-6006},
  url = {https://doi.org/10.18653/v1/w18-6006}
}

References

11 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Anders Björkelund, Özlem Çetinoğlu, Agnieszka Faleńska, Richárd Farkas, Thomas Müller, Wolfgang Seeker, and Zsolt Szántó. 2014. Self-training for Swedish Dependency Parsing – Initial Results and Analysis. In Proceedings of the Fifth Swedish Language Technology Conference (SLTC 2014). find
2. Elizabeth Coppock. 2001. find
3. Joakim Nivre, Marie-Catherine de Marneffe, Filip Ginter, Yoav Goldberg, Jan Hajič, Christopher Manning, Ryan McDonald, Slav Petrov, Sampo Pyysalo, Natalia Silveira, Reut Tsarfaty, and Daniel Zeman. 2016. Universal Dependencies v1: A Multilingual Treebank Collection. In Proceedings of the 10th International Conference on Language Resources and Gapping: In defense of Evaluation (LREC 2016), pages 1659–1666, Portorož, Slovenia. Task: Multilingual Parsing from Raw Text to Universal Dependencies, pages 1–19, Stroudsburg, PA, USA. Charles University, Association for Computational Linguistics. find our card
4. Ivan Sag. 1976. Deletion and Logical Form. MIT. PhD dissertation. find
5. Zhi-Hua Zhou and Ming Li. 2005. Tri-training: Exploiting unlabeled data using three classifiers. IEEE Transactions on knowledge and Data Engineering, 17(11):1529–1541. find
6. Kenji Sagae and Junichi Tsujii. 2007. Dependency parsing and domain adaptation with LR models and parser ensembles. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL). find
7. Sebastian Schuster, Joakim Nivre, and Christopher D. Manning. 2018. Sentences with Gapping: Parsing and Reconstructing Elided Predicates. In Proceedings of the 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL 2018). find our card
8. Anders Søgaard and Christian Rishøj. 2010. Semi-supervised dependency parsing using generalized tri-training. In Proceedings of the 23rd International Conference on Computational Linguistics, pages 1065–1073. Association for Computational Linguistics. find
9. Milan Straka and Jana Straková. 2017. Tokenizing, POS Tagging, Lemmatizing and Parsing UD 2.0 with UDPipe. In Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pages 88–99, Vancouver, Canada. Association for Computational Linguistics. find our card
10. David Weiss, Chris Alberti, Michael Collins, and Slav Petrov. 2015. Structured Training for Neural Network Transition-Based Parsing. In Proceedings of ACL 2015, pages 323–333. Daniel Zeman, Martin Popel, Milan Straka, Jan Hajič, Joakim Nivre, Filip Ginter, Juhani Luotolahti, Sampo Pyysalo, Slav Petrov, Martin Potthast, Francis Tyers, Elena Badmaeva, Memduh Gökrmak, Anna Nedoluzhko, Silvie Cinková, jr. find
11. Jan Hajič, Jaroslava Hlaváčová, Václava Kettnerová, Zdeňka Urešová, Jenna Kanerva, Stina Ojala, Anna Missilä, Christopher Manning, Sebastian Schuster, Siva Reddy, Dima Taji, Nizar Habash, Herman Leung, Marie-Catherine de Marneffe, Manuela Sanguinetti, Maria Simi, Hiroshi Kanayama, Valeria de Paiva, Kira Droganova, Héctor Martı́nez Alonso, Çağr Çöltekin, Umut Sulubacak, Hans Uszkoreit, Vivien Macketanz, Aljoscha Burchardt, Kim Harris, Katrin Marheinecke, Georg Rehm, Tolga Kayadelen, Mohammed Attia, Ali Elkahky, Zhuoran Yu, Emily Pitler, Saran Lertpradit, Michael Mandl, Jesse Kirchner, Hector Fernandez Alcalde, Jana Strnadová, Esha Banerjee, Ruli Manurung, Antonio Stella, Atsuko Shimada, Sookyoung Kwak, Gustavo Mendonça, Tatiana Lando, Rattima Nitisaroj, and Josie Li. 2017. CoNLL 2017 shared task: Multilingual parsing from raw text to universal dependencies. In Proceedings of the CoNLL 2017 Shared find our card

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/ud-2018-droganova-mind-gap-data-enrichment-dependency/