Automatically Selecting the Best Dependency Annotation Design with Dynamic Oracles

What it is about

Several conventions exist for the same dependency structure, for example whether a content word or a function word heads a case marker, a copula, a conjunction or a determiner. The paper gives each training sentence a set of alternative trees, and trains an arc-eager parser with a dynamic oracle that keeps, at each step, only the references it can still predict. The reference that survives training is the convention the parser learns most easily. Over 38 UD v1.3 languages and 266 transformed corpora, the UD convention beats the alternative in 129 conditions and loses in 93, with an average UAS difference of 0.66 points and a largest of 8.1. The selection criterion picks the better convention more often than the predictability and derivation-perplexity baselines.

Ideas

• I1. en/src/parse.rs: train the arc-standard parser on EWT with alternative references for the mova choices (case, cop, mark, cc), keep the surviving reference, and measure cv LAS on the train folds against the UD reference.
• I2. docs/dialect.md: before any training, compute predictability (entropy of the dependent's tag given the head's tag) and derivation perplexity for each dialect choice on EWT train, and check whether their ranking agrees with the cv LAS ranking of I1.
• I3. The paper's Table 2 shows the UD copula choice wins in only 25% of conditions; measure cv LAS and the prag events for copular sentences with the copula as head against the UD choice.

For Mova

The conventions at stake are the function-word choices of 10-word-classes, the copula in 21-clause-structure, and the multiword and determiner cases in 22-phrases. The code is en/src/convert.rs for the conventions, en/src/parse.rs for the training, and The Mova dialect: a UD metadialect, treebanks, and converters for the dialect. Measure first: the two cheap baselines of idea I2, since they cost no training and say whether the search in I1 is worth doing.

Topics: Dependency parsing

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/n18-2064

@inproceedings{wisniewski2018automatically,
  title = {{Automatically Selecting the Best Dependency Annotation Design with Dynamic Oracles}},
  author = {Wisniewski, Guillaume and Lacroix, Ophélie and Yvon, François},
  booktitle = {Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers)},
  year = {2018},
  doi = {10.18653/v1/n18-2064},
  url = {https://doi.org/10.18653/v1/n18-2064}
}

References

17 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Lauriane Aufrant, Guillaume Wisniewski, and François Yvon. 2017. Don’t stop me now! Using global dynamic oracles to correct training biases of transition-based dependency parsers. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume… find
2. Samar Husain and Bhasha Agrawal. 2012. Analyzing parser errors to improve parsing accuracy and to inform tree banking decisions. Linguistic Issues in Language Technology 7. find
3. Ryosuke Kohita, Hiroshi Noji, and Yuji Matsumoto. 2017. Multilingual back-and-forth conversion between content and function head for easy dependency parsing. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers. Associ… find
4. Hal Daumé III and Daniel Marcu. 2005. Learning as search optimization: approximate large margin methods for structured prediction. In ICML ’05: Proceedings of the 22nd international conference on Machine learning. ACM Press, New York, NY, USA, pages 169–176. https://doi.org/http://doi.acm. org/10.1… ↗
5. Joakim Nivre. 2008. Algorithms for deterministic incremental dependency parsing. Comput. Linguist. 34(4):513–553. https://doi.org/10. 1162/coli.07-056-R1-07-027. find our card
6. Miryam de Lhoneux and Joakim Nivre. 2016. Should Have, Would Have, Could Have. Investigating Verb Group Representations for Parsing with Universal Dependencies. In Proceedings of the Workshop on Multilingual and Cross-lingual Methods in NLP. Association for Computational Linguistics, San Diego, Cali… find
7. Joakim Nivre, Željko Agić, Lars Ahrenberg, Maria Jesus Aranzabe, Masayuki Asahara, Aitziber Atutxa, Miguel Ballesteros, John Bauer, Kepa Bengoetxea, Yevgeni Berzak, Riyaz Ahmad Bhat, Cristina Bosco, Gosse Bouma, Sam Bowman, Gülşen Cebirolu Eryiit, Giuseppe G. A. Celano, Çar Çöltekin, Miriam C… find
8. Farkas, Jennifer Foster, Daniel Galbraith, Sebastian Garza, Filip Ginter, Iakes Goenaga, Koldo Gojenola, Memduh Gokirmak, Yoav Goldberg, Xavier Gómez Guinovart, Berta Gonzáles Saavedra, Normunds Grūzītis, Bruno Guillaume, Jan Hajič, Dag Haug, Barbora Hladká, Radu Ion, Elena Irimia, Anders Joha… find
9. Roy Schwartz, Omri Abend, and Ari Rappoport. 2012. Learnability-based syntactic annotation design. In Proceedings of COLING 2012. find
10. The COL-ING 2012 Organizing Committee, Mumbai, India, pages 2405–2422. http://www.aclweb.org/ anthology/C12-1147. ↗
11. Natalia Silveira and Christopher Manning. 2015. Does Universal Dependencies need a parsing representation? An investigation of English. find our card
12. Anders Søgaard and Martin Haulrich. 2010. On the derivation perplexity of treebanks. In Proceedings of Treebanks and Linguistic Theories 9. find
13. Guillaume Wisniewski and Ophélie Lacroix. 2017. A systematic comparison of syntactic representations of dependency parsing. In Proceedings of the NoDaLiDa 2017 Workshop on Universal Dependencies (UDW 2017). Association for Computational Linguistics, Gothenburg, Sweden, pages 146–152. http://www.acl… ↗
14. Yue Zhang and Joakim Nivre. 2011. Transition-based Dependency Parsing with Rich Non-local Features. In Proceedings of ACL 2011, the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, Portland, Oregon, USA, pag… find
15. Tomoya Noro, Chimato Koike, Taiichi Hashimoto, Takenobu Tokunaga, and Hozumi Tanaka. 2005. Evaluation of a japanese cfg derived from a syntactically annotated corpus with respect to dependency measures. In Proceedings of the Fifth Workshop on Asian Language Resources (ALR-05) and First Symposium on… find
16. Timothy Osborne and Daniel Maxwell. 2015. A historical overview of the status of function words in dependency grammar. In Proceedings of the Third International Conference on Dependency Linguistics (Depling 2015). Uppsala University, Uppsala, Sweden, Uppsala, Sweden, pages 241–250. http:// www.aclwe… find
17. Martin Popel, David Mareček, Jan Štpánek, Daniel Zeman, and Zdnk Žabokrtský. 2013. Coordination structures in dependency treebanks. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Sofi… find our card

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/bank-2018-wisniewski-automatically-selecting-best-dependency-annotation/