All Roads Lead to UD: Converting Stanford and Penn Parses to English Universal Dependencies with Multilayer Annotations

Summary

The paper examines how to move an existing English treebank from older annotation schemes to UD 2.2, using the multilayer GUM corpus (eight genres, with entity, coreference, information-status and RST discourse layers). Two sources are compared: gold Stanford Typed Dependencies and Penn-style constituency trees. The Stanford-to-UD conversion is rule-based (in the DepEdit tool) and is enriched with information from the other annotation layers, such as entity types, to make distinctions like compound versus flat or to identify dislocated elements. Using syntax alone the rules err on roughly 1.5% of tokens; with the extra layers the error rate drops to about 0.4%. Converting constituency trees through CoreNLP yields about 10% errors even from gold trees, mainly because phrase functions are underspecified. The work shows that multilayer corpora make it cheap to keep treebanks in step with new UD guideline versions, and it produced a new open English UD resource.

How Mova uses it

• Coordination: the head is the first conjunct, cc is on the next one — cited for the observation that moving cc onto the following conjunct is the main source of non-projectivity in GUM after conversion; the seed's rules check that conj points rightwards from the first conjunct and that cc precedes its head.
• Gapping: orphan links the remnants of an omitted predicate — cited for the point that orphan cannot be reliably derived by conversion; the rule requires the head of orphan to be a promoted remnant (conj, root, advcl or parataxis).
• obl under a noun — suspected nmod — cited for obl/nmod confusion being a leading cause of constituency-conversion errors; the rule flags obl under a noun or pronoun without a copula.

Effectiveness in Mova

Not measured separately. The paper is background evidence for three annotation-error rules. The rules are validated on gold treebanks (for example, the obl-on-nominal check fires 243 times on EWT 2.18 with 15 violations, mostly letter headers; the orphan check has 0 violations out of 29 in EWT), but these counts measure the rules, not the paper's contribution.

👨‍🔬💥

More ideas (a second reading)

• I1. en/src/convert.rs: write the English converter as ordered rules over en dependency trees, not constituents, and report head and label error rates per rule against a hand-checked gold sample. Punctuation is left out of the attachment score, as in the paper.
• I2. Entity-type feature: the worst genres for the rule set are news and biographies, where multi-token names need entity types. Test whether the entity spans in en/src/ctx.rs reduce attachment errors on proper-name tokens on EWT test.
• I3. Conjunction check: after conversion, count non-projective arcs per 1,000 tokens with the en convert-check command, before and after the rule that attaches cc to the conjunct. Keep the rule only if the count falls on EWT test.

Topics: Universal Dependencies · Treebanks

Cite

The paper itself, not this card. BibTeX from the library's record of the work. arXiv 1909.00522

@inproceedings{peng2018all,
  title = {{All Roads Lead to UD: Converting Stanford and Penn Parses to English Universal Dependencies with Multilayer Annotations}},
  author = {Peng, Siyao and Zeldes, Amir},
  booktitle = {Proceedings of the Joint Workshop on Linguistic Annotation, Multiword Expressions and Constructions (LAW-MWE-CxG-2018)},
  year = {2018},
  eprint = {1909.00522},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1909.00522}
}

References

16 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ann Bies, Mark Ferguson, Karen Katz, and Robert MacIntyre. 1995. Bracketing guidelines for Treebank II style. Penn Treebank Project. CIS Technical Report MS-CIS-95-06, University of Pennsylvania. find
2. Ann Bies, Justin Mott, Colin Warner, and Seth Kulick. 2012. English Web Treebank. LDC2012T13, Linguistic Data Consortium, Philadelphia, PA. find
3. Jinho D. Choi and Martha Palmer. 2010. Robust constituent-to-dependency conversion for English. In Proceedings of the 9th International Workshop on Treebanks and Linguistic Theories (TLT 2010), pages 55–66, Tartu, Estonia. find
4. Marie-Catherine de Marneffe and Christopher D. Manning. 2013. Stanford typed dependencies manual. Technical report, Stanford University. find
5. Stefanie Dipper, Michael Götze, and Stavros Skopeteas. 2007. Information structure in cross-linguistic corpora: Annotation guidelines for phonology, morphology, syntax, semantics, and information structure. Interdisciplinary Studies on Information Structure, 7. find
6. Roger Garside and Nicholas Smith. 1997. A hybrid grammatical tagger: CLAWS4. In Roger Garside, Geoffrey Leech, and Anthony McEnery, editors, Corpus Annotation: Linguistic Information from Computer Text Corpora, pages 102–121. Longman, London. find
7. Geoffrey Leech, Tony McEnery, and Martin Weisser. 2003. SPAAC speech-act annotation scheme. Technical report, Lancaster University. find
8. William C. Mann and Sandra A. Thompson. 1988. Rhetorical Structure Theory: Toward a functional theory of text organization. Text, 8(3):243–281. ↗ our card
9. Christopher D. Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven J. Bethard, and Davide McClosky. 2014. The Stanford CoreNLP natural language processing toolkit. In Proceedings of ACL 2014: System Demonstrations, pages 55–60, Baltimore, MD. Joakim Nivre, Željko Agić, Lars Ahrenberg, Maria Jesus Aranzabe, Masayuki Asahara, Aitziber Atutxa, Miguel Ballesteros, John Bauer, Kepa Bengoetxea, Riyaz Ahmad Bhat, Eckhard Bick, Cristina Bosco, Gosse Bouma, Sam Bowman, Marie Candito, Gülşen Cebiroğlu Eryiğit, Giuseppe G. A. Celano, Fabricio Chalub, Jinho Choi, Çağrı Çöltekin, Miriam Connor, Elizabeth Davidson, Marie-Catherine de Marneffe, Valeria de Paiva, Arantza Diaz de Ilarraza, Kaja Dobrovoljc, Timothy Dozat, Kira Droganova, Puneet Dwivedi, Marhaba Eli, Tomaž Erjavec, Richárd Farkas, Jennifer Foster, Cláudia Freitas, Katarína Gajdošová, Daniel Galbraith, Marcos Garcia, Filip Ginter, Iakes Goenaga, Koldo Gojenola, Memduh Gökırmak, Yoav Goldberg, Xavier Gómez Guinovart, Berta Gonzáles Saavedra, Matias Grioni, Normunds Grūzı̄tis, Bruno Guillaume, Nizar Habash, Jan Hajič, Linh Hà Mỹ, Dag Haug, Barbora Hladká, Petter Hohle, Radu Ion, Elena Irimia, Anders Johannsen, Fredrik Jørgensen, Hüner Kaşıkara, Hiroshi Kanayama, Jenna Kanerva, Natalia Kotsyba, Simon Krek, Veronika Laippala, Lê Hồng, Alessandro Lenci, Nikola Ljubešić, Olga Lyashevskaya, Teresa Lynn, Aibek Makazhanov, Christopher Manning, Cătălina Mărănduc, David Mareček, Héctor Martínez Alonso, André Martins, Jan Mašek, Yuji Matsumoto, Ryan McDonald, Anna Missilä, Verginica Mititelu, Yusuke Miyao, Simonetta Montemagni, Amir More, Shunsuke Mori, Bohdan Moskalevskyi, Kadri Muischnek, Nina Mustafina, Kaili Müürisep, Luong Nguyễn Thị, Huyền Nguyễn Thị Minh, Vitaly Nikolaev, Hanna Nurmi, Stina Ojala, Petya Osenova, Lilja Øvrelid, Elena Pascual, Marco Passarotti, Cenel-Augusto Perez, Guy Perrier, Slav Petrov, Jussi Piitulainen, Barbara Plank, Martin Popel, Lauma Pretkalniņa, Prokopis Prokopidis, Tiina Puolakainen, Sampo Pyysalo, Alexandre Rademaker, Loganathan Ramasamy, Livy Real, Laura Rituma, Rudolf Rosa, Shadi Saleh, Manuela Sanguinetti, Baiba Saulı̄te, Sebastian Schuster, Djamé Seddah, Wolfgang Seeker, Mojgan Seraji, Lena Shakurova, Mo Shen, Dmitry Sichinava, Natalia Silveira, Maria Simi, Radu Simionescu, Katalin Simkó, Mária Šimková, Kiril Simov, Aaron Smith, Alane Suhr, Umut Sulubacak, Zsolt Szántó, Dima Taji, Takaaki Tanaka, Reut Tsarfaty, Francis Tyers, Sumire Uematsu, Larraitz Uria, Gertjan van Noord, Viktor Varga, Veronika Vincze, Jonathan North Washington, Zdeněk Žabokrtský, Amir Zeldes, Daniel Zeman, and Hanzhi Zhu. 2017. Universal dependencies 2.0. Technical report, LINDAT/CLARIN digital library at the Institute of Formal and Applied Linguistics (ÚFAL), Faculty of Mathematics and Physics, Charles University. find
10. Martin Popel, Zdenek Zabokrtský, and Martin Vojtek. 2017. Udapi: Universal API for Universal Dependencies. In Universal Dependencies Workshop at NoDaLiDa 2017, Gothenburg. find our card
11. Beatrice Santorini. 1990. Part-of-speech tagging guidelines for the Penn Treebank project (3rd revision). Technical report, University of Pennsylvania, University of Pennsylvania. find our card
12. Sebastian Schuster and Christopher D. Manning. 2016. Enhanced English Universal Dependencies: An improved representation for natural language understanding tasks. In Proceedings of LREC 2016, pages 2371–2378, Portorož, Slovenia. find our card
13. Sebastian Schuster, Joakim Nivre, and Christopher D. Manning. 2018. Sentences with gapping: Parsing and reconstructing elided predicates. In Proceedings of NAACL 2018, pages 1156–1168, New Orleans, LA. find our card
14. Natalia Silveira, Timothy Dozat, Marie-Catherine de Marneffe, Samuel R. Bowman, Miriam Connor, John Bauery, and Christopher D. Manning. 2014. A gold standard dependency corpus for English. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC-2014), pages 2897–2904, Reykjavik, Iceland. find
15. Amir Zeldes and Dan Simonson. 2016. Different flavors of GUM: Evaluating genre and sentence type effects on multilayer corpus annotation quality. In Proceedings of LAW X The 10th Linguistic Annotation Workshop, pages 68–78, Berlin. find our card
16. Amir Zeldes. 2017. The GUM Corpus: Creating multilayer resources in the classroom. Language Resources and Evaluation, 51(3):581–612. ↗

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/peng-2018-all-roads-lead-ud-converting/