Comparison by Conversion: Reverse-Engineering UCCA from Syntax and Lexical Semantics
What it is about
The paper asks how much of UCCA can be recovered from simpler layers: Universal Dependencies syntax and STREUSLE lexical semantics. Two methods are used on the English Web Treebank reviews section. The first is a hand-written rule converter from UD and STREUSLE to UCCA. The second is a supervised parser that gets only syntax and lexical-semantic features and no words, so it is delexicalised. Both recover the UCCA structure with accuracy close to a fully supervised UCCA parser, which suggests that UCCA is partly redundant with these layers. The paper then lists where they diverge. UCCA separates the relation between nouns in compounds, the adverb and linker uses of adverbs, and the scene-evoking status of nouns, possessives and adjectives. Adding gold lexical-semantic features improves the delexicalised parser.
Ideas
- I1. Rules as the explicit standard. Write the rules from UD plus
pragroles that map a sentence to meta-world scenes, and measure how much of the meta-world they recover. Thetools/roundtripstack already has some of these steps, so the count is cheap to take. - I2. How much is lexical? Train the encoder with syntax features only, with no concept features, and measure how far the meta-world is still recovered. The paper's result suggests that most of it is recoverable. For Mova, the question is which tokens still need a concept.
- I3. Known divergence classes. Compound relations and scene-evoking nouns are named gaps in this paper. Add them as residual classes in
tools/translate/src/measure.rs, so that they are counted separately from lexical errors.
For Mova
The paper is the clearest case that a syntax-based converter is a fair baseline before any learned step. The rule and delexicalised tests (I1 and I2) fit tools/roundtrip, and the divergence list (I3) fits the residual classes. The paper bears on open problem 3, the encoder writing trees, and on open problem 4, the function words that carry scene structure.
Cite
@inproceedings{hershcovich2020comparison,
title = {{Comparison by Conversion: Reverse-Engineering UCCA from Syntax and Lexical Semantics}},
author = {Hershcovich, Daniel and Schneider, Nathan and Dvir, Dotan and Prange, Jakob and Lhoneux, Miryam de and Abend, Omri},
booktitle = {Proceedings of the 28th International Conference on Computational Linguistics},
year = {2020},
doi = {10.18653/v1/2020.coling-main.264},
url = {https://doi.org/10.18653/v1/2020.coling-main.264}
}References
- Omri Abend and Ari Rappoport. 2013. Universal Conceptual Cognitive Annotation (UCCA). In ACL, pages 228–238. find our card
- Omri Abend and Ari Rappoport. 2017. The state of the art in semantic representation. In ACL, pages 77–89. E.g., a teacher is a person who teaches, and a friend is a person in a friendship relation with another person. find
- Lasha Abzianidze, Johannes Bjerva, Kilian Evang, Hessel Haagsma, Rik van Noord, Pierre Ludmann, Duc-Duy Nguyen, and Johan Bos. 2017. The parallel meaning bank: Towards a multilingual corpus of translations annotated with compositional meaning representations. In EACL, pages 242–247. find our card
- Daniel Zeman et al. 2020. Universal Dependencies 2.6. LINDAT/CLARIAH-CZ digital library at the Institute of Formal and Applied Linguistics (ÚFAL), Faculty of Mathematics and Physics, Charles University. find
- Collin F. Baker, Charles J. Fillmore, and John B. Lowe. 1998. The Berkeley FrameNet project. In ACL-COLING. find our card
- Laura Banarescu, Claire Bonial, Shu Cai, Madalina Georgescu, Kira Griffitt, Ulf Hermjakob, Kevin Knight, Martha Palmer, and Nathan Schneider. 2013. Abstract Meaning Representation for sembanking. In Linguistic Annotation Workshop. find our card
- Ann Bies, Justin Mott, Colin Warner, and Seth Kulick. 2012. English Web Treebank. Technical Report LDC2012T13, Linguistic Data Consortium, Philadelphia, PA. find
- Alexandra Birch, Omri Abend, Ondřej Bojar, and Barry Haddow. 2016. HUME: Human UCCA-based evaluation of machine translation. In EMNLP, pages 1264–1274. find our card
- Alena Böhmová, Jan Hajič, Eva Hajičová, and Barbora Hladká. 2003. The Prague Dependency Treebank: A three-level annotation scenario. In Anne Abeillé, editor, Treebanks: Building and Using Parsed Corpora, Text, Speech and Language Technology, pages 103–127. Springer, Dordrecht. find
- Wanxiang Che, Longxu Dou, Yang Xu, Yuxuan Wang, Yijia Liu, and Ting Liu. 2019. HIT-SCIR at MRP 2019: A unified pipeline for meaning representation parsing via efficient training and effective encoding. In CoNLL, pages 76–85. find
- Leshem Choshen and Omri Abend. 2018. Reference-less measure of faithfulness for grammatical error correction. In NAACL-HLT. find
- Ruixiang Cui and Daniel Hershcovich. 2020. Refining implicit argument annotation for UCCA. In DMR. find
- Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In NAACL-HLT, pages 4171–4186. find
- Robert M. W. Dixon. 2010, 2012. Basic Linguistic Theory. Oxford University Press. find
- Charles J. Fillmore and Collin Baker. 2009. A frames approach to semantic analysis. In Bernd Heine and Heiko Narrog, editors, The Oxford Handbook of Linguistic Analysis, pages 791–816. Oxford University Press. find
- Oliver Hellwig. 2017. Coarse semantic classification of rare nouns using cross-lingual data and recurrent neural networks. In IWCS. find
- Daniel Hershcovich, Omri Abend, and Ari Rappoport. 2017. A transition-based directed acyclic graph parser for UCCA. In ACL, pages 1127–1138. find
- Daniel Hershcovich, Omri Abend, and Ari Rappoport. 2018. Multitask parsing across semantic representations. In ACL, pages 373–385. find
- Daniel Hershcovich, Omri Abend, and Ari Rappoport. 2019. Content differences in syntactic and semantic representation. In NAACL-HLT, pages 478–488. find
- Jena D. Hwang, Archna Bhatia, Na-Rae Han, Tim O’Gorman, Vivek Srikumar, and Nathan Schneider. 2017. Double trouble: the problem of construal in semantic annotation of adpositions. In *SEM, pages 178–188. find
- Jena D. Hwang, Hanwool Choe, Na-Rae Han, and Nathan Schneider. 2020. K-SNACS: Annotating Korean adposition semantics. In DMR. find
- Nelson F. Liu, Daniel Hershcovich, Michael Kranzlein, and Nathan Schneider. 2020. Lexical semantic recognition. arXiv:2004.15008 [cs]. ↗
- Héctor Martínez Alonso, Anders Johannsen, Sussi Olsen, Sanni Nimb, Nicolai Hartvig Sørensen, Anna Braasch, Anders Søgaard, and Bolette Sandford Pedersen. 2015. Supersense tagging for Danish. In NODALIDA, pages 21–29. find
- Adam Meyers, Ruth Reeves, Catherine Macleod, Rachel Szekely, Veronika Zielinska, Brian Young, and Ralph Grishman. 2004. The NomBank project: an interim report. In Frontiers in Corpus Annotation Workshop, pages 24–31. find
- Edward Newell and Jackie Chi Kit Cheung. 2018. Constructing a lexicon of relational nouns. In LREC, pages 3405–3410. find
- Joakim Nivre, Marie-Catherine de Marneffe, Filip Ginter, Jan Hajič, Christopher D. Manning, Sampo Pyysalo, Sebastian Schuster, Francis Tyers, and Daniel Zeman. 2020. Universal Dependencies v2: An evergrowing multilingual treebank collection. In LREC. find our card
- Joakim Nivre, Marie-Catherine de Marneffe, Filip Ginter, Yoav Goldberg, Jan Hajič, Christopher D. Manning, Ryan McDonald, Slav Petrov, Sampo Pyysalo, Natalia Silveira, Reut Tsarfaty, and Daniel Zeman. 2016. Universal Dependencies v1: A multilingual treebank collection. In LREC, pages 1659–1666. find our card
- Stephan Oepen, Omri Abend, Jan Hajič, Daniel Hershcovich, Marco Kuhlmann, Tim O’Gorman, Nianwen Xue, Jayeol Chun, Milan Straka, and Zdeňka Urešová. 2019. MRP 2019: Cross-framework Meaning Representation Parsing. In CoNLL, pages 1 – 27. find
- Martha Palmer, Daniel Gildea, and Paul Kingsbury. 2005. The proposition bank: An annotated corpus of semantic roles. Computational Linguistics, 31(1). find our card
- Siyao Peng, Yang Liu, Yilun Zhu, Austin Blodgett, Yushi Zhao, and Nathan Schneider. 2020. A corpus of adpositional supersenses for Mandarin Chinese. In LREC. find
- Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. GloVe: Global vectors for word representation. In EMNLP, pages 1532–1543. find
- Davide Picca, Alfio Massimiliano Gliozzo, and Massimiliano Ciaramita. 2008. Supersense Tagger for Italian. In LREC, pages 2386–2390. find
- Jakob Prange, Nathan Schneider, and Omri Abend. 2019. Made for each other: Broad-coverage semantic structures meet preposition supersenses. In CoNLL. find
- Likun Qiu, Yunfang Wu, Yanqiu Shao, and Alexander Gelbukh. 2011. Combining contextual and structural information for supersense tagging of Chinese unknown words. In Computational Linguistics and Intelligent Text Processing: Proceedings of the 12th International Conference (CICLing’11), volume 6608 of Lecture Notes in Computer Science, pages 15–28. Springer, Berlin. find
- Carlos Ramisch, Silvio Ricardo Cordeiro, Agata Savary, Veronika Vincze, Verginica Barbu Mititelu, Archna Bhatia, Maja Buljan, Marie Candito, Polona Gantar, Voula Giouli, Tunga Güngör, Abdelati Hawwari, Uxoa Iñurrieta, Jolanta Kovalevskaiṫe, Simon Krek, Timm Lichte, Chaya Liebeskind, Johanna Monti, Carla Parra Escartín, Behrang QasemiZadeh, Renata Ramisch, Nathan Schneider, Ivelina Stoyanova, Ashwini Vaidya, and Abigail Walsh. 2018. Edition 1.1 of the PARSEME Shared Task on Automatic Identification of Verbal Multiword Expressions. In LAW-MWE-CxG-2018. find
- Nathan Schneider, Jena D. Hwang, Vivek Srikumar, Jakob Prange, Austin Blodgett, Sarah R. Moeller, Aviram Stern, Adi Bitan, and Omri Abend. 2018. Comprehensive supersense disambiguation of English prepositions and possessives. In ACL, pages 185–196. find
- Nathan Schneider, Behrang Mohit, Chris Dyer, Kemal Oflazer, and Noah A. Smith. 2013. Supersense tagging for Arabic: the MT-in-the-middle attack. In NAACL-HLT, pages 661–667. find
- Nathan Schneider and Noah A. Smith. 2015. A corpus and model integrating multiword expressions and supersenses. In NAACL-HLT, pages 1537–1547. find our card
- Karin Kipper Schuler. 2005. VerbNet: A broad-coverage, comprehensive verb lexicon. Ph.D. thesis, University of Pennsylvania. find
- Vered Shwartz and Ido Dagan. 2019. Still a pain in the neck: Evaluating text representations on lexical composition. Transactions of the Association for Computational Linguistics, 7:403–419. find
- Elior Sulem, Omri Abend, and Ari Rappoport. 2015. Conceptual annotations preserve structure across translations: A French-English case study. In S2MT, pages 11–22. find
- Elior Sulem, Omri Abend, and Ari Rappoport. 2018a. Semantic structural annotation for text simplification. In NAACL, pages 685–696. find
- Elior Sulem, Omri Abend, and Ari Rappoport. 2018b. Simple and effective text simplification using semantic and neural methods. In ACL, pages 162–173. find our card
- Yulia Tsvetkov, Nathan Schneider, Dirk Hovy, Archna Bhatia, Manaal Faruqui, and Chris Dyer. 2014. Augmenting English adjective senses with supersenses. In LREC, pages 4359–4365. find
- Abigail Walsh, Claire Bonial, Kristina Geeraert, John P. McCrae, Nathan Schneider, and Clarissa Somers. 2018. Constructing an annotated corpus of verbal MWEs for English. In LAW-MWE-CxG-2018, pages 193–200. find
- Aaron Steven White, Drew Reisinger, Keisuke Sakaguchi, Tim Vieira, Sheng Zhang, Rachel Rudinger, Kyle Rawlins, and Benjamin Van Durme. 2016. Universal decompositional semantics on Universal Dependencies. In EMNLP, pages 1713–1723. find our card
- Sheng Zhang, Xutai Ma, Rachel Rudinger, Kevin Duh, and Benjamin Van Durme. 2018. Cross-lingual decompositional semantic parsing. In EMNLP, pages 1664–1675. find
This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/hershcovich-2020-comparison-conversion-reverse-engineering-ucca/