Universal Decompositional Semantics on Universal Dependencies

What it is about

The Decomp project adds semantic annotation on top of UD treebanks by asking ordinary speakers simple, decomposed questions with Likert-scale answers instead of asking for labels like Agent. PredPatt, a deterministic, unlexicalised rule system over UD trees, extracts predicate-argument structures (also from appositives, possessives, copulas; resolving arguments of coordinated predicates, relative clauses and embedded clauses); its manual precision on English gold trees is 86%. The semantic proto-role protocol (SPR2), after Dowty, asks per argument how likely it is that it caused the event, chose to be involved, was aware, was sentient, changed location, existed before/during/after, changed possession, changed state, was used, benefited, was only partly involved. 3,806 arguments of 2,759 predicates in EWT were annotated twice (mean pairwise Spearman 0.62). An event protocol asks whether a word names an event and whether, by the author, it happened (raw agreement 0.82); a word-sense protocol decomposes noun senses.

Ideas

• I1. A realis flag on every frame: did the event happen according to the text? Mova has neg and cond; add «not realised» for the complement of want/plan/try/hope/refuse/promise, for future and modal frames, and for imperatives, so the student does not answer «Did the fox get the cheese?» yes from «the fox wanted to get the cheese». A rule over the frame's head verb class (VerbNet desire/promise classes) in prag/src/frames.rs; measure yes/no course questions and with world knowledge diagnose, with a minimal-pair test (wanted to X / did X).
• I2. Derive proto-role properties for each frame argument from what Mova already knows instead of a single label: sentient (animacy level 1), changed possession/location/state (the VerbNet semantic predicates in prag/data/verbnet.tsv, see brown-2019-verbnet-representations-subevent-semantics-transfer I1), instigated (subject of a non-unaccusative frame). Questions like «Who was hurt?», «What changed?», «Who was moved?» can then match a property rather than a role name. Count course questions of these shapes first.
• I3. Check Mova's frames against PredPatt's argument-resolution cases as a test list: relative-clause arguments («The books John borrowed are overdue» → books is the patient of borrowed and the subject of overdue), coordinated predicates with control («likes to sing and dance»), possessive and appositive relations. Each case a reader test with the expected frames.

More ideas (a second reading)

• I4. prag predicate extraction: a PredPatt-like rule module over the en UD trees, producing predicate names and argument spans with the paper's filters; measure precision on 100 extractions from EWT test judged by hand, and the number of arguments found per verb.
• I5. en/src/ctx.rs and prag: recover missing subjects of coordinated verbs and of relative clauses by argument resolution; measure the share of verbs with no subject in EWT test before and after, and the precision of the recovered subjects.
• I6. Proto-role rules in prag for volition, sentience and change of state, computed from UD relations and the verb lemma; measure agreement with a set of 100 arguments that we judge by hand, reported as Spearman, the same statistic the paper uses.
• I7. Event realis in prag: a rule for whether an event happened, from tense, negation, modals (aux, neg) and the sentence type; measure accuracy against 200 predicates we label by hand, with the paper's 0.82 agreement as a rough ceiling.

For Mova

28. Voice and the passive, 25-relative-clauses.md, 13-verbs.md (mood and modality), 30-pragmatics-overview.md §5. Code: prag/src/frames.rs (realis, relative-clause roles), prag/src/vn.rs, world/src/knowledge.rs. Measures: yes/no course questions right, diagnosis step shares, frame bench role F1, reader tests.

Topics: Universal Dependencies · Semantic role labeling

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/d16-1177

@inproceedings{white2016universal,
  title = {{Universal Decompositional Semantics on Universal Dependencies}},
  author = {White, Aaron Steven and Reisinger, Drew and Sakaguchi, Keisuke and Vieira, Tim and Zhang, Sheng and Rudinger, Rachel and Rawlins, Kyle and Durme, Benjamin Van},
  booktitle = {Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing},
  year = {2016},
  doi = {10.18653/v1/d16-1177},
  url = {https://doi.org/10.18653/v1/d16-1177}
}

References

31 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ann Bies, Justin Mott, Colin Warner, and Seth Kulick. English web treebank. Linguistic Data Consortium, Philadelphia, PA, 2012. find
2. Johan Bos, Valerio Basile, Kilian Evang, Noortje Venhuizen, and Johannes Bjerva. The Groningen Meaning Bank. In Nancy Ide and James Pustejovsky, editors, Handbook of Linguistic Annotation. Springer, Berlin, 2017. find
3. Marie-Catherine de Marneffe, Christopher D. Manning, and Christopher Potts. Did it happen? The pragmatic complexity of veridicality assessment. Computational Linguistics, 38(2):301–333, 2012. find
4. Marie-Catherine De Marneffe, Timothy Dozat, Natalia Silveira, Katri Haverinen, Filip Ginter, Joakim Nivre, and Christopher D. Manning. Universal Stanford dependencies: A cross-linguistic typology. In Proceedings of LREC, volume 14, pages 4585–4592, 2014. find our card
5. Luciano Del Corro and Rainer Gemulla. Clausie: clause-based open information extraction. In Proceedings of the 22nd international conference on World Wide Web, pages 355–366, 2013. find
6. David Dowty. Word Meaning and Montague Grammar. D. Reidel Publishing Company, 1979. find
7. David Dowty. Thematic proto-roles and argument selection. Language, 67(3):547–619, 1991. find
8. Pablo Gamallo, Marcos Garcia, and Santiago Fernández-Lanza. Dependency-based open information extraction. In Proceedings of the Joint Workshop on Unsupervised and Semi-Supervised Learning in NLP, pages 10–18, 2012. find
9. Stephan Greene and Philip Resnik. More than words: Syntactic packaging and implicit sentiment. In Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 503–511. Association for Computational Linguistics, 2009. ISBN 1-932432-41-8. find
10. Joshua K. Hartshorne, Claire Bonial, and Martha Palmer. The VerbCorner Project: Toward an Empirically-Based Semantic Decomposition of Verbs. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1438–1442, 2013. find
11. Luheng He, Mike Lewis, and Luke Zettlemoyer. Question-Answer Driven Semantic Role Labeling: Using Natural Language to Annotate Natural Language. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 643–653, 2015. find our card
12. Ray S. Jackendoff. Semantic Structures. MIT Press, 1990. find
13. Edward Kako. Thematic role properties of subjects and objects. Cognition, 101(1):1–42, 2006. find
14. Adam Kilgarriff. I don’t believe in word senses. Computers and the Humanities, 31(2):91–113, 1997. find our card
15. Paul Kiparsky. Partitive case and aspect. The Projection of Arguments: Lexical and compositional factors, 265:307, 1998. find
16. Manfred Krifka. Nominal reference, temporal constitution, and quantification in event semantics. In R. Bartsch, J. van Benthem, and P. von Emde Boas, editors, Semantics and Contextual Expression. Foris Publications, 1989. find
17. Beth Levin and Malka Rappaport Hovav. Argument realization. Cambridge University Press, 2005. find
18. Nitin Madnani, Jordan Boyd-Graber, and Philip Resnik. Measuring transitivity using untrained annotators. In Proceedings of the NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazon’s Mechanical Turk, pages 188–194, 2010. find
19. Mitchell P. Marcus, Mary Ann Marcinkiewicz, and Beatrice Santorini. Building a Large Annotated Corpus of English: The Penn Treebank. Computational Linguistics, 19(2): 313–330, June 1993. ISSN 0891-2017. URL http://dl.acm.org/citation.cfm? id=972470.972475. ↗ our card
20. George A. Miller. Wordnet: A lexical database for english. Commun. ACM, 38(11):39–41, November 1995. ISSN 0001-0782. doi: 10.1145/219717. 219748. URL http://doi.acm.org/10. 1145/219717.219748. ↗ our card
21. Roberto Navigli. Meaningful clustering of senses helps boost word sense disambiguation performance. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, pages 105–112, Sydney, Australia, July 2006. Association for Computational Linguistics. doi: 10.3115/1220175. 1220189. URL http://www.aclweb.org/ anthology/P06-1014. ↗
22. Joakim Nivre, Željko Agić, Maria Jesus Aranzabe, Masayuki Asahara, Aitziber Atutxa, Miguel Ballesteros, John Bauer, Kepa Bengoetxea, Riyaz Ahmad Bhat, Cristina Bosco, Sam Bowman, Giuseppe G. A. Celano, Miriam Connor, Marie-Catherine de Marneffe, Arantza Diaz de Ilarraza, Kaja Dobrovoljc, Timothy Dozat, Tomaž Erjavec, Richárd Farkas, Jennifer Foster, Daniel Galbraith, Filip Ginter, Iakes Goenaga, Koldo Gojenola, Yoav Goldberg, Berta Gonzales, Bruno Guillaume, Jan Hajič, Dag Haug, Radu Ion, Elena Irimia, Anders Johannsen, Hiroshi Kanayama, Jenna Kanerva, Simon Krek, Veronika Laippala, Alessandro Lenci, Nikola Ljubešić, Teresa Lynn, Christopher Manning, Cătălina Mărănduc, David Mareček, Héctor Martı́nez Alonso, Jan Mašek, Yuji Matsumoto, Ryan McDonald, Anna Missilä, Verginica Mititelu, Yusuke Miyao, Simonetta Montemagni, Shunsuke Mori, Hanna Nurmi, Petya Osenova, Lilja Øvrelid, Elena Pascual, Marco Passarotti, Cenel-Augusto Perez, Slav Petrov, Jussi Piitulainen, Barbara Plank, Martin Popel, Prokopis Prokopidis, Sampo Pyysalo, Loganathan Ramasamy, Rudolf Rosa, Shadi Saleh, Sebastian Schuster, Wolfgang Seeker, Mojgan Seraji, Natalia Silveira, Maria Simi, Radu Simionescu, Katalin Simkó, Kiril Simov, Aaron Smith, Jan Štěpánek, Alane Suhr, Zsolt Szántó, Takaaki Tanaka, Reut Tsarfaty, Sumire Uematsu, Larraitz Uria, Viktor Varga, Veronika Vincze, Zdeněk Žabokrtský, Daniel Zeman, and Hanzhi Zhu. Universal Dependencies 1.2. http://universaldependencies.github.io/docs/, November 2015. URL https://lindat. mff.cuni.cz/repository/xmlui/ handle/11234/1-1548. ↗
23. Martha Palmer, Daniel Gildea, and Paul Kingsbury. The proposition bank: An annotated corpus of semantic roles. Computational Linguistics, 31(1): 71–106, 2005. find our card
24. James Pustejovsky. The generative lexicon. Computational Linguistics, 17(4):409–441, 1991. find
25. M. Rappaport-Hovav and Beth Levin. Building verb meanings. In M. Butts and W. Geuder, editors, The Projection of Arguments: Lexical and compositional factors, pages 97–134. CSLI Publications, 1998. find
26. Drew Reisinger, Rachel Rudinger, Francis Ferraro, Craig Harman, Kyle Rawlins, and Benjamin Van Durme. Semantic Proto-Roles. Transactions of the Association for Computational Linguistics, 3:475–488, 2015. find
27. Rachel Rudinger and Benjamin Van Durme. Is the stanford dependency representation semantic? In ACL Workshop: EVENTS, 2014. find
28. Roser Saurı́ and James Pustejovsky. Are you sure that this happened? assessing the factuality degree of events in text. Computational Linguistics, 38(2):261–299, 2012. find
29. Rion Snow, Brendan O’Connor, Daniel Jurafsky, and Andrew Ng. Cheap and fast – but is it good? Evaluating non-expert annotations for natural language tasks. In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pages 254–263, Honolulu, Hawaii, October 2008. Association for Computational Linguistics. URL http://www.aclweb.org/anthology/D08-1027. ↗
30. Michael Sussna. Word sense disambiguation for free-text indexing using a massive semantic network. In Proceedings of the Second International Conference on Information and Knowledge Management, CIKM ’93, pages 67–74, New York, NY, USA, 1993. find
31. Carol Lee Tenny. Grammaticalizing aspect and affectedness. Thesis, Massachusetts Institute of Technology, 1987. URL http://dspace.mit.edu/handle/1721.1/14704. ↗

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/white-2016-universal-decompositional-semantics-universal-dependencies/