Can predicate-argument relationships be extracted from UD trees?

What it is about

The authors test how much predicate-argument structure a UD parse already holds, using the crowd-sourced QA-SRL data (questions like «What did something publish?» with answer spans). For each question they take the verb, collect its dependents (and its parent, for copulas and participles on nouns) in the tree, optionally after adding enhanced-UD arcs by rule (shared dependents of conjuncts, control and raising subjects, relative-clause antecedents), strip prepositions and determiners, and count a hit if a gold span matches. Plain trees give 68.3% (Udify) and 72.2% (Stanza); with the enhancement rules Udify reaches 79.1%. Almost every answer is a subtree somewhere in the parse (98.9%), so the losses come from attachment to the predicate. In 100 analysed errors 49 were faults of the data, 13 of the parser (often quotation marks ending the parse) and 38 of the method: arguments mediated by purpose clauses («urged citizens to receive a flu shot»), fronted time/place phrases that belong to a later participle («In the summer, the glacier melts, producing…»), anaphora, and causes not linked syntactically to the verb.

Ideas

• I1. Propagate a fronted time or place phrase to the dependent clauses of its head: an obl:tmod/time obl or place obl attached to the main verb also holds for a participial advcl/xcomp or a coordinated verb after it («In winter the bears sleep and dream» → both frames get when:winter). A rule in prag/src/frames.rs next to the subject sharing for conj/xcomp/advcl that already exists; measure place and attribute F1 on the frame bench and course questions that ask when/where.
• I2. Measure the frame reader the way the paper measures UD: for each gold role filler on the frame bench, is it a subtree anywhere in the parse (upper bound) and is it attached to the predicate (reachable by the rules)? The two numbers split parser errors from missing reader rules, so effort goes to the larger loss. A small report mode of prag/src/fbench.rs.
• I3. Classify frame-bench errors with the paper's three bins (gold fault, parser fault, reader fault) on a fixed sample of 100, and repeat after each release; quotes breaking the parse were a large share of the paper's parser errors and are a known weak point of Mova's speech reading too (chapter 38 §7.2).

More ideas (a second reading)

• I4. prag roles: use the extraction rules of this paper as the baseline for prag argument roles, and measure the share of argument-predicate pairs found on a QA-SRL-like sample with en output (test only if the dataset is non-commercial).
• I5. en/src/parse.rs with enhanced arcs: a Rust rule for the EUD arcs that the paper needs (propagated subjects, relatives), measured by the change in pair recall against the basic tree.
• I6. Error classes as a treeagree-style check: label each missed pair as dataset, parser or method error (the 100-case protocol) on the en output, to find which component to fix first.

For Mova

23. Complementation (control, purpose clauses), 30-pragmatics-overview.md §4 (from syntax to pragmatics). Code: prag/src/frames.rs (propagation of fronted obliques; the existing «a clause without its own subject shares the head's» rule), prag/src/fbench.rs (upper bound and attachment report). Measures: frame bench place/attr/agent F1, share of gold fillers reachable from the predicate. QA-SRL itself only as a test, after its licence is checked.

Topics: Coreference resolution · Universal Dependencies · Semantic role labeling

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/2021.law-1.5

@inproceedings{ek2021can,
  title = {{Can predicate-argument relationships be extracted from UD trees?}},
  author = {Ek, Adam and Bernardy, Jean-Philippe and Chatzikyriakidis, Stergios},
  booktitle = {Proceedings of the Joint 15th Linguistic Annotation Workshop (LAW) and 3rd Designing Meaning Representations (DMR) Workshop},
  year = {2021},
  doi = {10.18653/v1/2021.law-1.5},
  url = {https://doi.org/10.18653/v1/2021.law-1.5}
}

References

24 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Gabor Angeli, Melvin Jose Johnson Premkumar, and Christopher D. Manning. 2015. Leveraging linguistic structure for open domain information extraction. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 344–354, Beijing, China. Association for Computational Linguistics. find
2. Luheng He, Mike Lewis, and Luke Zettlemoyer. 2015. Question-answer driven semantic role labeling: Using natural language to annotate natural language. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 643–653, Lisbon, Portugal. Association for Computational Linguistics. find our card
3. Gosse Bouma, Djamé Seddah, and Daniel Zeman. 2020. Overview of the iwpt 2020 shared task on parsing into enhanced universal dependencies. In Proceedings of the 16th International Conference on Parsing Technologies and the IWPT 2020 Shared Task on Parsing into Enhanced Universal Dependencies, pages 151–161. find our card
4. Dan Kondratyuk and Milan Straka. 2019. 75 languages, 1 model: Parsing universal dependencies universally. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2779–2795, Hong Kong, China. Association for Computational Linguistics. find our card
5. Gosse Bouma, Djamé Seddah, and Daniel Zeman. 2021. From raw text to enhanced universal dependencies: The parsing shared task at iwpt 2021. In Proceedings of the 17th International Conference on Parsing Technologies (IWPT 2021), pages 146–157. find our card
6. Anshuman Mishra, Dhruvesh Patel, Aparna Vijayaku-mar, Xiang Li, Pavan Kapanipathi, and Kartik Talamadupula. 2020. Reading comprehension as natural language inference:a semantic analysis. In Proceedings of the Ninth Joint Conference on Lexical and Computational Semantics, pages 12–19, Barcelona, Spain (Online). Association for Computational Linguistics. find
7. Ekaterina Buyko and Udo Hahn. 2010. Evaluating the impact of alternative dependency graph encodings on solving event extraction tasks. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 982–992. find
8. Makoto Miwa, Sampo Pyysalo, Tadayoshi Hara, and Jun’ichi Tsujii. 2010a. Evaluating dependency representations for event extraction. In Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010), pages 779–787, Beijing, China. Coling 2010 Organizing Committee. find
9. Marie Candito, Bruno Guillaume, Guy Perrier, and Djamé Seddah. 2017. Enhanced UD dependencies with neutralized diathesis alternation. In Proceedings of the Fourth International Conference on Dependency Linguistics (Depling 2017), pages 42–53, Pisa,Italy. Linköping University Electronic Press. find our card
10. Makoto Miwa, Sampo Pyysalo, Tadayoshi Hara, and Jun’ichi Tsujii. 2010b. A comparative study of syntactic parsers for event extraction. In Proceedings of the 2010 Workshop on Biomedical Natural Language Processing, pages 37–45. find
11. Dorottya Demszky, Kelvin Guu, and Percy Liang. 2018. Transforming question answering datasets into natural language inference datasets. CoRR, abs/1809.02922. ↗
12. Joakim Nivre, Marie-Catherine de Marneffe, Filip Ginter, Yoav Goldberg, Jan Hajič, Christopher D. Manning, Ryan McDonald, Slav Petrov, Sampo Pyysalo, Natalia Silveira, Reut Tsarfaty, and Daniel Zeman. 2016. Universal Dependencies v1: A multilingual treebank collection. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 1659–1666, Portorož, Adam Ek and Jean-Philippe Bernardy. 2020. How much of enhanced UD is contained in UD? In Proceedings of the 16th International Conference on Parsing Technologies and the IWPT 2020 Shared Task on Parsing into Enhanced Universal Dependencies, pages 221–226, Online. Association for Computational Linguistics. find our card
13. Kyle Rawlins, and Benjamin Van Durme. 2016. Universal decompositional semantics on universal dependencies. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1713–1723. Slovenia. European Language Resources Association (ELRA). find our card
14. Timothy Osborne and Kim Gerdes. 2019. The status of function words in dependency grammar: A critique of universal dependencies (ud). Glossa (Online). find
15. Sheng Zhang, Rachel Rudinger, and Benjamin Van Durme. 2017. An evaluation of predpatt and open ie via stage 1 semantic role labeling. In IWCS 2017—12th International Conference on Computational Semantics—Short papers. find
16. Massimo Poesio, Florence Bruneseaux, and Laurent Romary. 1999. The mate meta-scheme for coreference in dialogues in multiple languages. In ACL’99 Workshop Towards Standards and Tools for Discourse Tagging, pages 65–74. find
17. Peng Qi, Yuhao Zhang, Yuhui Zhang, Jason Bolton, and Christopher D. Manning. 2020. Stanza: A Python natural language processing toolkit for many human languages. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations. ↗ our card
18. Siva Reddy, Oscar Täckström, Slav Petrov, Mark Steedman, and Mirella Lapata. 2017. Universal semantic parsing. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 89–101, Copenhagen, Denmark. Association for Computational Linguistics. find
19. Sebastian Schuster, Ranjay Krishna, Angel Chang, Li Fei-Fei, and Christopher D. Manning. 2015. Generating semantically precise scene graphs from textual descriptions for improved image retrieval. In Proceedings of the Fourth Workshop on Vision and Language, pages 70–80, Lisbon, Portugal. Association for Computational Linguistics. find
20. Sebastian Schuster and Christopher D Manning. 2016. Enhanced english universal dependencies: An improved representation for natural language understanding tasks. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 2371–2378. find our card
21. Natalia G Silveira. 2016. Designing syntactic representations for NLP: An empirical investigation. Ph.D. thesis, Stanford University. find
22. Gabriel Stanovsky, Jessica Ficler, Ido Dagan, and Yoav Goldberg. 2016. Getting more out of syntax with props. arXiv preprint arXiv:1603.01648. ↗
23. Mingming Sun, Wenyue Hua, Zoey Liu, Xin Wang, Kangjie Zheng, and Ping Li. 2020. A predicate-function-argument annotation of natural language for open-domain information expression. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2140–2150. find
24. Aryeh Tiktinsky, Yoav Goldberg, and Reut Tsarfaty. 2020. pybart: Evidence-based syntactic transformations for ie. arXiv preprint arXiv:2005.01306. Aaron Steven White, Drew Reisinger, Keisuke Sakaguchi, Tim Vieira, Sheng Zhang, Rachel Rudinger, ↗ our card

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/ek-2021-can-predicate-argument-relationships-be/