Annotation Scheme for English Argument Structure Constructions Treebank

What it is about

The authors define nine argument structure constructions (ASCs) for English, such as intransitive simple, ditransitive, caused-motion, transitive resultative, attributive and passive, and tag them by hand in a treebank of 10,204 sentences: 5,936 from the English Web Treebank (L1) and 1,948 written and 2,320 spoken from learner corpora (L2). Each ASC is mapped to a UD syntactic frame, such as nsubj-root-obj or nsubj:pass-aux:pass-root, and to a semantic frame, but annotators may ignore the UD tags when the frame does not fit. Two annotators tagged each token, and the first round reached 85.7% exact agreement with kappa 0.801. The earlier silver ASC treebank is also scored against the new gold, with F1 from 0.723 for intransitive motion to 0.977 for attributive.

Ideas

• I1. Frame-to-ASC rule in en/src/ud.rs: map the UD frame of each clause (Table 1 of the paper) to one of the nine ASC tags as a deterministic Rust rule, and measure exact agreement with the gold ASC tags on the 13,720 EWT tokens (test only, since the gold is under its own licence check).
• I2. Per-tag F1 as a gate for annot: when the teacher labels ASCs, compare its per-tag F1 with the silver figures in this paper (0.723 for intransitive motion, 0.740 for intransitive resultative) and keep only tags above the agreement of the two human annotators.
• I3. Passive detection in en/src/ud.rs: check that the nsubj:pass plus aux:pass pattern gives the PASSIVE tag, and measure the passive precision and recall on EWT against the paper's 0.924 precision and 0.948 recall for the silver method.
• I4. Learner ASCs as a test set: run en on the L2 spoken sentences (ESL-SP) and report LAS separately per ASC class, which shows where learner syntax fails.

For Mova

The frame logic is in en/src/ud.rs and the clause layers feed prag and world. The chapters that the ideas touch are 28. Voice and the passive, 21-clause-structure.md and 23-complementation.md. Measure first: exact ASC agreement on the EWT subset that the paper tagged, using the same nine labels.

Topics: Treebanks

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/2024.law-1.2

@inproceedings{sung2024annotation,
  title = {{Annotation Scheme for English Argument Structure Constructions Treebank}},
  author = {Sung, Hakyung and Kyle, Kristopher},
  booktitle = {Proceedings of the 18th Linguistic Annotation Workshop (LAW-XVIII)},
  year = {2024},
  doi = {10.18653/v1/2024.law-1.2},
  url = {https://doi.org/10.18653/v1/2024.law-1.2}
}

References

44 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Alan Akbik, Laura Chiticariu, Marina Danilevsky, Yunyao Li, Shivakumar Vaithyanathan, and Huaiyu Zhu. 2015. Generating high quality proposition banks for multilingual semantic role labeling. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 397–407. find
2. Adele E Goldberg. 2003. Constructions: A new theoretical approach to language. Trends in cognitive sciences, 7(5):219–224. find
3. Adele E. Goldberg. 2009. Constructions work. Cognitive Linguistics, 20(1):201–224. find
4. Giulia ML Bencini and Adele E Goldberg. 2000. The contribution of argument structure constructions to sentence meaning. Journal of Memory and Language, 43(4):640–651. find
5. Adele E Goldberg. 2013. The emergence of the semantics of argument structure constructions. In The emergence of language, pages 215–230. Psychology Press. find
6. Yevgeni Berzak, Jessica Kenney, Carolyn Spadine, Jing Xian Wang, Lucia Lam, Keiko Sophie Mori, Sebastian Garza, and Boris Katz. 2016. Universal dependencies for learner english. arXiv preprint arXiv:1605.04278. ↗ our card
7. Adele E Goldberg. 2019. Explain me this: Creativity, competition, and the partial productivity of constructions. Princeton University Press. find
8. Stefan Th Gries and Nick C Ellis. 2015. Statistical measures for usage-based linguistics. Language Learning, 65(S1):228–255. find
9. Ann Bies, Justin Mott, Colin Warner, and Seth Kulick. 2012. English web treebank. Linguistic Data Consortium, Philadelphia, PA. find
10. Haerim Hwang and Hyunwoo Kim. 2023. Automatic analysis of constructional diversity as a predictor of efl students’ writing proficiency. Applied Linguistics, 44(1):127–147. find
11. Claire Nicole Bonial. 2014. Take a look at this! Form, function and productivity of English light verb constructions. Ph.D. thesis, University of Colorado at Boulder. find
12. Jena D Hwang. 2014. Identification and representation of caused motion constructions. Ph.D. thesis, University of Colorado at Boulder. find
13. Thea Cameron-Faulkner, Elena Lieven, and Michael Tomasello. 2003. A construction based analysis of child directed speech. Cognitive science, 27(6):843– 873. ↗
14. Jonathan Dunn. 2017. Computational learning of construction grammars. Language and cognition, 9(2):254–292. find
15. Emi Izumi, Kiyotaka Uchimoto, and Hitoshi Isahara. 2004. The nict jle corpus: Exploiting the language learners’ speech database for research and education. International Journal of the Computer, the Internet and Management, 12(2):119–125. find
16. Nick C Ellis. 2002. Frequency effects in language processing: A review with implications for theories of implicit and explicit language acquisition. Studies in second language acquisition, 24(2):143–188. find
17. K. Kyle. 2016. Measuring syntactic development in L2 writing: Fine grained indices of syntactic complexity and usage-based indices of syntactic sophistication. Ph.D. thesis, Georgia State University, Atlanta, GA. find
18. Nick C Ellis and Fernando Ferreira-Junior. 2009. Construction learning as a function of frequency, frequency distribution, and function. The Modern language journal, 93(3):370–385. find
19. K. Kyle and S. Crossley. 2017. Assessing syntactic sophistication in l2 writing: A usage-based approach. Language Testing, 34(4):513–535. find
20. Kristopher Kyle, Scott Crossley, and Marjolijn Verspoor. 2021. Measuring longitudinal writing development using indices of syntactic complexity and sophistication. Studies in Second Language Acquisition, 43(4):781–812. find
21. Nick C Ellis and Diane Larsen-Freeman. 2009. Constructing a second language: Analyses and computational simulations of the emergence of linguistic constructions from usage. Language Learning, 59:90– 125. find
22. Kristopher Kyle, Masaki Eguchi, Aaron Miller, and Theodore Sither. 2022. A dependency treebank of spoken second language english. In Proceedings of the 17th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2022), pages 39–45. find our card
23. Charles J Fillmore. 1968. Lexical entries for verbs. Foundations of language, pages 373–393. find
24. Charles J Fillmore, Christopher R Johnson, and Miriam RL Petruck. 2003. Background to framenet. International journal of lexicography, 16(3):235– 250. find
25. Kristopher Kyle and Hakyung Sung. 2023. An argument structure construction treebank. In Proceedings of the First International Workshop on Construction Grammars and NLP (CxGs+ NLP, GURT/SyntaxFest 2023), pages 51–62. find
26. Kim Gerdes and Sylvain Kahane. 2016. Dependency annotation choices: Assessing theoretical and practical issues of universal dependencies. In LAW X (2016) The 10th Linguistic Annotation Workshop: 131. find
27. J Richard Landis and Gary G Koch. 1977. An application of hierarchical kappa-type statistics in the assessment of majority agreement among multiple observers. Biometrics, pages 363–374. find
28. Adele E Goldberg. 1995. Constructions: A construction grammar approach to argument structure. University of Chicago Press. find
29. Ronald W Langacker. 1987. Nouns and verbs. Language, pages 53–94. find
30. Kevin P Yancey, Geoffrey Laflair, Anthony Verardi, and Jill Burstein. 2023. Rating short l2 essays on the cefr scale with gpt-4. In Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023), pages 576– 584. find
31. Bai Li, Zining Zhu, Guillaume Thomas, Frank Rudzicz, and Yang Xu. 2022. Neural reality of argument structure constructions. arXiv preprint arXiv:2202.12246. ↗
32. Harish Tayyar Madabushi, Laurence Romain, Dagmar Divjak, and Petar Milin. 2020. Cxgbert: Bert meets construction grammar. arXiv preprint arXiv:2011.04134. ↗
33. Helen Yannakoudakis, Ted Briscoe, and Ben Medlock. 2011. A new dataset and method for automatically grading esol texts. In Proceedings of the 49th annual meeting of the association for computational linguistics: human language technologies, pages 180–189. find
34. Harish Tayyar Madabushi, Laurence Romain, Petar Milin, and Dagmar Divjak. 2023. Construction grammar and language models. arXiv preprint arXiv:2308.13315. ↗
35. Anat Ninio. 1999. Pathbreaking verbs in syntactic development and the question of prototypical transitivity. Journal of child language, 26(3):619–653. find
36. Joakim Nivre, Koenraad De Smedt, and Martin Volk. 2004. Treebanking in northern europe: A white paper. Nordisk Sprogteknologi, pages 2000–2004. find
37. Mary Catherine O’Connor and Paul Kay. 2003. Regularity and idiomaticity in grammatical constructions: The case of let alone. In The new psychology of language, pages 243–270. Psychology Press. find
38. Martha Palmer, Daniel Gildea, and Paul Kingsbury. 2005. The proposition bank: An annotated corpus of semantic roles. Computational linguistics, 31(1):71– 106. ↗ our card
39. Martha Palmer, Daniel Gildea, and Nianwen Xue. 2011. Semantic role labeling. Morgan & Claypool Publishers. find
40. Malka Rappaport Hovav and Beth Levin. 1998. Building verb meanings. The projection of arguments: Lexical and compositional factors, pages 97–134. find
41. Karin Kipper Schuler. 2005. VerbNet: A broad-coverage, comprehensive verb lexicon. Ph.D. thesis, University of Pennsylvania. find
42. Natalia Silveira, Timothy Dozat, Marie-Catherine De Marneffe, Samuel R Bowman, Miriam Connor, John Bauer, and Christopher D Manning. 2014. A gold standard dependency corpus for english. In LREC, pages 2897–2904. Citeseer. find
43. Hakyung Sung and Gyu-Ho Shin. 2023. Diversifying language models for lesser-studied languages and language-usage contexts: A case of second language Korean. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 11461– 11473, Singapore. Association for Computational Linguistics. find
44. Leonie Weissweiler, Taiqi He, Naoki Otani, David R Mortensen, Lori Levin, and Hinrich Schütze. 2023. Construction grammar provides unique insight into neural language models. arXiv preprint arXiv:2302.02178. ↗

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/bank-2024-sung-annotation-scheme-english-argument-structure/