Towards the Annotation of Penn TreeBank with Information Structure

What it is about

Information structure (here the thematicity part of Meaning-Text Theory's communicative structure: Specifier, Theme, Rheme, nested) is missing from the big English treebanks. The authors annotate 435 sentences of the CoNLL-2009 Penn Treebank with a manual procedure, using explicit criteria: fronted adjuncts are Specifiers, the subject is usually the Theme, negated or questioned material is Rheme, and relative pronouns in subject position are Themes. They compare a perceptron sequence tagger with a perceptron transition-based bracketing parser. On 35 test sentences (training on 360) the parser reaches 88.67% token accuracy, against 71.74% for the tagger; labelled bracket score 68.95 against 51.78. Errors are mainly consecutive Themes, consecutive Rhemes, and reduced clauses that are not labelled. Accuracy rises with the training size.

Ideas

• I1. 29. Ellipsis, word order and information structure and prag: write the paper's criteria as deterministic rules over UD trees: advmod/obl fronted before the root as Specifier, nsubj as Theme, a negated or wh-questioned clause as Rheme. Measure token accuracy on a hand-checked sample of school sentences.
• I2. Add a shift-reduce bracketing step to the parse in en/src/parse.rs only for the communicative layer, with a rule-based oracle built from I1, so the learning curve can be measured against rules rather than against a learner.
• I3. Learning-curve measure: report accuracy for 50, 100, 200 and 360 training sentences, as the paper does, so the claim that small samples are enough is tested on Mova's data.

For Mova

The chapter is 29. Ellipsis, word order and information structure, with 22. Phrases: noun, adjective, adverb and prepositional phrases for the constituents that count as Specifiers. The code is en/src/lin.rs (word order, where Theme comes first) and prag/src/frames.rs. Measure: token accuracy and bracket score for the rule version on a hand-checked sample, run twice. The paper's 435 sentences are not in the card; any test set must be built for Mova.

Topics: Treebanks

Cite

The paper itself, not this card. BibTeX from the library's record of the work.

@inproceedings{bohnet2013annotation,
  title = {{Towards the Annotation of Penn TreeBank with Information Structure}},
  author = {Bohnet, Bernd and Burga, Alicia and Wanner, Leo},
  booktitle = {Proceedings of the Sixth International Joint Conference on Natural Language Processing},
  year = {2013},
  url = {https://aclanthology.org/I13-1178/}
}

References

28 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. S. Calhoun, M. Nissim, M. Steedman and J. Brenier 2005. A Framework for Annotating Information Structure in Discourse. In Proceedings of the Workshop on Frontiers in Corpus Annotations II: Pie in the Sky, pages 45–52. Ann Arbor, MA: Association for Corpus Linguistics. find our card
2. M. Collins. 2002. Discriminative Training Methods for Hidden Markov Models: Theory and Experiments with Perceptron Algorithms. In EMNLP 2002., pages 1–8. find our card
3. M. Collins and B. Roark. 2004. Incremental parsing with the perceptron algorithm. In Proceedings of ACL, pages 112–119. find our card
4. F. Daneš. 1970. Zur linguistischen Analyse der Textstruktur. Folia Linguistica, 4:72–78. find
5. S. Dipper, M. Götze, M. Stede, and T. Wegst. 2004. ANNIS: A Linguistic Database for Exploring Information Structure. In S. Ishihara, M. Schmitz and A. Schwarz (eds.). Working Papers of the SFB 632, Interdisciplinary Studies on Information Structure 1, pages 245–279. Potsdam: University of Potsdam. find
6. N. Erteschik-Shir. 2007. Information Structure: The Syntax-Discourse Interface. Oxford University Press, Oxford. find
7. J. Firbas. 1966. Non-thematic Subjects in Contemporary English. Travaux Linguistiques de Prague, 2:229–236. find
8. Jan Hajič, Massimiliano Ciaramita, Richard Johansson, Daisuke Kawahara, Maria Antònia Martı́, Lluı́s Màrquez, Adam Meyers, Joakim Nivre, Sebastian Padó, Jan Štěpánek, Pavel Straňák, Mihai Surdeanu, Nianwen Xue, and Yi Zhang. 2009. The CONLL-2009 shared task: Syntactic and semantic dependencies in multiple languages. In Proceedings of the 2009 CoNLL Shared Task, pages 1–18. find our card
9. J. Hajič, J.Panevová, E. Hajicová, P. Sgall, P. Pajas, J. Štepánek, J. Havelka, M. Mikulová, Z. Žabokrtský. 2006. Prague Dependency Treebank 2.0. Charles University Prague. find
10. E. Hajičová. 2007. The Position of TFA (Information Structure) in a Dependency Based Description of Language . In K. Gerdes, T. Reuther, and L. Wanner (eds.). MTT 2007. Proceedings of the 3rd International Conference on Meaning-Text Theory. Wiener Slawistischer Almanach, Sonderband 69, Munich & Vienna. find
11. E. Hajičová, B. Partee, P. Sgall. 1998. Topic-Focus Articulation, Tripartite Structures, and Semantic Content. Kluwer Academic Publishers, Dordrecht. find
12. I. Kruijff-Korbayová and M. Steedman. 2003. Discourse and Information Structure. Journal of Logic, Language and Information, 12(3):249–259. find
13. K. Lambrecht. 1994. Information structure and sentence form: Topic, focus and the mental representations of discourse referents. Cambridge University Press, Cambridge. M.P. Marcus, B. Santorini and M.A. Marcinkiewicz. find
14. . 1993. Building a Large Annotated Corpus of English: The Penn Treebank. Computational Linguistics, 19(2):313–330. find our card
15. V. Mathesius. 1929. Zur Satzperspektive im modernen Englisch. Archiv für das Studium der neueren Sprachen und Literaturen, 155:202–210. find
16. I.A. Mel’čuk. 2001. Communicative Organization in Natural Language: The semantic-communicative structure of sentences. Benjamins Academic Publishers, Amsterdam. find
17. J. Nivre, J. Hall, and J. Nilsson. 2004. Memory-based dependency parsing. In Proceedings of CoNLL, pages 49–56. find
18. J. Nivre. 2004. Incrementality in deterministic dependency parsing. In Proceedings of the Workshop on Incremental Parsing: Bringing Engineering and Cognition Together (ACL), pages 50–57. find our card
19. M. Palmer, D. Gildea, and P. Kingsbury. 2005. The Proposition Bank: An Annotated Corpus of Semantic Roles. Computational Linguistics. Computational Linguistics, 31(1):71–105. find our card
20. P. Paggio 2006. Annotating Information Structure in a Corpus of Spoken Danish. In Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC), pages 1606–1609. find
21. O. Postolache, I. Kruijff-Korbayová and G.-J. M. Kruijff 2005. Data-driven approaches for information structure identification. In Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing, pages 9–16. find
22. P. Sgall. Functional Sentence Perspective in a Generative Description of Language. 1967. Prague Studies in Mathematical Linguistics, 2:203-225. find
23. M. Steedman. Information Structure and the Syntax-Phonology Interface. 2000. Linguistic Inquiry, 31(4):649-685. find
24. M. Taulé, M.A. Martı́ and M. Recasens. 2008. An-Cora: Multilevel Annotated Corpora for Catalan and Spanish. In Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC). Marrakesh, Morocco. C. Thielen, A. Schiller, S. Teufel and C. Stöckert. Guidelines für das Tagging deutscher Textkorpora mit STTS. 1999. Institute for Natural Language Processing, University of Stuttgart. http://www.ims.uni-stuttgart.de/projekte/corplex/TagSets/. ↗
25. E. Vallduvı́ 1992. Information Component. Garland, New York and London. find
26. H. Yamada and Y. Matsumoto. 2003. Statistical dependency analysis with support vector machines. In Proceedings of IWPT, pages 195–206. find our card
27. M. Zimmermann and C. Féry 2009. Information Structure: Theoretical, Typological, and Experimental Perspectives. Oxford University Press, Oxford. find
28. Y. Zhang and J. Nivre. 2011. Transition-based parsing with rich non-local features. In Proceedings of ACL. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/bohnet-2013-annotation-penn-treebank-information-structure/