The LinGO Redwoods Treebank: Motivation and Preliminary Applications

What it is about

The paper proposes a treebank that stores full HPSG analyses from a broad-coverage grammar (the LinGO English Resource Grammar) instead of one fixed tree format, so that users can extract the representation they need. Annotators do not draw trees. They see the parses the grammar gives and choose the preferred one by answering short binary questions about its properties (discriminants). The choices are kept, so that they can be replayed when the grammar changes. The first sentences come from VerbMobil dialogues, and one second-year undergraduate finished the first round in under ten weeks. For parse ranking, a PCFG over derivation trees with grandparent annotation, plus a tag-sequence trigram, picks the right parse for 74.03% of ambiguous sentences (random 25.81%), but only 45.33% of sentences with 20 or more analyses. Ranking derivation trees (71.73%) beats ranking phrase structure trees with the same model (65.65%).

Ideas

• I1. treebanks and tools/treeagree: keep, for each accepted choice on a silver tree, the property that decided it (the attachment, the label, the lemma). When en changes, replay the stored choices on the new candidate parses and count how many still select one analysis, and how many change. The paper names this as the main maintenance problem. Measure on the school silver sentences, twice, with identical results before any claim.
• I2. Ambiguity buckets for the en beam: when the parser keeps several trees for a sentence, rank them by a UPOS trigram and by the parser score, and report accuracy against EWT gold in the paper's buckets (2 or more, 5 or more, 10 or more, 20 or more candidates). Use only EWT test sentences where the beam gives more than one tree, so that the comparison is on ambiguous input.
• I3. Split the ranking error, as the paper's oracle row does: give the ranker the gold UPOS sequence and see how much accuracy comes back. Mova's parse-eval already prints scores with gold tags and with its own tags; this idea extends the same split to the ranking of alternative trees.

For Mova

The chapters are 20. Syntax: the Universal Dependencies frame (ambiguity and attachment) and 22. Phrases: noun, adjective, adverb and prepositional phrases. The code is en/src/parse.rs (the beam), en/src/explain.rs (a reason for each decision) and tools/treeagree. The Redwoods and VerbMobil material have their own terms: check Data and model licences before any use, and use restricted material for tests only. No Redwoods data is in the repo.

Topics: Treebanks

Cite

The paper itself, not this card. BibTeX from the library's record of the work.

@inproceedings{oepen2002lingo,
  title = {{The LinGO Redwoods Treebank: Motivation and Preliminary Applications}},
  author = {Oepen, Stephan and Toutanova, Kristina and Shieber, Stuart and Manning, Christopher and Flickinger, Dan and Brants, Thorsten},
  booktitle = {COLING 2002: The 17th International Conference on Computational Linguistics: Project Notes},
  year = {2002},
  url = {https://aclanthology.org/C02-2025/}
}

References

18 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Abney, S. P. (1997). Stochastic attribute-value grammars. Computational Linguistics, 23, 597 – 618. find
2. Agresti, A. (1990). Categorical data analysis. John Wiley & Sons. find
3. Bouma, G., Noord, G. van, & Malouf, R. (2001). Alpino. Wide-coverage computational analysis of Dutch. In W. Daelemans, K. Sima-an, J. Veenstra, & J. Zavrel (Eds.), Computational linguistics in the Netherlands (pp. 45 – 59). Amsterdam, The Netherlands: Rodopi. find
4. Carter, D. (1997). The TreeBanker. A tool for supervised training of parsed corpora. In Proceedings of the Workshop on Computational Environments for Grammar Development and Linguistic Engineering. Madrid, Spain. find
5. Charniak, E. (1997). Statistical parsing with a context-free grammar and word statistics. In Proceedings of the Fourteenth National Conference on Artificial Intelligence (pp. 598 – 603). Providence, RI. find
6. Charniak, E., & Caroll, G. (1994). Context-sensitive statistics for improved grammatical language models. In Proceedings of the Twelth National Conference on Artificial Intelligence (pp. 742 – 747). Seattle, WA. find
7. Collins, M. J. (1997). Three generative, lexicalised models for statistical parsing. In Proceedings of the 35th Meeting of the Association for Computational Linguistics and the 7th Conference of the European Chapter of the ACL (pp. 16 – 23). Madrid, Spain. find
8. Copestake, A. (2002). Implementing typed feature structure grammars. Stanford, CA: CSLI Publications. find
9. Copestake, A., Lascarides, A., & Flickinger, D. (2001). An algebra for semantic construction in constraint-based grammars. In Proceedings of the 39th Meeting of the Association for Computational Linguistics. Toulouse, France. find our card
10. Dipper, S. (2000). Grammar-based corpus annotation. In Workshop on linguistically interpreted corpora LINC-2000 (pp. 56 – 64). Luxembourg. find
11. Flickinger, D. (2000). On building a more efficient grammar by exploiting types. Natural Language Engineering, 6 (1) (Special Issue on Efficient Processing with HPSG), 15 – 28. find our card
12. Harris, T. E. (1963). The theory of branching processes. Berlin, Germany: Springer. find
13. Johnson, M., Geman, S., Canon, S., Chi, Z., & Riezler, S. (1999). Estimators for stochastic ‘unification-based’ grammars. In Proceedings of the 37th Meeting of the Association for Computational Linguistics (pp. 535 – 541). College Park, MD. find
14. Manning, C. D., & Schütze, H. (1999). Foundations of statistical Natural Language Processing. Cambridge, MA: MIT Press. find
15. Mullen, T., Malouf, R., & Noord, G. van. (2001). Statistical parsing of Dutch using Maximum Entropy models with feature merging. In Proceedings of the Natural Language Processing Pacific Rim Symposium. Tokyo, Japan. find
16. Oepen, S., & Callmeier, U. (2000). Measure for measure: Parser cross-fertilization. Towards increased component comparability and exchange. In Proceedings of the 6th International Workshop on Parsing Technologies (pp. 183 – 194). Trento, Italy. find
17. Toutanova, K., & Manning, C. D. (2002). Feature selection for a rich HPSG grammar using decision trees. In Proceedings of the sixth conference on natural language learning (CoNLL-2002). Taipei. find
18. Wahlster, W. (Ed.). (2000). Verbmobil. Foundations of speech-to-speech translation. Berlin, Germany: Springer. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/bank-2002-oepen-lingo-redwoods-treebank-motivation-preliminary/