Learning Accurate, Compact, and Interpretable Tree Annotation

What it is about

Hand-made tree annotations, which split a category such as NP into subject and object NPs, raise parser accuracy, but they are made by people for one language. The authors learn such splits automatically: every nonterminal is split into subsymbols by repeated split and merge cycles that maximize the likelihood of the training treebank, and each symbol can be split to a different degree. The learned grammar recovers the kinds of distinctions that linguists wrote by hand, and it is more compact and more accurate than earlier automatic splits with a fixed number of subsymbols. The parser reaches an F1 of 90.2% on the Penn Treebank, which is a 27% reduction in error over the base grammar.

Ideas

• I1. Latent subtags for the tagger in en/src/ptag.rs: split each tag into subtags by split and merge over the EWT training set, and measure UPOS accuracy on EWT dev, with the 94.74 test figure kept for the single release check.
• I2. Merge pass for compactness: after each split cycle, merge back the subtags that do not raise dev accuracy, and report model size in weights and dev UPOS against the current perceptron.
• I3. Interpretable splits: for each learned subtag, list its most frequent words and check on 100 sampled tokens whether it corresponds to a subject, object or modifier distinction; en explain can show the subtag chosen for each token.

For Mova

The tagger is en/src/ptag.rs with the TnT prior in en/src/tag.rs, the tag set is in en/src/ud.rs, and the explanation output is en/src/explain.rs. The chapter that the ideas touch is 10. Word classes. Measure first: UPOS on EWT dev with split subtags against the current tagger, run twice to confirm the result is not noise.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.3115/1220175.1220230

@inproceedings{petrov2006learning,
  title = {{Learning Accurate, Compact, and Interpretable Tree Annotation}},
  author = {Petrov, Slav and Barrett, Leon and Thibaux, Romain and Klein, Dan},
  booktitle = {Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics},
  year = {2006},
  doi = {10.3115/1220175.1220230},
  url = {https://doi.org/10.3115/1220175.1220230}
}

References

19 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. G. Ball and D. Hall. 1967. A clustering technique for summarizing multivariate data. Behavioral Science. find
2. S. Caraballo and E. Charniak. 1998. New figures of merit for best–first probabilistic chart parsing. In Computational Lingusitics, p. 275–298. find
3. E. Charniak and M. Johnson. 2005. Coarse-to-fine n-best parsing and maxent discriminative reranking. In ACL’05, p. 173–180. find our card
4. E. Charniak. 1996. Tree-bank grammars. In AAAI ’96, p. 1031–1036. find
5. E. Charniak. 2000. A maximum–entropy–inspired parser. In NAACL ’00, p. 132–139. find
6. D. Chiang and D. Bikel. 2002. Recovering latent information in treebanks. In Computational Linguistics. find
7. N. Chomsky. 1965. Aspects of the Theory of Syntax. MIT Press. find
8. M. Collins. 1999. Head-Driven Statistical Models for Natural Language Parsing. Ph.D. thesis, U. of Pennsylvania. find
9. J. Goodman. 1996. Parsing algorithms and metrics. In ACL ’96, p. 177–183. find
10. J. Henderson. 2004. Discriminative training of a neural network statistical parser. In ACL ’04. find
11. M. Johnson. 1998. PCFG models of linguistic tree representations. Computational Linguistics, 24:613–632. find
12. D. Klein and C. Manning. 2003. Accurate unlexicalized parsing. ACL ’03, p. 423–430. find our card
13. T. Matsuzaki, Y. Miyao, and J. Tsujii. 2005. Probabilistic CFG with latent annotations. In ACL ’05, p. 75–82. find
14. F. Pereira and Y. Schabes. 1992. Inside-outside reestimation from partially bracketed corpora. In ACL ’92, p. 128–135. find
15. D. Prescher. 2005. Inducing head-driven PCFGs with latent heads: Refining a tree-bank grammar for parsing. In ECML’05. find
16. H. Schuetze. 1998. Automatic word sense discrimination. Computational Linguistics, 24(1):97–124. find
17. S. Sekine and M. J. Collins. 1997. EVALB bracket scoring program. http://nlp.cs.nyu.edu/evalb/. ↗
18. K. Sima’an. 1992. Computatoinal complexity of probabilistic disambiguation. Grammars, 5:125–151. find
19. A. Stolcke and S. Omohundro. 1994. Inducing probabilistic grammars by bayesian model merging. In Grammatical Inference and Applications, p. 106–118. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2006-petrov-learning-accurate-compact-interpretable-tree/