Effective Self-Training for Parsing

What it is about

Self-training of parsers was thought to fail because the parser's own errors are amplified. This paper shows that it works when the self-labelled trees come from a two-stage parser and reranker: the 50-best parses of 24 million unlabelled News sentences are reranked, and the reranker's best tree is added to the Wall Street Journal training data, with the two counts merged. Adding the parser's own best trees does not help, but adding the reranker's best trees lifts the first-stage parser from 90.3 to 91.0 F on section 22 at 750k added sentences, and the full system reaches 92.1 F on WSJ, a 12% error reduction. The gain levels off near 91.0 F as more sentences are added.

Ideas

• I1. en/src/parse.rs training on silver: label the school corpus with the teacher's trees, and separately keep only the trees that treeagree agrees with the independent parsers; measure LAS on EWT dev (cv) as the silver set grows, and stop where the curve flattens, as this paper does.
• I2. en/src/ptag.rs and en/src/parse.rs: merge the counts of gold and silver events with a weight (for example 1 and 0.5) instead of equal weights, and measure LAS on EWT dev for both.
• I3. Keep a separate check of whether the silver set gains through the same features as the teacher's (the paper's own hypothesis): measure LAS on EWT dev with silver trees that the Mova parser itself chose, which should show less gain if that is the cause.

For Mova

The ideas touch the silver corpus in corpus/ and the agreement rules in tools/treeagree (src/rules.rs), and the training counts of en/src/parse.rs. The chapter that matters for the teacher-student setup is languages/en/37-register-and-variation, since the school corpus is not EWT. Measure first I1 on EWT dev, with the cv folds, and keep the EWT test for one run per release.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.3115/1220835.1220855

@inproceedings{mcclosky2006effective,
  title = {{Effective Self-Training for Parsing}},
  author = {McClosky, David and Charniak, Eugene and Johnson, Mark},
  booktitle = {Proceedings of the Human Language Technology Conference of the NAACL, Main Conference},
  year = {2006},
  doi = {10.3115/1220835.1220855},
  url = {https://doi.org/10.3115/1220835.1220855}
}

References

20 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Michiel Bacchiani, Michael Riley, Brian Roark, and Richard Sproat. 2006. MAP adaptation of stochastic grammars. Computer Speech and Language, 20(1):41–68. find
2. Avrim Blum and Tom Mitchell. 1998. Combining labeled and unlabeled data with co-training. In Proceedings of the 11th Annual Conference on Computational Learning Theory (COLT-98). find
3. Eugene Charniak and Mark Johnson. 2005. Coarse-to-fine n-best parsing and MaxEnt discriminative reranking. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL’05), pages 173–180, Ann Arbor, Michigan, June. Association for Computational Linguistics. find our card
4. Eugene Charniak. 1997. Statistical parsing with a context-free grammar and word statistics. In Proceedings of the Fourteenth National Conference on Artificial Intelligence, Menlo Park. AAAI Press/MIT Press. find
5. Eugene Charniak. 2000. A maximum-entropy-inspired parser. In 1st Annual Meeting of the NAACL. find
6. Stanley F. Chen and Joshua Goodman. 1996. An empirical study of smoothing techniques for language modeling. In Arivind Joshi and Martha Palmer, editors, Proceedings of the Thirty-Fourth Annual Meeting of the Association for Computational Linguistics. find
7. Stephen Clark, James Curran, and Miles Osborne. 2003. Bootstrapping POS-taggers using unlabelled data. In Proceedings of CoNLL-2003. find
8. Michael Collins. 2000. Discriminative reranking for natural language parsing. In Machine Learning: Proceedings of the 17th International Conference (ICML 2000), pages 175–182, Stanford, California. find our card
9. Sanjoy Dasgupta, M.L. Littman, and D. McAllester. 2001. PAC generalization bounds for co-training. In Advances in Neural Information Processing Systems (NIPS), 2001. find
10. Daniel Gildea. 2001. Corpus variation and parser performance. In Conference on Empirical Methods in Natural Language Processing (EMNLP). find our card
11. David Graff. 1995. North American News Text Corpus. Linguistic Data Consortium. LDC95T21. find
12. James Henderson. 2004. Discriminative training of a neural network statistical parser. In Proc. 42nd Meeting of Association for Computational Linguistics (ACL 2004), Barcelona, Spain. find
13. Donald Hindle and Mats Rooth. 1993. Structural ambiguity and lexical relations. Computational Linguistics, 19(1):103–120. find
14. Liang Huang and David Chang. 2005. Better k-best parsing. Technical Report MS-CIS-05-08, Department of Computer Science, University of Pennsylvania. find
15. Victor M. Jimenez and Andres Marzal. 2000. Computation of the n best parse trees for weighted and stochastic context-free grammars. In Proceedings of the Joint IAPR International Workshops on Advances in Pattern Recognition. find
16. Mark Johnson, Stuart Geman, Stephen Canon, Zhiyi Chi, and Stefan Riezler. 1999. Estimators for stochastic “unification-based” grammars. In The Proceedings of the 37th Annual Conference of the Association for Computational Linguistics, pages 535–541, San Francisco. Morgan Kaufmann. find
17. Dan Klein and Christopher Manning. 2002. A generative constituent-context model for improved grammar induction. In Proceedings of the 40th Annual Meeting of the ACL. find
18. Michell P. Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of English: The Penn Treebank. Computational Linguistics, 19(2):313–330. find our card
19. Anoop Sarkar. 2001. Applying cotraining methods to statistical parsing. In Proceedings of the 2001 NAACL Conference. find
20. Mark Steedman, Miles Osborne, Anoop Sarkar, Stephen Clark, Rebecca Hwa, Julia Hockenmaier, Paul Ruhlen, Steven Baker, and Jeremiah Crim. 2003. Bootstrapping statistical parsers from small datasets. In Proceedings of EACL 03. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2006-mcclosky-effective-self-training-parsing/