Adapting WSJ-Trained Parsers to the British National Corpus Using In-Domain Self-Training

What it is about

The question is whether a parser trained on WSJ text can be adapted to another domain without new annotation. The authors build a gold set of 1,000 BNC sentences (500 for development, 500 for test), chosen to contain verbs that do not occur in the WSJ training sections, and parse the whole BNC (6 million sentences, 99.8% parsed) with the Charniak and Johnson reranking parser. They retrain the parser on combinations of its own BNC parses and the WSJ trees, with the WSJ copies repeated. The best mix, one million BNC trees plus WSJ data repeated ten times, raises the BNC test F-score by 1.7 points to 85.6 and the WSJ section 23 F-score to 91.7 (+0.4). The paper concludes that in-domain self-training helps, while the seed domain must keep the original trees.

Ideas

• I1. Self-training for en/src/parse.rs: parse a large raw collection under an open licence with en, add those trees to the perceptron training, and repeat the gold EWT training trees 1, 5 or 10 times. Measure LAS on an en folds dev fold and on the seed domain, with the silver size varied (50k, 250k, 1000k trees).
• I2. Hard-verb subset: select held-out EWT sentences that contain a verb form unseen in the training trees, as the paper selects its gold sentences, and report LAS on that subset in en parse-eval. Measure it before and after I1.
• I3. Lower-bound check: report LAS on the other treebanks/en/ banks (childes, lines, gum, gentle) before and after I1, so that a gain in one domain cannot hide a loss in the seed domain.

For Mova

The chapters are 37. Register and variation and 20-syntax-overview.md. The code is en/src/parse.rs (training), with en folds, en parse-eval and treebanks/en/. The BNC data is NC, so the paper's gold set is test only and never part of what is built. Self-training uses Mova's own trees, so no large model is involved. The large raw run needs the operator's explicit word before it starts.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.3115/1621410.1621414

@inproceedings{foster2007adapting,
  title = {{Adapting WSJ-Trained Parsers to the British National Corpus Using In-Domain Self-Training}},
  author = {Foster, Jennifer and Wagner, Joachim and Seddah, Djamé and Genabith, Josef van},
  booktitle = {Proceedings of the Tenth International Conference on Parsing Technologies},
  year = {2007},
  doi = {10.3115/1621410.1621414},
  url = {https://doi.org/10.3115/1621410.1621414}
}

References

12 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Michiel Bacchiani, Michael Riley, Brian Roark, and Richard Sproat. 2006. Map adaptation of stochastic grammars. Computer Speech and Language, 20(1):41–68. find
2. Ann Bies, Mark Ferguson, Karen Katz, and Robert MacIntyre. 1995. Bracketing guidelines for treebank II style, Penn Treebank project. Technical Report MS-CIS-95-06, University of Pennsylvania. find
3. Lou Burnard. 2000. User reference guide for the British National Corpus. Technical report, Oxford University. find
4. Eugene Charniak and Mark Johnson. 2005. Coarse-to-fine nbest-parsing and maxent discriminative reranking. In Proceedings of ACL-05, pages 173–180, Barcelona. find
5. Eugene Charniak. 2000. A maximum-entropy-inspired parser. In Proceedings of NAACL-00, pages 132–139, Seattle. find
6. Michael Collins. 2003. Head-driven statistical models for natural language parsing. Computational Linguistics, 29(4):499–637. find our card
7. W. Nelson Francis and Henry Kučera. 1979. Brown Corpus Manual. Technical report, Brown University, Providence. find
8. Daniel Gildea. 2001. Corpus variation and parser performance. In Proceedings of EMNLP-01, pages 167–202, Barcelona. find our card
9. Mitchell P. Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of English: the Penn Treebank. Computational Linguistics, 19(2):313–330. ↗ our card
10. David McClosky, Eugene Charniak, and Mark Johnson. 2006a. Effective self-training for parsing. In Proceedings of HLT-NAACL-06, pages 152–159, New York. find our card
11. David McClosky, Eugene Charniak, and Mark Johnson. 2006b. Reranking and self-training for parser adaptation. In Proceedings of COLING-ACL-06, pages 337–344, Sydney. find our card
12. Mark Steedman, Miles Osbourne, Anoop Sarkar, Stephen Clark, Rebecca Hwa, Julia Hockenmaier, Paul Ruhlen, Steven Baker, and Jeremiah Crim. 2003. Boot-strapping statistical parsers from small datasets. In Proceedings of EACL-03, Budapest. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/lingbulk-2007-foster-adapting-wsj-trained-parsers-british/