Part-of-Speech Tagging for Historical English

What it is about

The paper tests domain adaptation for POS tagging of Early Modern English (PPCEME) and Modern British English (PPCMBE). The best method, Feature Embedding, adapts the whole feature vector and beats word embeddings and Brown clusters. In the temporal setting it cuts the error by about 30%. Adapting from the Penn Treebank gives 74.2% accuracy, which rises to 79.1% when spelling normalisation is added. Normalisation lowers the out-of-vocabulary rate in PPCEME from 23.0% to 13.5%. Accuracy on out-of-vocabulary tokens is 49.0%, against 81.7% on known tokens. Tagset mismatch is the largest source of error on known words.

Ideas

• I1. en/src/ptag.rs: add a backoff feature that maps an unknown form to its lemma through en/src/morph.rs, a cheap analogue of the paper's whole-feature adaptation. Measure the accuracy on out-of-vocabulary tokens of tools/gutenberg lessons, with the old and the new feature set, on two identical runs.
• I2. Normalise first, then tag: run the archaic mapping of en/src/morph.rs before en/src/ptag.rs. Report the out-of-vocabulary rate before and after, as the paper does, so a gain has a visible cause.
• I3. Tagset mismatch: keep a table of the historical-to-modern tag remaps in en/src/ptag.rs (the paper's Q-to-DT change gave 0.78 points). Test each remap alone on the EWT dev section; the EWT test set stays held out.

For Mova

The chapters are 09. Early Modern and archaic forms and 10. Word classes. The code is en/src/ptag.rs, en/src/morph.rs and en tag-eval. The Penn historical corpora are licensed data (not open): use them for tests only. Measure first the out-of-vocabulary rate of the current tagger on tools/gutenberg lessons.

Topics: Part-of-speech tagging

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/n16-1157

@inproceedings{yang2016part,
  title = {{Part-of-Speech Tagging for Historical English}},
  author = {Yang, Yi and Eisenstein, Jacob},
  booktitle = {Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies},
  year = {2016},
  doi = {10.18653/v1/n16-1157},
  url = {https://doi.org/10.18653/v1/n16-1157}
}

References

47 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Alistair Baron and Paul Rayson. 2008. Vard2: A tool for dealing with spelling variation in historical corpora. In Postgraduate conference in corpus linguistics. find
2. Alistair Baron, Paul Rayson, and Dawn Archer. 2009. Word frequency and key word statistics in corpus linguistics. Anglistik, 20(1):41–67. find
3. Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wortman Vaughan. 2010. A theory of learning from different domains. Machine learning, 79(1-2):151–175. find
4. John Blitzer, Ryan McDonald, and Fernando Pereira. 2006. Domain adaptation with structural correspondence learning. In Proceedings of Empirical Methods for Natural Language Processing (EMNLP), pages 120–128. find
5. Lars Borin and Markus Forsberg. 2008. Something old, something new: A computational morphological description of old swedish. In LREC 2008 workshop on language technology for cultural heritage data (LaTeCH 2008), pages 9–16. find
6. Peter F Brown, Peter V Desouza, Robert L Mercer, Vincent J Della Pietra, and Jenifer C Lai. 1992. Class-based n-gram models of natural language. Computational linguistics, 18(4):467–479. find
7. Minmin Chen, Z. Xu, Killian Weinberger, and Fei Sha. 2012. Marginalized denoising autoencoders for domain adaptation. In Proceedings of the International Conference on Machine Learning (ICML). find
8. Hal Daumé III. 2007. Frustratingly easy domain adaptation. In Proceedings of the Association for Computational Linguistics (ACL), Prague. find
9. William C Dougherty. 2010. The Google Books Project: will it make libraries obsolete? The Journal of Academic Librarianship, 36(1):86–89. find
10. Jacob Eisenstein. 2013. What to do about bad language on the internet. In Proceedings of the North American Chapter of the Association for Computational Linguistics (NAACL), pages 359–369, Atlanta, GA. find
11. Eugenia Eumeridou, Blaise Nkwenti-Azeh, and John McNaught. 2004. An analysis of verb subcategorization frames in three special language corpora with a view towards automatic term recognition. Computers and the Humanities, 38(1):37–60. find
12. Jenny R. Finkel and Christopher Manning. 2009. Hierarchical bayesian domain adaptation. In Proceedings of the North American Chapter of the Association for Computational Linguistics (NAACL), pages 602–610, Boulder, CO. find
13. Charlotte Galves and Pablo Faria. 2010. Tycho Brahe Parsed Corpus of Historical Portuguese. http://www.tycho.iel.unicamp.br/ ˜tycho/corpus/en/index.html. ↗
14. Yaroslav Ganin and Victor Lempitsky. 2015. Unsupervised domain adaptation by backpropagation. In Proceedings of the International Conference on Machine Learning (ICML). find
15. Roger Garside and Nicholas Smith. 1997. A hybrid grammatical tagger: Claws4. Corpus annotation: Linguistic information from computer text corpora, pages 102–121. find
16. Jesús Giménez and Lluis Marquez. 2004. SVMTool: A general POS tagger generator based on support vector machines. In Proceedings of the Language Resources and Evaluation Conference. find
17. Rafael Giusti, A Candido, Marcelo Muniz, Lı́via Cucatto, and Sandra Aluı́sio. 2007. Automatic detection of spelling variation in historical corpus. In Proceedings of the Corpus Linguistics Conference (CL). find
18. Iris Hendrickx, Michel Généreux, and Rita Marquilhas. 2011. Automatic pragmatic text segmentation of historical letters. In Language Technology for Cultural Heritage, pages 135–153. Springer. find
19. Jing Jiang and ChengXiang Zhai. 2007. Instance weighting for domain adaptation in nlp. In Proceedings of the Association for Computational Linguistics (ACL), Prague. find
20. Mahesh Joshi, Mark Dredze, William W. Cohen, and Carolyn P. Rosé. 2013. What’s in a domain? multi-domain learning for multi-attribute data. In Proceedings of the North American Chapter of the Association for Computational Linguistics (NAACL), pages 685– 690, Atlanta, GA. find
21. Bryan Jurish. 2011. Finite-state canonicalization techniques for historical German. Ph.D. thesis, Universitätsbibliothek. find
22. Anthony Kroch and Ann Taylor. 2000. Penn-Helsinki Parsed Corpus of Middle English, second edition. http://www.ling.upenn.edu/hist-corpora/PPCME2-RELEASE-3/ index.html. ↗
23. Anthony Kroch, Beatrice Santorini, and Ariel Diertani. 2004. Penn-Helsinki Parsed Corpus of Early Modern English. http://www.ling.upenn.edu/hist-corpora/ PPCEME-RELEASE-2/index.html. ↗
24. Anthony Kroch, Beatrice Santorini, and Ariel Diertani. 2010. The Penn Parsed Corpus of Modern British English. http://www.ling.upenn.edu/hist-corpora/PPCMBE-RELEASE-1/ index.html. ↗
25. Vivek Kulkarni, Rami Al-Rfou, Bryan Perozzi, and Steven Skiena. 2015. Statistically significant detection of linguistic change. In Proceedings of International Conference on World Wide Web (WWW), pages 625–635. find
26. Yuri Lin, Jean-Baptiste Michel, Erez Lieberman Aiden, Jon Orwant, Will Brockman, and Slav Petrov. 2012. Syntactic annotations for the google books ngram corpus. In Proceedings of the ACL 2012 system demonstrations, pages 169–174. find
27. Christopher D. Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven J. Bethard, and David McClosky. 2014. The Stanford CoreNLP natural language processing toolkit. In Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 55–60. find
28. Yishay Mansour, Mehryar Mohri, and Afshin Rostamizadeh. 2009. Domain adaptation with multiple sources. In Neural Information Processing Systems (NIPS), pages 1041–1048. find
29. Mitchell P. Marcus, Mary Ann Marcinkiewicz, and Beatrice Santorini. 1993. Building a large annotated corpus of English: The Penn Treebank. Computational Linguistics, 19(2):313–330. find our card
30. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Neural Information Processing Systems (NIPS), pages 3111–3119, Lake Tahoe. find
31. Taesun Moon and Jason Baldridge. 2007. Part-of-speech tagging for middle english through alignment and projection of parallel diachronic texts. In Proceedings of Empirical Methods for Natural Language Processing (EMNLP), pages 390–399. find our card
32. Aditi Muralidharan and Marti A Hearst. 2013. Supporting exploratory text analysis in literature study. Literary and linguistic computing, 28(2):283–295. find
33. F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825– 2830. find
34. Eva Pettersson and Joakim Nivre. 2011. Automatic verb extraction from historical swedish texts. In Proceedings of the 5th ACL-HLT Workshop on Language Technology for Cultural Heritage, Social Sciences, and Humanities, pages 87–95. find
35. Xuan-Hieu Phan. 2006. CRFTagger: CRF English POS Tagger. http://crftagger.sourceforge.net. ↗
36. Michael Piotrowski. 2012. Natural language processing for historical texts. Synthesis Lectures on Human Language Technologies, 5(2):1–157. find
37. Adwait Ratnaparkhi. 1996. A maximum entropy model for part-of-speech tagging. In Proceedings of Empirical Methods for Natural Language Processing (EMNLP), pages 133–142. find
38. Paul Rayson, Dawn Archer, Alistair Baron, Jonathan Culpeper, and Nicholas Smith. 2007. Tagging the bard: Evaluating the accuracy of a modern pos tagger on early modern english corpora. In Corpus Linguistics Conference. find
39. Matti Rissanen, Merja Kytö, and Minna Palander-Collin. 1993. Early English in the computer age: Explorations through the Helsinki Corpus. Number 11. Walter de Gruyter. find
40. Silke Scheible, Richard J Whitt, Martin Durrell, and Paul Bennett. 2011. Evaluating an ’off-the-shelf’ POS-tagger on early modern German text. In Proceedings of the 5th ACL-HLT workshop on language technology for cultural heritage, social sciences, and humanities, pages 19–23. find
41. Tobias Schnabel and Hinrich Schütze. 2014. Flors: Fast and simple domain adaptation for part-of-speech tagging. Transactions of the Association of Computational Linguistics, 2:51–62. find
42. Gerold Schneider, Hans Martin Lehmann, and Peter Schneider. 2014. Parsing early and late modern english corpora. Literary and Linguistic Computing. find
43. Kristina Toutanova, Dan Klein, Christopher D. Manning, and Yoram Singer. 2003. Feature-rich part-of-speech tagging with a cyclic dependency network. In Proceedings of the North American Chapter of the Association for Computational Linguistics (NAACL). find
44. Joseph Turian, Lev Ratinov, and Yoshua Bengio. 2010. Word Representation: A Simple and General Method for Semi-Supervised Learning. In Proceedings of the Association for Computational Linguistics (ACL), pages 384–394, Uppsala, Sweden. find
45. Eric Tzeng, Judy Hoffman, Trevor Darrell, and Kate Saenko. 2015. Simultaneous deep transfer across domains and tasks. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pages 4068–4076. find
46. Yi Yang and Jacob Eisenstein. 2014. Fast easy unsupervised domain adaptation with marginalized structured dropout. In Proceedings of the Association for Computational Linguistics (ACL), Baltimore, MD. find
47. Yi Yang and Jacob Eisenstein. 2015. Unsupervised multi-domain adaptation with feature embeddings. In Proceedings of the North American Chapter of the Association for Computational Linguistics (NAACL), Denver, CO. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2016-yang-part-speech-tagging-historical-english/