Parsing Early Modern English for Linguistic Search

What it is about

The question is whether a neural constituency parser is accurate enough for syntactic research on Early Modern English. The parser is trained on the Penn-Helsinki Parsed Corpus of Early Modern English (about 1.7 million words, 1500–1712), with an 8-fold cross-validation split that puts 90% of the source texts in training and balances the three time periods. Contextual embeddings are pre-trained on the 1.5-billion-word EEBO corpus, which the authors say improves evalb by about 2 points over modern embeddings. Standard evaluation gives a bracketing score of 90.5 F1 and 98.3% tag accuracy on test. The real test is query-based: six sentence types that track the rise of auxiliary "do" (negative declaratives, negative imperatives, questions, in both old and new forms) are retrieved with CorpusSearch from gold and parsed trees. Recall is 93–95% for the common types and very unstable for imperatives (72–86%); the standard deviations across splits are high, and questions score lower on test than on dev because of how sentence types are distributed across splits.

Ideas

• I1. 8. Word order and phrase structure and en/src/parse.rs: define the six sentence types as rule patterns over the UD tree (negation position, subject position relative to "do"). Measure the pattern recall and precision against gold sentences drawn from the Early Modern lessons.
• I2. Cross-validate the Early Modern tests by source text, not by sentence, as the paper does. Report the spread over splits, so one hard text cannot pass as a score.
• I3. Use the paper's split-variance result as a gate: a rule change is accepted only if the change is not driven by one split.

For Mova

Chapters: 5. Auxiliaries and modals, 7. Do, its absence, and negation, 8. Word order and phrase structure. Code: en/src/parse.rs, en/src/lin.rs, tools/parsecmp. Measure: pattern recall and precision for the do-support sentence types, with the per-split spread. The paper is a parser study with a neural model; we take only the evaluation design and the sentence-type test, not the architecture.

Cite

The paper itself, not this card. BibTeX from the library's record of the work.

@inproceedings{kulick2022parsing,
  title = {{Parsing Early Modern English for Linguistic Search}},
  author = {Kulick, Seth and Ryant, Neville and Santorini, Beatrice},
  booktitle = {Proceedings of the Society for Computation in Linguistics 2022},
  year = {2022},
  url = {https://aclanthology.org/2022.scil-1.12/}
}

References

48 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Sanjeev Arora, Yingyu Liang, and Tengyu Ma. 2017. A simple but tough-to-beat baseline for sentence embeddings. In International conference on learning representations. find
2. Iz Beltagy, Arman Cohan, and Kyle Lo. 2019. Scibert: Pretrained contextualized embeddings for scientific text. arXiv preprint arXiv:1903.10676. ↗
3. Don Blaheta. 2003. Function Tagging. Ph.D. thesis, Brown University. find
4. Don Blaheta and Eugene Charniak. 2000. Assigning function tags to parsed text. In 1st Meeting of the North American Chapter of the Association for Computational Linguistics. find
5. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. ↗
6. Aaron Ecay. 2015. A multi-step analysis of the evolution of English do-support. Ph.D. thesis, University of Pennsylvania. find
7. Dan Flickinger. 2011. Accuracy vs. robustness in grammar engineering. In E. M. Bender and J. E. Arnold, editors, Language from a cognitive perspective: Grammar, usage, and processing, CSLI Publications, pages 31–50. Stanford. find
8. Ryan Gabbard, Seth Kulick, and Mitchell Marcus. 2006. Fully parsing the Penn Treebank. In Proceedings of the Human Language Technology Conference of the NAACL, Main Conference, pages 184– 191, New York City, USA. Association for Computational Linguistics. find our card
9. Charlotte Galves. 2020. Relaxed V-Second in Classical Portuguese. In Rebecca Woods and Sam Wolfe, editors, Rethinking Verb-Second, pages 368–395. Oxford University Press. find
10. Charlotte Galves, Aroldo Leal de Andrade, and Pablo Faria. 2017. Tycho Brahe Parsed Corpus of Historical Portuguese. http://www.tycho.iel. unicamp.br/˜tycho/corpus/texts/psd.zip . ↗
11. Kyle Gorman and Steven Bedrick. 2019. We need to talk about standard splits. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2786–2791. find
12. Qiao Jin, Bhuwan Dhingra, William W Cohen, and Xinghua Lu. 2019. Probing biomedical embeddings from language models. arXiv preprint arXiv:1904.02181. ↗
13. Andres Karjus. 2020. Competition, selection and communicative need in language change: An investigation using corpora, computational modelling and experimentation. Ph.D. thesis, University of Edinburgh. find
14. Jungo Kasai, Robert Frank, Pauli Xu, William Merrill, and Owen Rambow. 2018. End-to-end graph-based tag parsing with neural networks. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1181–1194. find
15. Nikita Kitaev, Steven Cao, and Dan Klein. 2019. Multilingual constituency parsing with self-attention and pre-training. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3499–3505, Florence, Italy. Association for Computational Linguistics. find
16. Nikita Kitaev and Dan Klein. 2018. Constituency parsing with a self-attentive encoder. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2676–2686, Melbourne, Australia. Association for Computational Linguistics. find
17. Anthony Kroch. 1989. Reflexes of grammar in patterns of language change. Language Variation and Change, 1(3):199–244. find
18. Anthony Kroch. 2020. Penn Parsed Corpora of Historical English. LDC2020T16 Web Download. Philadelphia: Linguistic Data Consortium. Contains Penn-Helsinki Parsed Corpus of Middle English, second edition, Penn-Helsinki Parsed Corpus of Early Modern English, and Penn Parsed Corpus of Modern British English. find
19. Anthony Kroch and Beatrice Santorini. 2021. Penn-BFM Parsed Corpus of Historical French, version 1.0. https://github.com/beatrice57/ mcvf-plus-ppchf . ↗
20. Anthony Kroch, Beatrice Santorini, and Lauren Delfs. 2004. Penn-Helsinki Parsed Corpus of Early Modern English (PPCEME). CD-ROM, first edition, release 3. http://www.ling.upenn.edu/ppche/ ppche-release-2016/PPCEME-RELEASE-3 . ↗
21. Anthony Kroch, Beatrice Santorini, and Ariel Diertani. 2016. Penn Parsed Corpus of Modern British English (ppcmbe2). CD-ROM, second edition, release 1. http://www.ling.upenn.edu/ppche/ ppche-release-2016/PPCMBE2-RELEASE-1 . ↗
22. Anthony Kroch, Ann Taylor, and Donald Ringe. 2000a. The Middle English verb-second constraint: A case study in language contact and language change. In Susan Herring, Lene Schoessler, and Peter van Reenen, editors, Textual parameters in older language, pages 353–391. Benjamins. find
23. Anthony Kroch, Ann Taylor, and Beatrice Santorini. 2000b. Penn-Helsinki Parsed Corpus of Middle English (PPCME2). CD-ROM, second edition, release 4. http://www.ling.upenn.edu/ppche/ ppche-release-2016/PPCME2-RELEASE-4 . ↗
24. Seth Kulick, Anthony Kroch, and Beatrice Santorini. 2014. The Penn Parsed Corpus of Modern British English: First parsing results and analysis. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 662–667, Baltimore, Maryland. Association for Computational Linguistics. find our card
25. Jonathan K. Kummerfeld, David Hall, James R. Curran, and Dan Klein. 2012. Parser showdown at the Wall Street corral: An empirical investigation of error types in parser output. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 1048–1059, Jeju Island, Korea. Association for Computational Linguistics. find our card
26. Paola Merlo and Gabriele Musillo. 2005. Accurate function parsing. In Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing, pages 620–627, Vancouver, British Columbia, Canada. Association for Computational Linguistics. find
27. Taesun Moon and Jason Baldridge. 2007. Part-of-speech tagging for Middle English through alignment and projection of parallel diachronic texts. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), pages 390–399, Prague, Czech Republic. Association for Computational Linguistics. find our card
28. Sebastian Nagel. 2016. Cc-news. https://commoncrawl.org/2016/10/ news-dataset-available/ . ↗
29. Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. arXiv preprint arXiv:1802.05365. ↗
30. Beth Randall. 2010. CorpusSearch 2: a tool for linguistic research. Download site: http://corpussearch.sourceforge.net/CS.html . User guide: https://www.ling.upenn.edu/˜beatrice/corpus-ling/CS-users-guide/ index.html . ↗
31. Beatrice Santorini. 2021. find
32. Parsed Ellegård 1953 dataset. https://github.com/beatrice57/ ellegard-examples-parsed . ↗
33. Djamé Seddah, Sandra Kübler, and Reut Tsarfaty. 2014. Introducing the SPMRL 2014 shared task on parsing morphologically-rich languages. In Proceedings of the First Joint Workshop on Statistical Parsing of Morphologically Rich Languages and Syntactic Analysis of Non-Canonical Languages, pages 103– 109, Dublin, Ireland. Dublin City University. find
34. Satoshi Sekine and Michael Collins. 2008. http://nlp.cs.nyu.edu/evalb/ . evalb. ↗
35. Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang. 2019. BioBERT: pre-trained biomedical language representation model for biomedical text mining. arXiv preprint arXiv:1901.08746. ↗
36. Mitchell Stern, Jacob Andreas, and Dan Klein. 2017. A minimal span-based neural constituency parser. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistic (Volume 1: Long Papers), pages 818–827, Vancouver, Canada. Association for Computational Linguistics. find
37. Mitchell P. Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of English: The Penn Treebank. Computational Linguistics, 19(2):313–330. ↗ our card
38. Ann Taylor, Arja Nurmi, Anthony Warner, Susan Pintzuk, and Terttu Nevalainen. 2006. Parsed Corpus of Early English Correspondence. Distributed through the Oxford Text Archive. find
39. France Martineau, Paul Hirschbühler, Anthony Kroch, and Yves Charles Morin. 2021. MCVF Corpus, parsed, version 2.0. https://github.com/ beatrice57/mcvf-plus-ppchf . ↗
40. Ann Taylor, Anthony Warner, Susan Pintzuk, and Frans Beths. 2003. York-Toronto-Helsinki Parsed Corpus of Old English Prose. Distributed by the Oxford Text archive. find
41. Text Creation Partnership. 2019. English Books Online. Early https:// textcreationpartnership.org/tcp-texts/ eebo-tcp-early-english-books-online/ . ↗
42. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. arXiv preprint arXiv:1706.03762. ↗
43. Joel C. Wallenberg. 2016. Extraposition is disappearing. Language, 92(4):e237–e256. find
44. Joel C. Wallenberg, Rachael Bailes, Christine Cuskley, and Anton Karl Ingason. 2021. Smooth signals and syntactic change. Languages, 6(2):60. find
45. Joel C. Wallenberg, Anton Karl Ingason, Einar Freyr Sigurdhsson, and Eirı́kur Rögnvaldsson. 2011. Icelandic Parsed Historical Corpus (IcePaHC), v0.9. http://www.linguist.is/icelandic_ treebank . ↗
46. Yi Yang and Jacob Eisenstein. 2016. Part-of-speech tagging for historical English. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1318– 1328, San Diego, California. Association for Computational Linguistics. find our card
47. Yukun Zhu, Ryan Kiros, Rich Zemel, Ruslan Salakhutdinov, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. 2015. Aligning books and movies: Towards story-like visual explanations by watching movies and reading books. In Proceedings of the IEEE international conference on computer vision, pages 19– 27. find
48. Richard Zimmermann. 2017. Formal and quantitative approaches to the study of syntactic change: Three case studies from the history of English. Ph.D. thesis, University of Geneva. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/kulick-2022-parsing-early-modern-english-linguistic/