Lemmatising Verbs in Middle English Corpora: The Benefit of Enriching the Penn-Helsinki Parsed Corpus of Middle English 2 (PPCME2), the Parsed Corpus of Middle English Poetry (PCMEP), and A Parsed Linguistic Atlas of Early Middle English (PLAEME)

What it is about

The paper adds verb lemmas to three annotated Middle English treebanks, so that a query can find a verb by its dictionary entry and not by one of its many spellings. Four trained annotators and one author assigned 19,320 verb forms by hand to 2,979 lemmas, linked to Middle English Dictionary IDs, and kept 4,973 further possible lemmas. Insertion works in three steps: a lookup of the form, a lookup after spelling substitutions (such as e/y, v/u, th/+t), and a lookup after stripping Middle English inflectional endings. A verb with no match gets NA, and a warning attribute records which step matched. On PPCME2 (128,523 verbs), 86.23% matched directly, 5.03% by substitution and 8.06% by stripping, and 0.68% got no lemma; on a 100-token sample, precision was 94.85%, recall 98.92% and accuracy 94%. The same process was then applied to two smaller corpora, to fill a data gap for 1250 to 1350, and a study of reflexives gives different curves once those texts are added.

Ideas

• I1. en/src/morph.rs: the two short lists in archaic (ONLY_VERBS, IRREGULAR) could become a form-to-lemma inventory for the tale verbs, built as the paper does. List the verb forms of the book lessons, assign lemmas by hand to the most frequent, and keep the fallbacks in the paper's order: exact form, spelling rule, suffix stripping, then NA. Give each lemma a warning flag, as the paper's attribute does. Measure on 100 sampled tokens with the paper's categories, and on morph-eval.
• I2. Use the warning flag: a lemma found by stripping or marked doubtful should not count in the lexicon counts or in global/data/register-tale.tsv. Measure how much the tale register ranking changes when flagged lemmas are left in and when they are removed.
• I3. Coverage by book: for each book lesson, report the share of verb tokens that got a lemma, and the NA rate, from the parsedb. The paper's main point is that coverage gaps in one period distort the picture, so a book with low coverage should be marked as such before any count from it is used.

For Mova

The chapters are 09. Early Modern and archaic forms (its section on how Mova handles archaic forms, and the unlemmatised -en plurals and y- participles it lists) and 14. Irregular verbs. The code is en/src/morph.rs (archaic, ending_verb, lemmatize) and en/src/emode.rs. The inventory and MED IDs used in the paper are not ours to copy: build ours from our own texts, and check any outside dictionary for its licence first.

Cite

The paper itself, not this card. BibTeX from the library's record of the work.

@inproceedings{trips2020lemmatising,
  title = {{Lemmatising Verbs in Middle English Corpora: The Benefit of Enriching the Penn-Helsinki Parsed Corpus of Middle English 2 (PPCME2), the Parsed Corpus of Middle English Poetry (PCMEP), and A Parsed Linguistic Atlas of Early Middle English (PLAEME)}},
  author = {Trips, Carola and Percillier, Michael},
  booktitle = {Proceedings of the Twelfth Language Resources and Evaluation Conference},
  year = {2020},
  url = {https://aclanthology.org/2020.lrec-1.886/}
}

References

23 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Einenkel, E. (1916). Geschichte der englischen Sprache, volume II: Historische Syntax. Trübner, Strasbourg. find
2. Johanson, L. (2002). Contact-induced change in a code-copying framework. In Mari C. Jones et al., editors, Language change: the interplay of internal, external and extra-linguistic factors, pages 285–313. de Gruyter, Berlin. find
3. Keenan, E. (2009). Linguistic theory and the historical creation of English reflexives. In Paola Crisma et al., editors, Historical Syntax and Linguistic Theory, chapter 2, pages 17–40. Oxford University Press, Oxford. find
4. Kemmer, S. (1993). The Middle Voice. Number 23 in Typological Studies in Language. Benjamins Publishing, Amsterdam and Philadelphia. find
5. Kroch, A. and Taylor, A. (2000). The Penn-Helsinki Parsed Corpus of Middle English, Second Edition (PPCME2), release 3. University of Pennsylvania, Philadelphia. find
6. König, E. and Siemund, P. (2000). The development of complex reflexives and intensifiers in English. Diachronica, XVII(1):39–84. find
7. Laing, M. (2013). A Linguistic Atlas of Early Middle English, 1150–1325, Version 3.2. The University of Edinburgh, Edinburgh. find
8. Matti Rissanen, M. K. et al. (1991). The Helsinki Corpus of English Texts. Department of Modern Languages, University of Helsinki, Helsinki. find
9. McWhorter, J. (2004). What happened to English? In Hartmut Czepluch et al., editors, Focus on Germanic Typology, number 6 in Studia typologica, pages 19–60. Akademie Verlag, Berlin. find
10. Mustanoja, T. F. (1960). A Middle English Syntax Part I. Number 23 in Mémoires de la Société Néophilologique de Helsinki. Société Néophilologique, Helsinki. find
11. Peitsara, K. (1997). The development of reflexive strategies in english. In Matti Rissanen, et al., editors, Grammaticalization at Work: Studies of Long-Term Developments in English, pages 277–370. Mouton de Gruyter, Berlin. find
12. Percillier, M. (2016). Verb lemmatization and semantic verb classes in a Middle English corpus. In Proceedings of the 13th Conference on Natural Language Processing (KONVENS 2016), pages 209–214. find
13. Percillier, M. (2019). Dynamic modelling of medieval language contact: The case of Anglo-Norman and Middle English. In Roger Schöntag et al., editors, Diachrone Migrationslinguistik: Mehrsprachigkeit in historischen Sprachkontaktsituationen, pages 79–99. Peter Lang, Berlin. find
14. Michael Proffitt, editor. (2019). Oxford English Dictionary. Oxford University Press, Oxford, 3 edition. find
15. Randall, B. (2010). CorpusSearch (Version 2.003.00)[Computer Software]. find
16. Paul Schaffner, et al., editors. (2018). Middle English Dictionary. University of Michigan, Ann Arbor. find
17. Trips, C. and Stein, A. (2018). A comparison of multi-genre and single-genre corpora in the context of contact-induced change. In Richard Whitt, editor, Diachronic corpora, genre and language change, Studies in Corpus Linguistics, pages 241–260. Benjamins, Amsterdam and Philadelphia. find
18. Trips, C. and Stein, A. (2019). Contact-induced changes in the argument structure of Middle English verbs on the model of Old French. In Eitan Grossman, et al., editors, Journal of Language Contact. Special Issue on Valency and Transitivity in Contact, pages 232–267. Brill, Leiden. find
19. Truswell, R., Alcorn, R., Donaldson, J., and Wallenberg, J. (2018). A Parsed Linguistic Atlas of Early Middle English. University of Edinburgh. find
20. Truswell, R., Alcorn, R., Donaldson, J., and Wallenberg, J. (2019). A Parsed Linguistic Atlas of Early Middle English. In Rhona Alcorn, et al., editors, Historical Dialectology in the Digital Age, pages 19–38. Edinburgh University Press, Edinburgh. find
21. van Gelderen, E. (2000). A history of the English reflexive pronouns: person, self, interpretability. Linguistik Aktuell/Linguistics Today 39. John Benjamins, Amsterdam and Philadelphia. find
22. Visser, F. T. (1963). An Historical Syntax of the English Language. E. J. Brill, Leiden. find
23. Zimmermann, R. (2018). The Parsed Corpus of Middle English Poetry (PCMEP). find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/gram-2020-trips-lemmatising-verbs-middle-english-corpora/