On the Role of Morphological Information for Contextual Lemmatization

What it is about

Contextual lemmatizers are often trained with fine-grained morphological tags, although nobody has shown that these help the downstream task. The authors test this in six languages: Basque, Turkish, Russian, Czech, Spanish and English. They also test the lemmatizers outside their training domain, which is how they are mostly used. Their results are surprising. Fine-grained features do not help, even for the agglutinative languages. Modern contextual word representations already encode enough morphology to give competitive lemmatizers with no explicit morphological signal. Out of domain, the best lemmatizers are those with simple UPOS tags or no morphology at all. The authors also argue that the current evaluation practice does not separate the models well and needs other metrics. The English comparison uses EWT as the in-domain set and GUM as the out-of-domain set.

Ideas

• I1. en/src/morph.rs: test whether the lemma rules need the fine-grained tags or only the UPOS tag. Train the rules with each feature set on EWT train, then measure the lemma accuracy on EWT test and on GUM test, where GUM is test only.
• I2. Add an out-of-domain line to en morph-eval, reported next to the in-domain one, so that a rule that looks good on EWT is not chosen by default.
• I3. Add a second metric to en morph-eval beside accuracy, for example the share of lemma errors that are for words in the dictionary, since the paper argues accuracy alone does not separate the models.

For Mova

The code is en/src/morph.rs, with en morph-eval and en tag-eval. The chapter is 13. Verbs for verbs and 37. Register and variation for the register issue. GUM is NC, so it is for tests and measurement only, never in what is built. Measure first on EWT test with the in-domain numbers, then on GUM test, twice, to check the stability.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.1162/coli_a_00497

@article{toporkov2024role,
  title = {{On the Role of Morphological Information for Contextual Lemmatization}},
  author = {Toporkov, Olia and Agerri, Rodrigo},
  journal = {Computational Linguistics},
  year = {2024},
  doi = {10.1162/coli\_a\_00497},
  url = {https://doi.org/10.1162/coli_a_00497}
}

References

27 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Agerri, Rodrigo, Josu Bermudez, and German Rigau. 2014. IXA pipeline: Efficient and ready to use multilingual NLP tools. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 3823–3828. find
2. Agerri, Rodrigo and German Rigau. 2016. Robust multilingual named entity recognition with shallow semi-supervised features. Artificial Intelligence, 238(2):63–82. Chakrabarty. https://doi.org/10.1016 /j.artint.2016.05.003 Agerri, Rodrigo, Iñaki San Vicente, Jon Ander Campos, Ander Barrena, Xabier S… find
3. Sigmorphon 2019 task 2 system description paper: Morphological analysis in context for many languages, with supervision from only a few. In Proceedings of the 16th Workshop on Computational Research in Phonetics, Phonology, and Morphology, pages 87–94. https://doi.org/10 .18653/v1/W19-4211 Akbik, Al… find
4. Arkhipov, Mikhail, Maria Trofimova, Yuri Kuratov, and Alexey Sorokin. 2019. Tuning multilingual transformers for language-specific named entity recognition. In Proceedings of the 7th Workshop on Balto-Slavic Natural Language Processing, pages 89–93. https://doi .org/10.18653/v1/W19-3712 Belinkov, Yo… find
5. Bergmanis, Toms and Sharon Goldwater. 2018. Context sensitive neural lemmatization with Lematus. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1391–1400. https://doi.org… ↗
6. Chakrabarty, Abhisek, Onkar Arun Pandit, and Utpal Garain. 2017. Context sensitive lemmatization using two successive bidirectional gated recurrent networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1481–1491. https://d… find
7. Chrupala, Grzegorz, Georgiana Dinu, and Josef van Genabith. 2008. Learning morphology with Morfette. In Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC’08). find our card
8. Collins, Michael. 2002. Discriminative training methods for Hidden Markov Models: Theory and experiments with perceptron algorithms. In Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing (EMNLP 2002), pages 1–8. https://doi.org/10.3115 /1118693.1118694 Conneau, Al… find
9. Cotterell, Ryan and Georg Heigold. 2017. Cross-lingual character-level neural morphological tagging. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 748–759. https://doi.org/10.18653/v1/D17-1078 Dayanik, Erenay, Ekin Akyürek, and Deniz Yuret. 2018. M… ↗
10. Devlin, Jacob, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies… find
11. Grave, Edouard, Piotr Bojanowski, Prakhar Gupta, Armand Joulin, and Tomas Mikolov. 2018. Learning word vectors for 157 languages. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018). find
12. Heigold, Georg, Guenter Neumann, and Josef van Genabith. 2017. An extensive empirical evaluation of character-based morphological tagging for 14 languages. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 505… find
13. Jongejan, Bart and Hercules Dalianis. 2009. Automatic training of lemmatization rules that handle morphological changes in pre-, in- and suffixes alike. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Process… find
14. Kondratyuk, Dan. 2019. Cross-lingual lemmatization and morphology tagging with two-stage multilingual BERT fine-tuning. In Proceedings of the 16th Workshop on Computational Research in Phonetics, Phonology, and Morphology, pages 12–18. https://doi.org/10 .18653/v1/W19-4203 Kondratyuk, Dan and Milan… find our card
15. Liu, Yinhan, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized BERT pretraining approach. CoRR, abs/1907.11692. find
16. Lyashevkaya, Olga, Kira Droganova, Daniel Zeman, Maria Alexeeva, Tatiana Gavrilova, Nina Mustafina, and Elena Shakurova. 2016. Universal Dependencies for Russian: A new syntactic dependencies tagset. SSRN Electronic Journal. https:// doi.org/10.2139/ssrn.2859998 Ma, Xuezhe and Eduard Hovy. 2016. End… find
17. Malaviya, Chaitanya, Shijie Wu, and Ryan Cotterell. 2019. A simple joint model for improved contextual neural lemmatization. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Pap… find our card
18. McCarthy, Arya D., Christo Kirov, Matteo Grella, Amrit Nidhi, Patrick Xia, Kyle Gorman, Ekaterina Vylomova, Sabrina J. Mielke, Garrett Nicolai, Miikka Silfverberg, Timofey Arkhangelskiy, Nataly Krizhanovsky, Andrew Krizhanovsky, Elena Klyachko, Alexey Sorokin, John Mansfield, Valts Ernštreits, Yuva… find
19. McCarthy, Arya D., Ekaterina Vylomova, Shijie Wu, Chaitanya Malaviya, Lawrence Wolf-Sonkin, Garrett Nicolai, Christo Kirov, Miikka Silfverberg, Sabrina J. Mielke, Jeffrey Heinz, Ryan Cotterell, and Mans Hulden. 2019. find
20. The SIGMORPHON 2019 shared task: Morphological analysis in context and cross-lingual transfer for inflection. In Proceedings of the 16th Workshop on Computational Research in Phonetics, Phonology, and Morphology, pages 229–244. https://doi.org/10 .18653/v1/W19-4226 Müller, Thomas, Ryan Cotterell, A… find
21. Oflazer, Kemal. 1993. Two-level description of Turkish morphology. In Sixth Conference of the European Chapter of the Association for Computational Linguistics. https://doi .org/10.3115/976744.976810 Ortiz Suárez, Pedro Javier, Benoı̂t Sagot, and Laurent Romary. 2019. Asynchronous pipelines for pro… find
22. Segalovich, Ilya. 2003. A fast morphological algorithm with unknown word guessing induced by a dictionary for a Web search engine. In MLMTA, page 273. find
23. Silveira, Natalia, Timothy Dozat, Marie-Catherine de Marneffe, Samuel Bowman, Miriam Connor, John Bauer, and Chris Manning. 2014. A gold standard dependency corpus for English. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 2897–2904. find
24. Straka, Milan. 2018. UDPipe 2.0 prototype at CoNLL 2018 UD shared task. In Proceedings of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pages 197–207. Straka, Milan, Jana Straková, and Jan Hajic. 2019. UDPipe at SIGMORPHON 2019: Contextualized embeddings,… find
25. Sylak-Glassman, John. 2016. The composition and use of the universal morphological feature schema (UniMorph schema). Technical report. Taulé, Mariona, M. Antònia Martı́, and Marta Recasens. 2008. AnCora: Multilevel annotated corpora for Catalan and Spanish. In Proceedings of the Sixth Internationa… find
26. Wang, Zeqiang, Yile Wang, Jiageng Wu, Zhiyang Teng, and Jie Yang. 2022. YATO: Yet another deep learning based text analysis open toolkit. arXiv preprint arXiv:2209.13877. https://doi.org/10 .18653/v1/2023.emnlp-demo.11 Wolf, Thomas, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Ant… find
27. CoNLL 2017 shared task: Multilingual parsing from raw text to Universal Dependencies. In Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pages 1–19. https://doi.org/10 .18653/v1/K17-3001 find our card

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/cl-2024-toporkov-role-morphological-information-contextual-lemmatization/