MorphyNet: a Large Multilingual Database of Derivational and Inflectional Morphology

What it is about

MorphyNet is a multilingual database of morphology built from Wiktionary. Its first version covers 15 languages with 519k derivational and 10.1M inflectional entries, each with lemma, part of speech, morphological tags and affixes. The extraction uses hand-written rules for Wiktionary inflection tables and headword lines, plus an enrichment step that copies derivation patterns across languages. Enrichment adds 25.6% to the number of derivations; for English it adds only 3%, against 57% for Spanish. Manual checks on 1,000 sampled entries per language give 98.9% precision overall and 99.5% for English.

Ideas

• I1. en/src/morph.rs: take the English derivation pairs as a reference list of affixed forms and their bases. Measure how many of them the current lemma module already maps to the same base on EWT test forms. The pairs are Wiktionary-derived, so they are test data only.
• I2. en/src/lexgen.rs: add a counted suffix rule for English derivations (for example -ation from -ate or -e) that is accepted only when the paper-style evidence count passes a threshold. Measure precision and recall of the new derivations against the English reference pairs, on a held-out split.
• I3. Sample audit: draw 100 derived forms from en output, check them by hand, and report the precision with its interval, as the paper does with its 1,000-entry sample. Use it as a quality gate before any new derivation rule is added.

For Mova

The chapters are 17. Derivation II: suffixes and 16-derivation-prefixes.md. The code is en/src/morph.rs and en/src/lexgen.rs, and the check is en morph-eval. The paper's English gain is small, so the English value comes from the checks, not from the data. Its data is derived from Wiktionary, so use it for tests and as a reference only, never in what is built. Measure first: the share of English reference derivations that en morph-eval already reproduces.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/2021.sigmorphon-1.5

@inproceedings{batsuren2021morphynet,
  title = {{MorphyNet: a Large Multilingual Database of Derivational and Inflectional Morphology}},
  author = {Batsuren, Khuyagbaatar and Bella, Gabor and Giunchiglia, Fausto},
  booktitle = {SIGMORPHON 2021 (ACL workshop)},
  year = {2021},
  doi = {10.18653/v1/2021.sigmorphon-1.5},
  url = {https://doi.org/10.18653/v1/2021.sigmorphon-1.5}
}

References

50 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Duygu Ataman and Marcello Federico. 2018. Compositional representation of morphologically-rich input for neural machine translation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 305–311. find
2. Khuyagbaatar Batsuren, Gábor Bella, and Fausto Giunchiglia. 2021. A large and evolving cognate database. Language Resources and Evaluation. find
3. A major piece of ongoing work concerns the representation of MorphyNet derivational data as a lexico-semantic graph, as it is done in wordnets (Miller, 1998; Giunchiglia et al., 2017) where derivationally related word senses are interconnected by associative relationships. This effort, justifying th… find
4. Khuyagbaatar Batsuren, Gábor Bella, and Fausto Giunchiglia. 2019. Cognet: a large-scale cognate database. In Proceedings of ACL 2019, Florence, Italy. find
5. Kenneth R Beesley and Lauri Karttunen. 2003. Finite-state morphology: Xerox tools and techniques. CSLI, Stanford. find
6. Eduard Bejček, Eva Hajičová, Jan Hajič, Pavlína Jínová, Václava Kettnerová, Veronika Kolářová, Marie Mikulová, Jiří Mírovskỳ, Anna Nedoluzhko, Jarmila Panevová, et al. 2013. Prague dependency treebank 3.0. find
7. We are happy to offer the MorphyNet extraction logic to be reused on a community basis. As extending the tool with new Wiktionary extraction rules is straightforward, we hope that the availability of the tool will allow language coverage to grow even further. We also hope that the MorphyNet data and… find
8. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics, 5:135–146. word-level prediction. Transactions of the Association for Computational Linguistics, 6:451– 465. find
9. Cristina Bosco, Montemagni Simonetta, and Simi Maria. 2013. Converting italian treebanks: Towards an italian stanford dependency treebank. In 7th Linguistic Annotation Workshop and Interoperability with Discourse, pages 61–69. The Association for Computational Linguistics. find
10. Fausto Giunchiglia, Khuyagbaatar Batsuren, and Gabor Bella. 2017. Understanding and exploiting language diversity. In IJCAI, pages 4009– 4017. find
11. Kyle Gorman, Arya D McCarthy, Ryan Cotterell, Ekaterina Vylomova, Miikka Silfverberg, and Magdalena Markowska. 2019. Weird inflects but ok: Making sense of morphological generation errors. In Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL), pages 140–151. find
12. Ryan Cotterell, Christo Kirov, John Sylak-Glassman, Géraldine Walther, Ekaterina Vylomova, Arya D McCarthy, Katharina Kann, Sebastian J Mielke, Garrett Nicolai, Miikka Silfverberg, et al. 2018. The conll–sigmorphon 2018 shared task: Universal morphological reinflection. In Proceedings of the CoNLL–… find
13. Bruno Guillaume, Marie-Catherine de Marneffe, and Guy Perrier. 2019. Conversion et améliorations de corpus du français annotés en universal dependencies. Traitement Automatique des Langues, 60(2):71–95. find
14. Ryan Cotterell, Christo Kirov, John Sylak-Glassman, Géraldine Walther, Ekaterina Vylomova, Patrick Xia, Manaal Faruqui, Sandra Kübler, David Yarowsky, Jason Eisner, et al. 2017. find
15. Conll-sigmorphon 2017 shared task: Universal morphological reinflection in 52 languages. In Proceedings of the CoNLL SIGMORPHON 2017 Shared Task: Universal Morphological Reinflection, pages 1–30. find
16. Nizar Habash and Bonnie Dorr. 2003. Catvar: A database of categorial variations for english. In Proceedings of the MT Summit, pages 471–474. Citeseer. find our card
17. Nabil Hathout and Fiammetta Namer. 2014. Démonette, a french derivational morpho-semantic network. In Linguistic Issues in Language Technology, Volume 11, 2014-Theoretical and Computational Morphology: New Trends and Synergies. find
18. Gerard De Melo. 2014. Etymological wordnet: Tracing the history of words. In LREC, pages 1148–1154. Citeseer. find our card
19. Valeria de Paiva, Livy Real, Alexandre Rademaker, and Gerard de Melo. 2014. Nomlex-pt: A lexicon of portuguese nominalizations. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 2851–2858. find
20. Inxight. 2005. Linguistx natural language processing platform. find
21. Christo Kirov, Ryan Cotterell, John Sylak-Glassman, Géraldine Walther, Ekaterina Vylomova, Patrick Xia, Manaal Faruqui, Sabrina J Mielke, Arya D McCarthy, Sandra Kübler, et al. 2018. Unimorph 2.0: Universal morphology. In Proceedings of the Eleventh International Conference on Language Resources and… find
22. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… find
23. Christo Kirov, John Sylak-Glassman, Roger Que, and David Yarowsky. 2016. Very-large scale parsing and normalization of wiktionary morphological paradigms. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 3121–3126. find our card
24. Daniela Gerz, Ivan Vulić, Edoardo Ponti, Jason Naradowsky, Roi Reichart, and Anna Korhonen. 2018. Language modeling for morphologically rich languages: Character-aware modeling for Lukáš Kyjánek, Zdeněk Žabokrtskỳ, Magda Ševčíková, and Jonáš Vidra. 2020. Universal derivations 1.0, a growing collecti… find
25. George A Miller. 1998. WordNet: An electronic lexical database. MIT press. find
26. Joakim Nivre, Željko Agić, Lars Ahrenberg, Lene Antonsen, Maria Jesus Aranzabe, Masayuki Asahara, Luma Ateyah, Mohammed Attia, Aitziber Atutxa, Liesbeth Augustinus, et al. 2017. Universal dependencies 2.1. find
27. Krister Lindén and Lauri Carlson. 2010. Finnwordnet–finnish wordnet by translation. LexicoNordica–Nordic Journal of Lexicography, 17:119–140. find
28. Joakim Nivre, Marie-Catherine De Marneffe, Filip Ginter, Yoav Goldberg, Jan Hajic, Christopher D Manning, Ryan McDonald, Slav Petrov, Sampo Pyysalo, Natalia Silveira, et al. 2016. Universal dependencies v1: A multilingual treebank collection. In Proceedings of the Tenth International Conference on L… find our card
29. Krister Lindén, Jyrki Niemi, and Mirka Hyvärinen. 2012. Extending and updating the finnish wordnet. In Shall We Play the Festschrift Game?, pages 67–98. Springer. find
30. Eleonora Litta, Marco Passarotti, and Chris Culy. 2016. Formatio formosa est. Building a Word Formation Lexicon for Latin. In Proceedings of the Third Italian Conference on Computational Linguistics (CLIC–IT 2016), pages 185–189. find
31. Joakim Nivre and Beata Megyesi. 2007. Bootstrapping a swedish treebank using cross-corpus harmonization and annotation projection. In Proceedings of the 6th international workshop on treebanks and linguistic theories, pages 97–102. Citeseer. find
32. Olga Lyashevskaya, Kira Droganova, Daniel Zeman, Maria Alexeeva, Tatiana Gavrilova, Nina Mustafina, Elena Shakurova, et al. 2016. Universal dependencies for russian: A new syntactic dependencies tagset. Lyashevkaya, K. Droganova, D. Zeman, M. Alexeeva, T. Gavrilova, N. Mustafina, E. Shakurova//Highe… find
33. Mārcis Pinnis, Rihards Krišlauks, Daiga Deksne, and Toms Miks. 2017. Neural machine translation for morphologically rich languages with improved sub-word units and synthetic data. In International Conference on Text, Speech, and Dialogue, pages 237–245. Springer. find
34. Ivan Provilkov, Dmitrii Emelianenko, and Elena Voita. 2020. Bpe-dropout: Simple and effective subword regularization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1882–1892. find
35. Arya D McCarthy, Christo Kirov, Matteo Grella, Amrit Nidhi, Patrick Xia, Kyle Gorman, Ekaterina Vylomova, Sabrina J Mielke, Garrett Nicolai, Miikka Silfverberg, et al. 2020. Unimorph 3.0: Universal morphology. In Proceedings of The 12th Language Resources and Evaluation Conference, pages 3922–3931. find
36. Sampo Pyysalo, Jenna Kanerva, Anna Missilä, Veronika Laippala, and Filip Ginter. 2015. Universal dependencies for finnish. In Proceedings of the 20th Nordic Conference of Computational Linguistics (Nodalida 2015), pages 163–172. find
37. Arya D McCarthy, Miikka Silfverberg, Ryan Cotterell, Mans Hulden, and David Yarowsky. 2018. Marrying universal dependencies and universal morphology. In Proceedings of the Second Workshop on Universal Dependencies (UDW 2018), pages 91–101. find
38. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715–1725. find
39. Eleni Metheniti and Günter Neumann. 2020. Wikinflection corpus: A (better) multilingual, morpheme-annotated inflectional corpus. In Proceedings of The 12th Language Resources and Evaluation Conference, pages 3905–3912. find
40. Natalia Silveira, Timothy Dozat, Marie-Catherine de Marneffe, Samuel Bowman, Miriam Connor, John Bauer, and Christopher D. Manning. 2014. find
41. A gold standard dependency corpus for English. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC-2014). find
42. John Sylak-Glassman. 2016. The composition and use of the universal morphological feature schema (unimorph schema). Johns Hopkins University. find
43. Luigi Talamo, Chiara Celata, and Pier Marco Bertinetto. 2016. Derivatario: An annotated lexicon of italian derivatives. Word Structure, 9(1):72–102. find
44. Mariona Taulé, Maria Antònia Martí, and Marta Recasens. 2008. Ancora: Multilevel annotated corpora for catalan and spanish. In Lrec. find
45. Viktor Trón, Gyögy Gyepesi, Péter Halácsy, András Kornai, László Németh, and Dániel Varga. 2005. Hunmorph: open source word analysis. In Proceedings of Workshop on Software, pages 77–85. find
46. Jonáš Vidra, Zdeněk Žabokrtskỳ, Magda Ševčíková, and Lukáš Kyjánek. 2019. Derinet 2.0: towards an all-in-one word-formation resource. In Proceedings of the Second International Workshop on Resources and Tools for Derivational Morphology, pages 81–89. find
47. Veronika Vincze, Dóra Szauter, Attila Almási, György Móra, Zoltán Alexin, and János Csirik. 2010. Hungarian dependency treebank. find
48. Daniil Vodolazsky. 2020. Derivbase. ru: A derivational morphology resource for russian. In Proceedings of The 12th Language Resources and Evaluation Conference, pages 3937–3943. find
49. Ekaterina Vylomova, Trevor Cohn, Xuanli He, and Gholamreza Haffari. 2017. Word representation models for morphologically rich languages in neural machine translation. In Proceedings of the First Workshop on Subword and Character Level Models in NLP, pages 103–108. find
50. Britta Zeller, Jan Šnajder, and Sebastian Padó. 2013. Derivbase: Inducing and evaluating a derivational morphology resource for german. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1201–1211. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/lex-2021-batsuren-morphynet/