Mova0.5 pre-release⌕
☰
home / docs / papers / Semitic languages / MoVoC: Morphology-Aware Subword Construction for Ge'ez Script Languages

MoVoC: Morphology-Aware Subword Construction for Ge'ez Script Languages

What it is about

The paper argues that BPE subwords cut Ge'ez-script languages (Amharic, Tigrinya, Ge'ez, Tigre) at the wrong places, splitting stems and affixes. The authors annotate morphemes for the four languages (80K, 80K, 20K and 32K items): HornMorpho is used for Amharic and Tigrinya, and manual annotation for Ge'ez and Tigre. Their MoVoC vocabulary is a fixed share r of morpheme units plus a BPE vocabulary for each language, merged into one; MoVoC-Tok then forbids BPE merges that cross a morpheme boundary. The tokenizer scores higher than BPE and WordPiece on morpheme boundary precision (for example 83.9 against 74.6 for Tigre, and 85.6 against 73.9 for Ge'ez) and on MorphScore, and the Rényi entropy is lower. In English-to-language MT with MarianMT, the gains in BLEU and chrF++ are consistent but small, and the paper says they are not significant.

Ideas

For Mova

I1 and I2 are for tools/hbo, measured against MACULA morpheme boundaries. I3 is a constraint in the same segmenter. The paper's own MT result is small, so use the morpheme measures first and not a downstream score.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.findings-emnlp.706

@inproceedings{teklehaymanot2025movoc,
  title = {{MoVoC: Morphology-Aware Subword Construction for Ge'ez Script Languages}},
  author = {Teklehaymanot, Hailay Kidu and Fazlija, Dren and Nejdl, Wolfgang},
  booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2025},
  year = {2025},
  doi = {10.18653/v1/2025.findings-emnlp.706},
  url = {https://doi.org/10.18653/v1/2025.findings-emnlp.706}
}

References

56 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Rami Al-Rfou, Dokook Choe, Noah Constant, Mandy Guo, and Llion Jones. 2019. Character-level language modeling with deeper self-attention. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01):3159–3166. find
  2. Mehdi Ali, Michael Fromm, Klaudia Thellmann, Richard Rutmann, Max Lübbering, Johannes Leveling, Katrin Klug, Jan Ebert, Niclas Doll, Jasper Buschhoff, Charvi Jain, Alexander Weber, Lena Jurkschat, Hammam Abdelwahab, Chelsea John, Pedro Ortiz Suarez, Malte Ostendorff, Samuel Weinbach, Rafet Sifa, Stefan Kesselheim, and Nicolas Flores-Herr. 2024. Tokenizer choice for LLM training: Negligible or crucial? In Findings of the Association for Computational Linguistics: NAACL 2024, pages 3907–3924, Mexico City, Mexico. Association for Computational Linguistics. find
  3. Catherine Arnett and Benjamin Bergen. 2025. Why do language models perform worse for morphologically complex languages? In Proceedings of the 31st International Conference on Computational Linguistics, pages 6607–6623, Abu Dhabi, UAE. Association for Computational Linguistics. find
  4. Ehsaneddin Asgari, Yassine El Kheir, and Mohammad Ali Sadraei Javaheri. 2025. Morphbpe: A morpho-aware tokenizer bridging linguistic complexity for efficient llm training across morphologies. arXiv preprint arXiv:2502.00894. ↗
  5. Thomas Bauwens and Pieter Delobelle. 2024. BPE-knockout: Pruning pre-existing BPE tokenisers with backwards-compatible morphological semi-supervision. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 5810– 5832, Mexico City, Mexico. Association for Computational Linguistics. find
  6. Ayele Bekerie. 2003. Historical overview of ethiopic writing system’s possible influence on the development of the armenian alphabet. International Journal of Ethiopian Studies, pages 33–58. find
  7. Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Jauvin. 2003. A neural probabilistic language model. Journal of machine learning research, 3(Feb):1137–1155. find
  8. Kaj Bostrom and Greg Durrett. 2020. Byte pair encoding is suboptimal for language model pretraining. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 4617–4624, Online. Association for Computational Linguistics. find
  9. Georgie Botev, Arya D. McCarthy, Winston Wu, and David Yarowsky. 2022. Deciphering and characterizing out-of-vocabulary words for morphologically rich languages. In Proceedings of the 29th International Conference on Computational Linguistics, pages 5309–5326, Gyeongju, Republic of Korea. International Committee on Computational Linguistics. find
  10. Marta R Costa-Jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, et al. 2022. No language left behind: Scaling human-centered machine translation. arXiv preprint arXiv:2207.04672. ↗
  11. Ryan Cotterell, Sabrina J. Mielke, Jason Eisner, and Brian Roark. 2018. Are all languages equally hard to language-model? In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 536–541, New Orleans, Louisiana. Association for Computational Linguistics. find
  12. Mathias Creutz and Krista Lagus. 2002. Unsupervised discovery of morphemes. In Proceedings of the ACL-02 Workshop on Morphological and Phonological Learning, pages 21–30. Association for Computational Linguistics. find
  13. Vikrant Dewangan, Bharath Raj S, Garvit Suri, and Raghav Sonavane. 2025. When every token counts: Optimal segmentation for low-resource language models. In Proceedings of the First Workshop on Language Models for Low-Resource Languages, pages 294–308, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics. find
  14. Fitsum Gaim, Wonsuk Yang, and Jong C. Park. 2022. GeezSwitch: Language identification in typologically related low-resourced East African languages. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 6578– 6584, Marseille, France. European Language Resources Association. find
  15. Leonidas Gee, Leonardo Rigutini, Marco Ernandes, and Andrea Zugarini. 2023. Multi-word tokenization for sequence compression. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 612– 621, Singapore. Association for Computational Linguistics. find
  16. Daniela Gerz, Ivan Vulić, Edoardo Ponti, Jason Naradowsky, Roi Reichart, and Anna Korhonen. 2018. Language modeling for morphologically rich languages: Character-aware modeling for word-level prediction. Transactions of the Association for Computational Linguistics, 6:451–465. find
  17. Andargachew Mekonnen Gezmu and Andreas Nürnberger. 2023. Morpheme-based neural machine translation models for low-resource fusion languages. ACM Transactions on Asian and Low-Resource Language Information Processing, 22(9):1–19. find
  18. Gebrearegawi Gebremariam Gidey, Hailay Kidu Teklehaymanot, and Gebregewergs Mezgebe Atsbha. 2024. Morphological synthesizer for Ge‘ez language: Addressing morphological complexity and resource limitations. In Proceedings of the Fifth Workshop on Resources for African Indigenous Languages @ LREC-COLING 2024, pages 94–106, Torino, Italia. ELRA and ICCL. find our card
  19. Dan Gillick, Cliff Brunk, Oriol Vinyals, and Amarnag Subramanya. 2016. Multilingual language processing from bytes. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1296–1306, San Diego, California. Association for Computational Linguistics. find
  20. Omer Goldman, Avi Caciularu, Matan Eyal, Kris Cao, Idan Szpektor, and Reut Tsarfaty. 2024. Unpacking tokenization: Evaluating text compression and its correlation with model performance. In Findings of the Association for Computational Linguistics: ACL 2024, pages 2274–2286, Bangkok, Thailand. Association for Computational Linguistics. find
  21. Naman Goyal, Cynthia Gao, Vishrav Chaudhary, Peng-Jen Chen, Guillaume Wenzek, Da Ju, Sanjana Krishnan, Marc’Aurelio Ranzato, Francisco Guzmán, and Angela Fan. 2022. The Flores-101 evaluation benchmark for low-resource and multilingual machine translation. Transactions of the Association for Computational Linguistics, 10:522–538. find
  22. Stig-Arne Grönroos, Sami Virpioja, Peter Smit, and Mikko Kurimo. 2014. Morfessor FlatCat: An HMM-based method for unsupervised and semi-supervised learning of morphology. In Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pages 1177– 1185, Dublin, Ireland. Dublin City University and Association for Computational Linguistics. find
  23. Eylon Gueta, Omer Goldman, and Reut Tsarfaty. 2023. Explicit morphological knowledge improves pre-training of language models for hebrew. arXiv preprint arXiv:2311.00658. ↗ our card
  24. Barry Haddow, Rachel Bawden, Antonio Valerio Miceli Barone, Jindřich Helcl, and Alexandra Birch. 2022. Survey of low-resource machine translation. Computational Linguistics, 48(3):673–732. find
  25. Valentin Hofmann, Hinrich Schuetze, and Janet Pierrehumbert. 2022. An embarrassingly simple method to mitigate undesirable properties of pretrained language model tokenizers. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 385–393, Dublin, Ireland. Association for Computational Linguistics. find
  26. Jue Hou, Anisia Katinskaia, Anh-Duc Vu, and Roman Yangarber. 2023. Effects of sub-word segmentation on performance of transformer language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7413–7425, Singapore. Association for Computational Linguistics. find
  27. Haris Jabbar. 2023. Morphpiece: A linguistic tokenizer for large language models. arXiv preprint arXiv:2307.07262. ↗
  28. Marcin Junczys-Dowmunt, Kenneth Heafield, Hieu Hoang, Roman Grundkiewicz, and Anthony Aue. 2018. Marian: Cost-effective high-quality neural machine translation in C++. In Proceedings of the 2nd Workshop on Neural Machine Translation and Generation, pages 129–135, Melbourne, Australia. Association for Computational Linguistics. find
  29. Taku Kudo and John Richardson. 2018. SentencePiece: A simple and language independent subword tokenizer and detokenizer for neural text processing. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 66–71, Brussels, Belgium. Association for Computational Linguistics. find
  30. Jungseob Lee, Hyeonseok Moon, Seungjun Lee, Chanjun Park, Sugyeong Eo, Hyunwoong Ko, Jaehyung Seo, Seungyoon Lee, and Heuiseok Lim. 2024. Length-aware byte pair encoding for mitigating over-segmentation in Korean machine translation. In Findings of the Association for Computational Linguistics: ACL 2024, pages 2287–2303, Bangkok, Thailand. Association for Computational Linguistics. find
  31. Maja Popović. 2017. chrF++: words helping character n-grams. In Proceedings of the Second Conference on Machine Translation, pages 612–618, Copenhagen, Denmark. Association for Computational Linguistics. find our card
  32. Paul Lerner and François Yvon. 2025. Unlike “likely”, “unlike” is unlikely: BPE-based segmentation hurts morphological derivations in LLMs. In Proceedings of the 31st International Conference on Computational Linguistics, pages 5181–5190, Abu Dhabi, UAE. Association for Computational Linguistics. find
  33. Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. 2020. COMET: A neural framework for MT evaluation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2685–2702, Online. Association for Computational Linguistics. find our card
  34. Jindřich Libovický and Jindřich Helcl. 2024. Lexically grounded subword segmentation. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 7403–7420, Miami, Florida, USA. Association for Computational Linguistics. find
  35. Alfréd Rényi. 1961. On measures of entropy and information. In Proceedings of the fourth Berkeley symposium on mathematical statistics and probability, volume 1: contributions to the theory of statistics, volume 4, pages 547–562. University of California Press. find
  36. Tomasz Limisiewicz, Jiří Balhar, and David Mareček. 2023. Tokenization impacts multilingual language modeling: Assessing vocabulary allocation and overlap across languages. In Findings of the Association for Computational Linguistics: ACL 2023, pages 5661–5681, Toronto, Canada. Association for Computational Linguistics. find
  37. Jonne Saleva and Constantine Lignos. 2021. The effectiveness of morphology-aware segmentation in low-resource neural machine translation. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Student Research Workshop, pages 164–174, Online. Association for Computational Linguistics. find
  38. Beso Mikaberidze, Temo Saghinadze, Guram Mikaberidze, Raphael Kalandadze, Konstantine Pkhakadze, Josef van Genabith, Simon Ostermann, Lonneke van der Plas, and Philipp Müller. 2024. A comparison of different tokenization methods for the Georgian language. In Proceedings of the 7th International Conference on Natural Language and Speech Processing (ICNLSP 2024), pages 199–208, Trento. Association for Computational Linguistics. find
  39. Gabriella F Scelta and Pilar Quezzaire-Belle. 2001. The comparative origin and usage of the ge’ez writing system of ethiopia. Unpublished manuscript, Boston University, Boston. Retrieved July, 25:2009. find
  40. Benjamin Minixhofer, Jonas Pfeiffer, and Ivan Vulić. 2023. CompoundPiece: Evaluating and improving decompounding performance of language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 343–359, Singapore. Association for Computational Linguistics. find
  41. Javad Nouri and Roman Yangarber. 2016. A novel evaluation method for morphological segmentation. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC‘16), pages 3102–3109, Portorož, Slovenia. European Language Resources Association (ELRA). find
  42. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318. find our card
  43. Hyunji Hayley Park, Katherine J. Zhang, Coleman Haley, Kenneth Steimel, Han Liu, and Lane Schwartz. 2021. Morphology matters: A multilingual language modeling analysis. Transactions of the Association for Computational Linguistics, 9:261–276. find
  44. Craig W Schmidt, Varshini Reddy, Haoran Zhang, Alec Alameddine, Omri Uzan, Yuval Pinter, and Chris Tanner. 2024. Tokenization is more than compression. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 678–702, Miami, Florida, USA. Association for Computational Linguistics. find
  45. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715– 1725, Berlin, Germany. Association for Computational Linguistics. find
  46. Hyopil Shin and Hyunjo You. 2009. Hybrid n-gram probability estimation in morphologically rich languages. In Proceedings of the 23rd Pacific Asia Conference on Language, Information and Computation, Volume 2, pages 511–520, Hong Kong. City University of Hong Kong. find
  47. Michaela Socolof, Jacob Louis Hoover, Richard Futrell, Alessandro Sordoni, and Timothy J. O’Donnell. 2022. Measuring morphological fusion using partial information decomposition. In Proceedings of the 29th International Conference on Computational Linguistics, pages 44–54, Gyeongju, Republic of Korea. International Committee on Computational Linguistics. find
  48. Xinying Song, Alex Salcianu, Yang Song, Dave Dopson, and Denny Zhou. 2021. Fast WordPiece tokenization. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 2089–2103, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics. find
  49. Ahmed Tawfik, Mahitab Emam, Khaled Essam, Robert Nabil, and Hany Hassan. 2019. Morphology-aware word-segmentation in dialectal Arabic adaptation of neural machine translation. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 11–17, Florence, Italy. Association for Computational Linguistics. find
  50. Yemane Tedla and Kazuhide Yamamoto. 2018. Morphological segmentation with lstm neural networks for tigrinya. Int. J. Nat. Lang. Comput, 7(2):29–44. find
  51. Avijit Thawani, Saurabh Ghanekar, Xiaoyuan Zhu, and Jay Pujara. 2023. Learn your tokens: Word-pooled tokenization for language modeling. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 9883–9893, Singapore. Association for Computational Linguistics. find
  52. Jörg Tiedemann. 2012. Parallel data, tools and interfaces in opus. In Lrec, volume 2012, pages 2214– 2218. Citeseer. find
  53. Thinh Truong, Yulia Otmakhova, Karin Verspoor, Trevor Cohn, and Timothy Baldwin. 2024. Revisiting subword tokenization: A case study on affixal negation in large language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 5082–5095, Mexico City, Mexico. Association for Computational Linguistics. find
  54. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, 30. find
  55. Changhan Wang, Kyunghyun Cho, and Jiatao Gu. 2020. Neural machine translation with byte-level subwords. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 9154–9160. find
  56. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45, Online. Association for Computational Linguistics. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/findings-2025-teklehaymanot-movoc-morphology-aware-subword-construction/