Morphemes without Borders: Evaluating Root–Pattern Morphology in Arabic Tokenizers and LLMs

What it is about

The authors ask whether the tokenizers of seven LLMs (GPT-4, GPT-4o, LLaMA 3, Qwen 3, Cohere, and the Arabic-centric Fanar and ALLaM) keep Arabic morphemes intact. They compare them with gold segmentations in two undiacritized corpora (ATB3 for MSA and BOLT for Egyptian), and with the CAMeL and Farasa segmenters. The analysers reach fertility 1–2; GPT-4 over-segments, with fertility above 3 and boundary precision near 23%. A new productivity test uses real roots (130 root-pattern forms over 13 patterns) and nonce roots (20 roots by 5 patterns, 100 combinations) with 0-shot and 1-shot prompts. Alignment does not predict generation: GPT-4 and GPT-4o lead the tasks despite weak alignment, while ALLaM and Fanar fail on nonce words, which the authors read as memorisation.

Ideas

• I1. A nonce-root test for Mova: take consonant strings that are not Hebrew roots, combine them with a binyan or pattern table, and check whether a rule-based generator gets them all right. A memorising method should fail. Use it as a held-out check for pattern rules in The weak verbs.
• I2. Report the Morpheme Coverage Rate (the share of gold morphemes kept whole inside tokens) next to boundary F1 when measuring tools/hbo against gold morphemes. MCR gives an upper bound on how much of a morpheme survives segmentation.
• I3. Do not use boundary alignment as the quality measure of a segmenter variant. The paper finds alignment does not predict downstream performance, so tools/hbo variants should be compared on parse accuracy as well.

For Mova

I1 goes into The weak verbs as a held-out check, once a pattern table exists. I2 and I3 are measures for tools/hbo, recorded in 13. Biblical Hebrew in Mova: the tools. Only the summary above is kept; the paper's data is not carried into the repository.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.63317/3bk9j7feheoq

@inproceedings{alakeel2026morphemes,
  title = {{Morphemes without Borders: Evaluating Root–Pattern Morphology in Arabic Tokenizers and LLMs}},
  author = {Alakeel, Yara Yousif and Qwaider, Chatrine and Aldarmaki, Hanan and Alqahtani, Sawsan},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/3bk9j7feheoq},
  url = {https://doi.org/10.63317/3bk9j7feheoq}
}

References

38 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ummar Abbas et al. 2025. Fanar: An arabic-centric multimodal generative ai platform. arXiv preprint arXiv:2501.13944. ↗
2. Josh Achiam et al. 2024. Gpt-4 technical report. arXiv preprint arXiv:2303.08774. ↗
3. Arash Ahmadian et al. 2025. Command a: An enterprise-ready large language model. arXiv preprint arXiv:2504.00698. ↗
4. Mehdi Ali, Michael Fromm, Klaudia Thellmann, Richard Rutmann, Max Lübbering, Johannes Leveling, Katrin Klug, Jan Ebert, Niclas Doll, Jasper Buschhoff, et al. 2024. Tokenizer choice for llm training: Negligible or crucial? In Findings of the Association for Computational Linguistics: NAACL 2024, pages 3907–3924. find
5. Catherine Arnett and Benjamin Bergen. 2025. Why do language models perform worse for morphologically complex languages? In Proceedings of the 31st International Conference on Computational Linguistics, pages 6607–6623, Abu Dhabi, UAE. Association for Computational Linguistics. find
6. Catherine Arnett, Marisa Hudspeth, and Brendan O’Connor. 2025. Evaluating morphological alignment of tokenizers in 70 languages. In Proceedings of the ICML 2025 Tokenization Workshop (TokShop). International Conference on Machine Learning. find
7. Ehsaneddin Asgari, Yassine El Kheir, and Mohammad Ali Sadraei Javaheri. 2025. Morphbpe: A morpho-aware tokenizer bridging linguistic complexity for efficient llm training across morphologies. arXiv preprint arXiv:2502.00894. ↗
8. M Saiful Bari, Yazeed Alnumay, Norah Alzahrani, Nouf Alotaibi, Hisham Alyahya, AlRashed Al-Rashed, Faisal Mirza, Shaykhah Alsubaie, Hassan Alahmed, Ghadah Alabduljabbar, Raghad Alkhathran, Yousef Almushayqih, Raneem Alnajim, Salman I Alsubaihi, Maryam Al Mansour, Saad Hassan, Majed Alrubaian, Ali Alammari, Zaki Alawami, Abdulmohsen Al-Thubaity, Ahmed Abdelali, Jeril Kuriakose, Abdalghani Abujabal, Nora Al-Twairesh, Areeb Alowisheq, and Haidar Khan. 2025. Allam: Large language models for arabic and english. In International Conference on Learning Representations, volume 2025, pages 34179–34214. find
9. Kenneth R Beesley and Lauri Karttunen. 2003. Finite-state morphology: Xerox tools and techniques. CSLI, Stanford, pages 359–375. find
10. Kaj Bostrom and Greg Durrett. 2020. Byte pair encoding is suboptimal for language model pretraining. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 4617– 4624, Online. Association for Computational Linguistics. find
11. Anh Dang, Limor Raviv, and Lukas Galke. 2024. Morphology matters: Probing the cross-linguistic morphological generalization abilities of large language models through a wug test. In 13th edition of the Workshop on Cognitive Modeling and Computational Linguistics (CMCL 2024), pages 177–188. Association for Computational Linguistics (ACL). find
12. Kareem Darwish and Hamdy Mubarak. 2016. Farasa: A new fast and accurate arabic word segmenter. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 1070–1074. find
13. Philip Gage. 1994. A new algorithm for data compression. C Users J., 12(2):23–38. find
14. Bar Gazit, Shaltiel Shmidman, Avi Shmidman, and Yuval Pinter. 2025. Splintering nonconcatenative languages for better tokenization. In Findings of the Association for Computational Linguistics: ACL 2025, pages 22405–22417, Vienna, Austria. Association for Computational Linguistics. find
15. Aaron Grattafiori et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783. ↗
16. Nizar Y Habash. 2010. Introduction to Arabic natural language processing. Morgan & Claypool Publishers. find
17. Valentin Hofmann, Hinrich Schuetze, and Janet Pierrehumbert. 2022. An embarrassingly simple method to mitigate undesirable properties of pretrained language model tokenizers. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 385–393, Dublin, Ireland. Association for Computational Linguistics. find
18. Valentin Hofmann, Leonie Weissweiler, David R Mortensen, Hinrich Schütze, and Janet B Pierrehumbert. 2025. Derivational morphology reveals analogical generalization in large language models. Proceedings of the National Academy of Sciences, 122(19):e2423232122. find
19. Binyuan Hui et al. 2025. Qwen2.5 technical report. arXiv preprint arXiv:2409.12186. ↗
20. Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276. ↗
21. Mete Ismayilzada, Defne Circi, Jonne Sälevä, Hale Sirin, Abdullatif Köksal, Bhuwan Dhingra, Antoine Bosselut, Duygu Ataman, and Lonneke Van Der Plas. 2025. Evaluating morphological compositional generalization in large language models. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics. find
22. Haris Jabbar. 2024. Morphpiece : A linguistic tokenizer for large language models. arXiv preprint arXiv:2307.07262. ↗
23. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. 2023. Mistral 7b. find
24. Taku Kudo. 2018. Subword regularization: Improving neural network translation models with multiple subword candidates. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 66–75, Melbourne, Australia. Association for Computational Linguistics. find
25. Sabrina J. Mielke, Zaid Alyafeai, Elizabeth Salesky, Colin Raffel, Manan Dey, Matthias Gallé, Arun Raja, Chenglei Si, Wilson Y. Lee, Benoît Sagot, and Samson Tan. 2021. Between words and characters: A brief history of open-vocabulary modeling and tokenization in nlp. find
26. Ossama Obeid, Nasser Zalmout, Salam Khalifa, Dima Taji, Mai Oudah, Bashar Alhafni, Go Inoue, Fadhl Eryani, Alexander Erdmann, and Nizar Habash. 2020. CAMeL tools: An open source python toolkit for Arabic natural language processing. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 7022–7032, Marseille, France. European Language Resources Association. find
27. Artidoro Pagnoni, Ram Pasunuru, Pedro Rodriguez, John Nguyen, Benjamin Muller, Margaret Li, Chunting Zhou, Lili Yu, Jason Weston, Luke Zettlemoyer, et al. 2024. Byte latent transformer: Patches scale better than tokens. arXiv preprint arXiv:2412.09871. ↗
28. Aleksandar Petrov, Emanuele La Malfa, Philip Torr, and Adel Bibi. 2023. Language model tokenizers introduce unfairness between languages. Advances in neural information processing systems, 36:36963–36990. find
29. Phillip Rust, Jonas Pfeiffer, Ivan Vulić, Sebastian Ruder, and Iryna Gurevych. 2021. How good is your tokenizer? on the monolingual performance of multilingual language models. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 3118–3135, Online. Association for Computational Linguistics. find
30. Craig W Schmidt, Varshini Reddy, Chris Tanner, and Yuval Pinter. 2025. Boundless byte pair encoding: Breaking the pre-tokenization barrier. arXiv preprint arXiv:2504.00178. ↗
31. Mike Schuster and Kaisuke Nakajima. 2012. Japanese and korean voice search. In 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 5149–5152. find
32. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715–1725, Berlin, Germany. Association for Computational Linguistics. find
33. Jean Seo, Jaeyoon Kim, SungJoo Byun, and Hyopil Shin. 2025. How does a language-specific tokenizer affect llms? arXiv preprint arXiv:2502.12560. ↗
34. Leonie Weissweiler, Valentin Hofmann, Anjali Kantharuban, Anna Cai, Ritam Dutt, Amey Hengle, Anubha Kabra, Atharva Kulkarni, Abhishek Vijayakumar, Haofei Yu, Hinrich Schuetze, Kemal Oflazer, and David Mortensen. 2023. Counting the bugs in ChatGPT’s wugs: A multilingual investigation into the morphological capabilities of a large language model. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6508–6524, Singapore. Association for Computational Linguistics. find our card
35. Leonie Weissweiler, Kyle Mahowald, and Adele Goldberg. 2025. Linguistic generalizations are not rules: Impacts on evaluation of lms. arXiv preprint arXiv:2502.13195. ↗
36. Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Jeff Klingner, Apurva Shah, Melvin Johnson, Xiaobing Liu, ukasz Kaiser, Stephan Gouws, Yoshikiyo Kato, Taku Kudo, Hideto Kazawa, Keith Stevens, George Kurian, Nishant Patil, Wei Wang, Cliff Young, Jason Smith, Jason Riesa, Alex Rudnick, Oriol Vinyals, Greg Corrado, Macduff Hughes, and Jeffrey Dean. 2016. Google’s neural machine translation system: Bridging the gap between human and machine translation. 7. find
37. Language Resource References Maamouri, Mohamed and Bies, Ann and Buckwalter, Tim and Mekki, Wigdan. 2010. Arabic Treebank: Part 3 v 3.2 LDC2010T08. ISLRN 770-467-034-042-0. Web Download. Philadelphia: Linguistic Data Consortium. find
38. Maamouri, Mohamed and et al. 2021. BOLT Egyptian Arabic Treebank - Conversational Telephone Speech LDC2021T12. ISLRN 430-645-589-448-0. Web Download. Philadelphia: Linguistic Data Consortium. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/lingbulk-2026-alakeel-morphemes-without-borders-evaluating-root/