Mova0.5 pre-release⌕
☰
home / docs / papers / Modern Hebrew / Modelling the Morphology of Verbal Paradigms: A Case Study in the Tokenization of Turkish and Hebrew

Modelling the Morphology of Verbal Paradigms: A Case Study in the Tokenization of Turkish and Hebrew

What it is about

The paper asks whether sentence embeddings encode verb paradigms (active, passive, causative, causative-passive) and how tokenisation changes that. It uses Blackbird Language Matrices: three complete context pairs and one incomplete pair, with four candidate answers. The sentences come from Universal Dependencies treebanks, with Turkish voice from Penn and Kenet and Hebrew binyan (PAAL, NIFAL, HIFIL, HUFAL) from HTB and IAHLTWiki, and Hebrew is unvocalised. A feed-forward model over averaged token embeddings is compared across AlephBERT, BERTurk and multilingual ELECTRA. Turkish succeeds with both atomic and fragmented tokenisation. For Hebrew the monolingual model reaches mean F1 0.835 and the multilingual character-level model 0.333, close to chance. On a synthetic verb-only set the multilingual model improves but still lags, and the main confusion is causative against causative-passive. The authors conclude that for Hebrew the form must stay whole or be cut at morpheme boundaries.

Ideas

For Mova

I1 and I3 are test sets for tools/hbo and for the paradigm tables in 05b. Verb paradigms: the strong verb in seven binyanim and the weak roots. I2 is a design check for the segmenter in tools/hbo, and the verb chapter 05. The verb: system, binyanim, tense and use takes the binyan facts. The paper's embeddings are not used by Mova.

Topics: Universal Dependencies · Treebanks

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.sigturk-1.8

@inproceedings{samo2026modelling,
  title = {{Modelling the Morphology of Verbal Paradigms: A Case Study in the Tokenization of Turkish and Hebrew}},
  author = {Samo, Giuseppe and Merlo, Paola},
  booktitle = {Proceedings of the Second Workshop Natural Language Processing for Turkic Languages (SIGTURK 2026)},
  year = {2026},
  doi = {10.18653/v1/2026.sigturk-1.8},
  url = {https://doi.org/10.18653/v1/2026.sigturk-1.8}
}

References

54 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Thao Anh Dang, Limor Raviv, and Lukas Galke. 2024. Tokenization and morphology in multilingual language models: A comparative analysis of mt5 and byt5. Preprint, arXiv:2410.11627. ↗
  2. Orevaoghene Ahia, Sachin Kumar, Hila Gonen, Jungo Kasai, David Mortensen, Noah Smith, and Yulia Tsvetkov. 2023. Do all languages cost the same? tokenization in the era of commercial language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 9904–9923, Singapore. Association for Computational Linguistics. find
  3. Marie-Catherine De Marneffe, Christopher D Manning, Joakim Nivre, and Daniel Zeman. 2021. Universal dependencies. Computational linguistics, 47(2):255– 308. find our card
  4. Aixiu An, Chunyang Jiang, Maria A. Rodriguez, Vivi Nastase, and Paola Merlo. 2023. BLM-AgrF: A new French benchmark to investigate generalization of agreement in neural networks. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 1363– 1374, Dubrovnik, Croatia. find
  5. Miguel Domingo, Mercedes Garcıa-Martınez, Alexandre Helle, Francisco Casacuberta, and Manuel Herranz. 2019. How much does tokenization affect neural machine translation? Preprint, arXiv:1812.08621. ↗
  6. Maya Arad. 2005. Roots and Patterns: Hebrew Morpho-Syntax. Springer, New York. find
  7. Nomi Erteschik-Shir, Lena Ibnbari, and Sharon Taube. 2013. Missing objects as topic drop. Lingua, 136:145–169. find
  8. Ehsaneddin Asgari, Yassine El Kheir, and Mohammad Ali Sadraei Javaheri. 2025. Morphbpe: A morpho-aware tokenizer bridging linguistic complexity for efficient llm training across morphologies. Preprint, arXiv:2502.00894. ↗
  9. Joseph Fruchter and Alec Marantz. 2015. Decomposition, lookup, and recombination: Meg evidence for the full decomposition model of complex visual word recognition. Brain and Language, 143:81–96. find
  10. Duygu Ataman and Marcello Federico. 2018. An evaluation of two vocabulary reduction methods for neural machine translation. In Proceedings of the 13th Conference of the Association for Machine Translation in the Americas (Volume 1: Research Track), pages Prakhar Gautam, Jitendra Singh Thakur, and Ashish Mishra. 2024. Subject–verb agreement error handling using rnn architectures. In International Conference on Innovations in Computational Intelligence and Computer Vision, pages 215–224. Springer. find
  11. Jaklin Kornfilt. 1997. Turkish. Routledge, London. find
  12. Omer Goldman and Reut Tsarfaty. 2022. Morphology without borders: Clause-level morphology. Transactions of the Association for Computational Linguistics, 10:1455–1472. find
  13. Beth Levin. 1993. English Verb Classes and Alternations A Preliminary Investigation. University of Chicago Press, Chicago and London. find
  14. Thomas L Griffiths, Jian-Qiao Zhu, Erin Grant, and R Thomas McCoy. 2024. Bayes in the age of intelligent machines. Current Directions in Psychological Science, 33(5):283–291. find
  15. Tal Linzen and Marco Baroni. 2021. Syntactic structure from deep learning. Annual Review of Linguistics, 7(1):195–212. find
  16. John J. McCarthy. 1979. Formal Problems in Semitic Phonology and Morphology. Ph.D. thesis, Massachusetts Institute of Technology. find
  17. Eylon Gueta, Omer Goldman, and Reut Tsarfaty. 2023. Explicit morphological knowledge improves pre-training of language models for hebrew. Preprint, arXiv:2311.00658. ↗ our card
  18. Ryan T McDonald, Joakim Nivre, Yvonne Quirmbach-Brundage, Yoav Goldberg, Dipanjan Das, Kuzman Ganchev, Keith B Hall, Slav Petrov, Hao Zhang, Oscar Täckström, and 1 others. 2013. Universal dependency annotation for multilingual parsing. In Proc. of ACL. find
  19. Aslı Göksel and Celia Kerslake. 2005. Turkish: A Comprehensive Grammar. Routledge, London. find
  20. Martin Haspelmath, Andreea Calude, Michael Spagnol, Heiko Narrog, and Elif Bamyaci. 2014. Coding causal–noncausal verb alternations: A form– frequency correspondence explanation1. Journal of linguistics, 50(3):587–625. find
  21. Hasan Mesut Meral. 2014. Silent objects and topic drop in turkish. Dilbilim Araştırmaları A. Sumru Özsoy Armağanı, 25:131–145. find
  22. Zachary William Hopton, Yves Scherrer, and Tanja Samardzic. 2025. Functional lexicon in subword tokenization. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 7839–7853, Albuquerque, New Mexico. Association for Computational Linguistics. find
  23. Paola Merlo. 2023a. Blackbird language matrices (BLM), a new task for rule-like generalization in neural networks: Can Large Language Models pass the test? In Findings of the 2023 Conference on Empirical Methods in Natural Language Processing. find
  24. Paola Merlo. 2023b. Blackbird language matrices (blm), a new task for rule-like generalization in neural networks: Motivations and formal specifications. Preprint, arXiv:2306.11444. ↗
  25. Chunyang Jiang, Giuseppe Samo, Vivi Nastase, and Paola Merlo. 2024. BLM-it - blackbird language matrices for Italian: A CALAMITA challenge. In Proceedings of the 10th Italian Conference on Computational Linguistics (CLiC-it 2024), pages 1135– 1143, Pisa, Italy. CEUR Workshop Proceedings. find
  26. Paola Merlo, Aixiu An, and Maria A. Rodriguez. 2022. Blackbird’s language matrices (BLMs): a new benchmark to investigate disentangled generalisation in neural networks. arXiv cs.CL.2205.10866. find
  27. Jaap Jumelet, Leonie Weissweiler, Joakim Nivre, and Arianna Bisazza. 2025. MultiBLiMP 1.0: A Massively Multilingual Benchmark of Linguistic Minimal Pairs. arXiv preprint arXiv:2504.02768. ↗
  28. Benjamin Muller, Antonios Anastasopoulos, Benoît Sagot, and Djamé Seddah. 2021. When being unseen from mBERT is just the beginning: Handling new languages with multilingual language models. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 448–462, Online. Association for Computational Linguistics. find
  29. Vani Kanjirangat, Tanja Samardžić, Ljiljana Dolamic, and Fabio Rinaldi. 2023. Optimizing the size of subword vocabularies in dialect classification. In Tenth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial 2023), pages 14–30, Dubrovnik, Croatia. Association for Computational Linguistics. find
  30. Mihai Nadas, Laura Diosan, and Andreea Tomescu. 2025. Synthetic data generation using large language models: Advances in text and code. arXiv preprint arXiv:2503.14023. ↗
  31. Katharina Kann, Alex Warstadt, Adina Williams, and Samuel R. Bowman. 2019. Verb argument structure alternations in word and sentence embeddings. In Proceedings of the Society for Computation in Linguistics (SCiL) 2019, pages 287–297. find
  32. Vivi Nastase, Giuseppe Samo, Chunyang Jiang, and Paola Merlo. 2024a. Exploring italian sentence embeddings properties through multi-tasking. In Proceedings of the Tenth Italian Conference on Computational Linguistics (Clic-It 2024), pages 1–10. find
  33. Itamar Kastner. 2019. Templatic morphology as an emergent property: Roots and functional heads in hebrew. Natural Language & Linguistic Theory, 37:571–619. find
  34. Vivi Nastase, Giuseppe Samo, Chunyang Jiang, and Paola Merlo. 2024b. Exploring syntactic information in sentence embeddings through multilingual subject-verb agreement. In Proceedings of the Tenth Italian Gregory Key. 2013. The morphosyntax of the Turkish caustive construction. The University of Arizona. find
  35. Conference on Computational Linguistics (Clic-It 2024), pages 1–13. find
  36. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715–1725, Berlin, Germany. Association for Computational Linguistics. find
  37. Garrett Nicolai, Eleanor Chodroff, Frederic Mailhot, and Çağrı Çöltekin, editors. 2024. Proceedings of the 21st SIGMORPHON workshop on Computational Research in Phonetics, Phonology, and Morphology. Association for Computational Linguistics, Mexico City, Mexico. find
  38. Rafael Soto Setzke. 2024. Linguistic paradigms as cognitive entities: A domain-general approach. Yearbook of the German Cognitive Linguistics Association, 12(1):73–94. find
  39. Malvina Nissim, Danilo Croce, Viviana Patti, Pierpaolo Basile, Giuseppe Attanasio, Elio Musacchio, Matteo Rinaldi, Federico Borazio, Maria Francis, Jacopo Gili, Daniel Scalena, Begoña Altuna, Ekhi Azurmendi, Valerio Basile, Luisa Bentivogli, Arianna Bisazza, Marianna Bolognesi, Dominique Brunato, Tommaso Caselli, and 62 others. 2025. Challenging the abilities of large language models in italian: a community initiative. Preprint, arXiv:2512.04759. ↗
  40. Avi Shmidman and Aynat Rubinstein. 2024. Computational methods for the analysis of complementizer variability in language and literature: The case of Hebrew “she-” and “ki”. In Proceedings of the 4th International Conference on Natural Language Processing for Digital Humanities, pages 294–307, Miami, USA. Association for Computational Linguistics. find our card
  41. Joakim Nivre. 2015. Towards a universal grammar for natural language processing. In International conference on intelligent text processing and computational linguistics, pages 3–16. Springer. find
  42. Cagri Toraman, Eyup Halit Yilmaz, Furkan Şahin , uç, and Oguzhan Ozcelik. 2023. Impact of tokenization on language models: An analysis for turkish. ACM Transactions on Asian and Low-Resource Language Information Processing, 22(4):1–21. find
  43. Kemal Oflazer. 1993. Two-level description of Turkish morphology. In Sixth Conference of the European Chapter of the Association for Computational Linguistics, Utrecht, The Netherlands. Association for Computational Linguistics. find
  44. Reut Tsarfaty. 2004. ‘binyanim ba’avir’: An investigation of Aspect Semantics in Modern Hebrew. Master’s thesis, Universiteit van Amsterdam, Institute for Logic Language and Computation, December. find
  45. Riccardo Orlando, Luca Moroni, Pere-Lluís Huguet Cabot, Simone Conia, Edoardo Barba, Sergio Orlandini, Giuseppe Fiameni, and Roberto Navigli. 2024. Minerva LLMs: The first family of large language models trained from scratch on Italian data. In Proceedings of the Tenth Italian Conference on Computational Linguistics (CLiC-it 2024), pages 707–719, Pisa, Italy. CEUR Workshop Proceedings. find
  46. Reut Tsarfaty. 2013. A unified morpho-syntactic scheme of stanford dependencies. In Proc. of ACL. find
  47. Anne Vainikka and Yonata Levy. 1999. Empty subjects in finnish and hebrew. Natural Language & Linguistic Theory, 17(3):613–671. find
  48. Alex Warstadt, Yu Cao, Ioana Grosu, Wei Peng, Hagen Blix, Yining Nie, Anna Alsop, Shikha Bordia, Haokun Liu, Alicia Parrish, Sheng-Fu Wang, Jason Phang, Anhad Mohananey, Phu Mon Htut, Paloma Jeretic, and Samuel R. Bowman. 2019. Investigating BERT’s knowledge of language: Five analysis methods with NPIs. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2877–2887, Hong Kong, China. Association for Computational Linguistics. find
  49. Mattia Proietti, Gianluca Lebani, and Alessandro Lenci. 2022. Does BERT recognize an agent? modeling Dowty’s proto-roles with contextual embeddings. In Proceedings of the 29th International Conference on Computational Linguistics, pages 4101–4112, Gyeongju, Republic of Korea. International Committee on Computational Linguistics. find
  50. Nived Rajaraman, Jiantao Jiao, and Kannan Ramchandran. 2024. Toward a theory of tokenization in llms. Preprint, arXiv:2404.08335. ↗
  51. Tanja Samardžić and Paola Merlo. 2018. The probability of external causation: An empirical account of crosslinguistic variation in lexical causatives. Linguistics, 56(5):895–938. find
  52. Alex Warstadt, Alicia Parrish, Haokun Liu, Anhad Mohananey, Wei Peng, Sheng-Fu Wang, and Samuel R. Bowman. 2020. BLiMP: The benchmark of linguistic minimal pairs for English. Transactions of the Association for Computational Linguistics, 8:377– 392. find our card
  53. Giuseppe Samo, Vivi Nastase, Chunyang Jiang, and Paola Merlo. 2023. BLM-s/lE: A structured dataset of English spray-load verb alternations for testing generalization in LLMs. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 12276–12287, Singapore. Association for Computational Linguistics. find
  54. Michael Wilson, Jackson Petty, and Robert Frank. 2023. How abstract is linguistic generalization in large language models? experiments with argument structure. Transactions of the Association for Computational Linguistics, 11:1377–1395. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2026-samo-modelling-morphology-verbal-paradigms-case/