Semitic Root Encoding: Tokenization Based on the Templatic Morphology of Semitic Languages in NMT

What it is about

BPE and unigram tokenizers split words only into continuous strings, so they hide the root-and-template structure of Semitic words. Semitic Root Encoding (SRE) first runs a morphological analyser (CAMeL Tools, calima-msar) on each word and rewrites the stem as a root token (for example <KTB>) plus a template-stem token (for example 12u3), then applies SentencePiece BPE. In English-to-Arabic NMT with a BART-style model, SRE gains an average of 1.15 BLEU over BPE across four trials (27.54 against 26.39), and a small human ranking of 100 sentences by three evaluators favours SRE. The gain disappears at a 20,000-token vocabulary (27.62 against 28.03). Ablations show that random "roots" (Fake-SRE) lose more than 3 BLEU against real roots, and whole-stem tokens (Stem-SRE) lose too, which suggests the model uses the root and template separately. Generated words with a root and template that do not combine in Arabic are rare (2 in 9,669 sentences in one trial, 0 in another).

Ideas

• I1. A root-and-template table for tools/hbo: for each Hebrew lemma, store the root (consonant radicals) and the pattern (binyan and template) as two symbols, the way the paper's <KTB> + 12u3 pairs work. The lex and pre tables in the segmenter can hold these as integer IDs, not strings.
• I2. A dubious-stem check: generate root × pattern combinations and count how many occur in the MACULA lexicon. The paper's method (count the combinations that are not words, then review by hand) is a measure of how far the SLM over-generates in morphology.
• I3. A negative control for any root-based rule: replace the real roots with random consonant sets and check that the score falls. If it does not fall, the rule is memorising surface forms, not using morphology. This is the paper's Fake-SRE idea, and it fits the "a gate that cannot show red" rule.

For Mova

I1 goes into The Noun, the Adjective and the Numeral and The verb, and into the lex tables of tools/hbo. I2 is a measure for tools/hbo and macula dict. I3 is a control to add to any root-based test in tools/hbo. The NMT result itself does not apply, since Mova does not translate.

Topics: Machine translation

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.arabicnlp-main.3

@inproceedings{hatch2025semitic,
  title = {{Semitic Root Encoding: Tokenization Based on the Templatic Morphology of Semitic Languages in NMT}},
  author = {Hatch, Brendan T. and Richardson, Stephen D.},
  booktitle = {Proceedings of The Third Arabic Natural Language Processing Conference},
  year = {2025},
  doi = {10.18653/v1/2025.arabicnlp-main.3},
  url = {https://doi.org/10.18653/v1/2025.arabicnlp-main.3}
}

References

22 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ahmed Abdelali, Kareem Darwish, Nadir Durrani, and Hamdy Mubarak. 2016. Farasa: A fast and furious segmenter for arabic. In Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: Demonstrations, pages 11–16. find our card
2. Mohamed Alkaoud and Mairaj Syed. 2020. On the importance of tokenization in arabic embedding models. In Proceedings of the fifth Arabic natural language processing workshop, pages 119–129. find our card
3. Abdulrahman Almuhareb, Waleed Alsanie, and Abdulmohsen Al-Thubaity. 2019. Arabic word segmentation with long short-term memory neural networks and word embedding. IEEE Access, 7:12879–12887. https://themqm.org/ ↗
4. Manel Aloui, Hasna Chouikhi, Ghaith Chaabane, Haithem Kchaou, and Chehir Dhaouadi. 2024. 101 billion arabic words dataset. Preprint, arXiv:2405.01590. International Conference on Computational Linguistics: Technical Papers, pages 1398–1408, Osaka, Japan. ↗
5. The COLING 2016 Organizing Committee. find
6. Zaid Alyafeai, Maged S Al-shaibani, Mustafa Ghaleb, and Irfan Ahmad. 2023. Evaluating various tokenizers for arabic text classification. Neural Processing Letters, 55(3):2911–2933. find
7. Francisco Guzmán, Peng-Jen Chen, Myle Ott, Juan Pino, Guillaume Lample, Philipp Koehn, Vishrav Chaudhary, and Marc’Aurelio Ranzato. 2019. Two new evaluation datasets for low-resource machine translation: Nepali-english and sinhala-english. find
8. Chantal Amrhein and Rico Sennrich. 2021. How suitable are subword segmentation strategies for translating non-concatenative morphology? In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 689–705, Punta Cana, Dominican Republic. Association for Computational Linguistics. find
9. Taku Kudo. 2018. Subword regularization: Improving neural network translation models with multiple subword candidates. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 66–75, Melbourne, Australia. Association for Computational Linguistics. find
10. Aditi Chaudhary, Chunting Zhou, Lori Levin, Graham Neubig, David R. Mortensen, and Jaime Carbonell. 2018. Adapting word embeddings to new languages with morphological and phonological subword representations. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3285–3295, Brussels, Belgium. Association for Computational Linguistics. find
11. Taku Kudo and John Richardson. 2018. SentencePiece: A simple and language independent subword tokenizer and detokenizer for neural text processing. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 66–71, Brussels, Belgium. Association for Computational Linguistics. find
12. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7871–7880, Online. Association for Computational Linguistics. find
13. Anne N. De Roeck and Waleed Al-Fares. 2000. A morphologically sensitive clustering algorithm for identifying Arabic roots. In Proceedings of the 38th Annual Meeting of the Association for Computational Linguistics, pages 199–206, Hong Kong. Association for Computational Linguistics. find our card
14. Ahmed El-Kishky, Xingyu Fu, Aseel Addawood, Nahil Sobh, Clare Voss, and Jiawei Han. 2019. Constrained sequence-to-sequence Semitic root extraction for enriching word embeddings. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 88–96, Florence, Italy. Association for Computational Linguistics. find our card
15. Ossama Obeid, Nasser Zalmout, Salam Khalifa, Dima Taji, Mai Oudah, Bashar Alhafni, Go Inoue, Fadhl Eryani, Alexander Erdmann, and Nizar Habash. 2020. CAMeL tools: An open source python toolkit for Arabic natural language processing. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 7022–7032, Marseille, France. European Language Resources Association. find
16. Angela Fan, Shruti Bhosale, Holger Schwenk, Zhiyi Ma, Ahmed El-Kishky, Siddharth Goyal, Mandeep Baines, Onur Celebi, Guillaume Wenzek, Vishrav Chaudhary, Naman Goyal, Tom Birch, Vitaliy Liptchinsky, Sergey Edunov, Michael Auli, and Armand Joulin. 2021. Beyond english-centric multilingual machine translation. Journal of Machine Learning Research, 22(107):1–48. find
17. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics. find our card
18. Philip Gage. 1994. A new algorithm for data compression. The C Users Journal, 12(2):23–38. find
19. Arfath Pasha, Mohamed Al-Badrashiny, Mona Diab, Ahmed El Kholy, Ramy Eskander, Nizar Habash, Manoj Pooleery, Owen Rambow, and Ryan Roth. 2014. MADAMIRA: A fast, comprehensive tool for morphological analysis and disambiguation of Arabic. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 1094–1101, Reykjavik, Iceland. European Language Resources Association (ELRA). find our card
20. Naman Goyal, Cynthia Gao, Vishrav Chaudhary, Peng-Jen Chen, Guillaume Wenzek, Da Ju, Sanjana Krishnan, Marc’Aurelio Ranzato, Francisco Guzmán, and Angela Fan. 2022. The Flores-101 evaluation benchmark for low-resource and multilingual machine translation. Transactions of the Association for Computational Linguistics, 10:522–538. find
21. Francisco Guzmán, Houda Bouamor, Ramy Baly, and Nizar Habash. 2016. Machine translation evaluation for Arabic using morphologically-enriched embeddings. In Proceedings of COLING 2016, the 26th Maja Popović. 2015. chrF: character n-gram F-score for automatic MT evaluation. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 392–395, Lisbon, Portugal. Association for Computational Linguistics. Evaluation (LREC’12), pages 2214–2218, Istanbul, Turkey. European Language Resources Association (ELRA). find our card
22. Matt Post. 2018. A call for clarity in reporting BLEU scores. In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186– 191, Brussels, Belgium. Association for Computational Linguistics. find our card

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2025-hatch-semitic-root-encoding-tokenization-based/