Mova0.5 pre-release⌕
☰
home / docs / papers / Semitic languages / A Comprehensive Full-Form Lexicon for Arabic NLP and Speech Technology

A Comprehensive Full-Form Lexicon for Arabic NLP and Speech Technology

What it is about

The paper presents ArabLEX, a full-form lexicon of about 570 million TSV lines, with modules for general vocabulary (about 83 million), personal names (about 218 million), foreign names (about 226 million) and place names (about 6 million). Each entry has up to 25 fields: IPA and CARS phonemic transcription, stem and clitics, gender, number, case, person, definiteness, root and lemma, plus orthographic variants. The argument rests on the graphemic underrepresentation of Arabic: the script has seven layers, but everyday text writes only the first two or three, so one consonant string can stand for many wordforms. The authors contrast full-form lexicons, which are large (ArabLEX is over 59 GB) and static, with morphological generators, which are small (CAMeL Morph is 80 MB) and can be updated by rules. They cite a survey in which often 50%, and sometimes 80%, of words were mispronounced by speech systems.

Ideas

For Mova

I1 goes into the unpointed mode of tools/hbo and into 1. Script and pointing. I2 goes into tools/hbo (the lex table) and The lexicon. I3 goes into tools/ocrfix and 1. Script and pointing. The paper's data is not used; it is Arabic, and Mova does not parse Arabic yet.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.63317/2gbvmmu4ix5e

@inproceedings{haralambous2026comprehensive,
  title = {{A Comprehensive Full-Form Lexicon for Arabic NLP and Speech Technology}},
  author = {Haralambous, Yannis and Halpern, Jack},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/2gbvmmu4ix5e},
  url = {https://doi.org/10.63317/2gbvmmu4ix5e}
}

References

49 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. A. Abdelali, K. Darwish, N. Durrani, and H. Mubarak. 2016. Farasa: A fast and furious segmenter for Arabic. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations, San Diego, CA, pages 11–16. Association for Computational Linguistics. find our card
  2. F. Alshargi, S. Dibas, S. Alkhereyf, R. Faraj, B. Abdulkareem, S. Yagi, O. Kacha, N. Habash, and O. Rambow. 2019. Morphologically Annotated Corpora for Seven Arabic Dialects: Taizi, Sanaani, Najdi, Jordanian, Syrian, Iraqi and Moroccan. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 137–147. find our card
  3. H. AlShuhayeb, B. Minaei-Bidgoli, M. E. Shenassa, and S. Hossayni. 2023. Noor-Ghateh: A Benchmark Dataset for Evaluating Arabic Word Segmenters in Hadith Domain. ArXiv preprint. find
  4. M. Attia, P. Pecina, L. Tounsi, A. Toral, and J. van Genabith. 2011. A lexical database for modern standard Arabic interoperable with a finite state morphological transducer. In Systems and Frameworks for Computational Morphology. find
  5. Second International Workshop, SFCM 2011, Zurich, volume 100 of Communications in Computer and Information Science. Springer. find
  6. M. Boudchiche, A. Mazroui, M. Ould Abdallahi Ould Bebah, A. Lakhouaja, and A. Boudlal. 2017. AlKhalil Morpho Sys 2: A robust Arabic morphosyntactic analyzer. Journal of King Saud University – Computer and Information Sciences, 29(2):141–146. find
  7. T. Buckwalter. 2002. Buckwalter Arabic Morphological Analyzer Version 1.0. Linguistic Data Consortium, University of Pennsylvania. 2002, LDC Catalog No.: LDC2002L49. find
  8. T. Buckwalter. 2011. A Frequency Dictionary of Arabic. Core Vocabulary for Learners. Routledge. find
  9. J. Carbonell, S. Klein, D. Miller, M. Steinbaum, T. Grassiany, and J. Frei. 2006. Context-Based Machine Translation. In Proceedings of the 7th Conference of the Association for Machine Translation in the Americas: Technical Papers, Cambridge, MA, pages 19–28. Association for Machine Translation in the Americas. find
  10. R. Cotterell and H. Schütze. 2015. Morphological word-embeddings. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1287–1292, Denver, Colorado. Association for Computational Linguistics. find
  11. J. Dichy. 2017. The Analytics of Writing, Exemplified by Arabic, the Youngest of the Semitic Scripts. In Approaches to the History and Dialectology of Arabic, in Honour of Pierre Larcher, pages 29–56, Leiden, Boston. Brill. find
  12. S. Feng, M. Tu, R. Xia, C. Huang, and Y. Wang. 2023. Language-universal phonetic encoder for low-resource speech recognition. arXiv. ArXiv preprint. find
  13. N. Gadelli. 2015. The morphological integration of loanwords into Modern Standard Arabic: Towards a morphological categorization of loanwords. Ph.D. thesis, Lund University, Sweden. find
  14. N. Habash, F. Eryani, S. Khalifa, O. Rambow, D. Abdulrahim, A. Erdmann, R. Faraj, W. Zaghouani, H. Bouamor, N. Zalmout, S. Hassan, F. Al-Shargi, S. Alkhereyf, B. Abdulkareem, R. Eskander, M. Salameh, and H. Saddiki. 2018. Unified guidelines and resources for Arabic dialect orthography. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Association (ELRA). find
  15. N. Habash, O. Rambow, and R. Roth. 2009. MADA+TOKAN: A toolkit for Arabic tokenization, diacritization, morphological disambiguation, POS 150 tagging, stemming and lemmatization. In Proceedings of the Second International Conference on Arabic Language Resources and Tools, Cairo. find
  16. J. Halpern. 2009a. CJKI Arabic Romanization System (CARS). In Romanization of Arabic Names: Proceedings of the International Symposium on Arabic Transliteration Standard: Challenges and Solutions, Abu Dhabi, UAE, 15–16 December 2009. Ministry of Culture, Youth and Community Development. find
  17. J. Halpern. 2009b. Lexicon-Driven Approach to the Recognition of Arabic Named Entities. In Proceedings of the Second International Conference on Arabic Language Resources and Tools, Cairo. find
  18. J. Halpern. 2009c. Word stress and vowel neutralization in modern standard Arabic. In Proceedings of the Second International Conference on Arabic Language Resources and Tools, Cairo, Egypt, April. The MEDAR Consortium. find
  19. J. Halpern. 2019. Lexicographic Criteria for Selecting Multiword Units for MT Lexicons. find
  20. J. Halpern. 2020. Enhancing Arabic Speech Technology with Comprehensive Arabic Training Lexicon. find
  21. Y. Haralambous. 2021. Breaking Arabic: the creative inventiveness of Uyghur script reforms. Design Regression. find
  22. Y. Haralambous. 2024. A Course in Natural Language Processing. Springer. find
  23. E. Haugen. 1972. Schizoglossia and the Linguistic Norm. In Studies by Einar Haugen, pages 441–445. De Gruyter Mouton, Berlin, Boston. find
  24. International Phonetic Association. 1999. Handbook of the International Phonetic Association: A Guide to the Use of the International Phonetic Alphabet. Cambridge University Press. find
  25. Ch. Khairallah, S. Khalifa, R. Marzouk, M. Nassar, and N. Habash. 2024. Camel morph MSA: A large-scale open-source morphological analyzer for Modern Standard Arabic. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 2683–2691, Torino, Italia. ELRA and ICCL. find our card
  26. P. Koehn. 2020. Neural Machine Translation. Cambridge University Press, Cambridge, UK. find
  27. X. Lu, P. West, R. Zellers, R. Le Bras, C. Bhagavatula, and Y. Choi. 2021. Neurologic decoding: (un)supervised neural text generation with predicate logic constraints. ArXiv preprint. find
  28. M. Maamouri, A. Bies, T. Buckwalter, and W. Mekki. 2004. The Penn Arabic Treebank: Building a Large-Scale Annotated Arabic Corpus. In NEM-LAR Conference on Arabic Language Resources and Tools, Cairo, Egypt, pages 102–109. find
  29. G. Mejdell. 2014. Luġat al-ʾumm and al-luġa al-ʾumm - the ‘mother tongue’ in the Arabic context. In Arabic and Semitic Linguistics Contextualized, pages 214–226. Harrassovitz. find
  30. D. Meletis and Ch. Dürscheid. 2022. Writing Systems and Their Use. An Overview of Grapholinguistics, volume 369 of Trends in Linguistics. de Gruyter. find
  31. J.R. Osborn. 2017. Letters of Light. Arabic Script in Calligraphy, Print, and Digital Design. Harvard University Press. find
  32. A. Pasha, M. Al-Badrashiny, M. Diab, A. El Kholy, R. Eskander, N. Habash, M. Pooleery, O. Rambow, and R. M. Roth. 2014. MADAMIRA: A Fast, Comprehensive Tool for Morphological Analysis and Disambiguation of Arabic. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), Reykjavik, Iceland, May. European Language Resources Association (ELRA). find our card
  33. C. Pinon. 2017. Intégrer les variations dans l’enseignement de l’arabe langue étrangère: enjeux et méthodes. In Arabe standard et variations régionales. Quelle(s) politique(s) linguistique(s)? Quelle(s) didactiques)? Éditions des archives contemporaines. find
  34. K. C. Ryding. 2005. A Reference Grammar of Modern Standard Arabic. Cambridge University Press, Cambridge, UK. find
  35. M. S. S. Sawalha. 2011. Open-source resources and standards for Arabic word structure analysis: Fine-grained morphological analysis of Arabic text corpora. Ph.D. thesis, The University of Leeds School of Computing. find
  36. R. Sennrich, B. Haddow, and A. Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715–1725, Berlin, Germany. Association for Computational Linguistics. find
  37. O. Smrž. 2007. ElixirFM — Implementation of Functional Arabic Morphology. In Proceedings of the 2007 Workshop on Computational Approaches to Semitic Languages: Common Issues and Resources, Prague, Czech Republic, pages 1–8. Association for Computational Linguistics. find our card
  38. A. Soudi and A. Eisele. 2004. Generating an Arabic Full-form Lexicon for Bidirectional Morphology Lookup. In Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC’04), Lisbon, Portugal. European Language Resources Association (ELRA). find
  39. M. Tahon, R. Qader, G. Lecorvé, and D. Lolive. 2016. Improving TTS with corpus-specific pronunciation adaptation. Interspeech, San Francisco, CA. find
  40. D. Taji, S. Khalifa, O. Obeid, F. Eryani, and N. Habash. 2018. An Arabic Morphological Analyzer and Generator with Copious Features. In Proceedings of the Fifteenth Workshop on Computational Research in Phonetics, Phonology, and Morphology, Brussels, pages 140–150. Association for Computational Linguistics. find our card
  41. The CJK Dictionary Institute. 2011. The CJKI Arabic Verb Conjugator. find
  42. The CJK Dictionary Institute. 2025a. DiaLEX: Arabic Dialects Full-form Lexicon. find
  43. The CJK Dictionary Institute. 2025b. The CJKI Arabic Learner’s Dictionary. find
  44. J. C. Wells. 1997. SAMPA computer readable phonetic alphabet. In D. Gibbon, R. Moore, and R. Winski, editors, Handbook of Standards and Resources for Spoken Language Systems. Mouton de Gruyter. Part IV, section B, Berlin and New York. find
  45. Q. Zhu, X. Hu, P. Ji, W. Wu, and K. Tu. 2024. Unsupervised morphological tree tokenizer. ArXiv preprint. 8. find
  46. Language Resource References The CJK Dictionary Institute. 2019a. ArabLEX: Database of Arab Names (DAN). distributed via ELRA: ELRA-Id ELRA-M0107, Lexicon, 1.0, ISLRN 773-974-582-139-4. find
  47. The CJK Dictionary Institute. 2019b. ArabLEX: Database of Arabic General Vocabulary (DAG). distributed via ELRA: ELRA-Id ELRA-L0131, Lexicon, 1.0, ISLRN 879-334-992-724-8. find
  48. The CJK Dictionary Institute. 2019c. ArabLEX: Database of Arabic Place Names (DAP). distributed via ELRA: ELRA-Id ELRA-M0105, Lexicon, 1.0, ISLRN 161-842-321-771-2. find
  49. The CJK Dictionary Institute. 2019d. ArabLEX: Database of Foreign Names in Arabic (DAF). distributed via ELRA: ELRA-Id ELRA-M0106, Lexicon, 1.0, ISLRN 943-592-129-040-2. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/lingbulk-2026-haralambous-comprehensive-full-form-lexicon-arabic/