Morphotactic Modeling in an Open-source Multi-dialectal Arabic Morphological Analyzer and Generator

What it is about

The paper describes CAMeL Morph, an open framework for writing Arabic morphology as specifications and compiling them into a database that the Camel Tools analyser and generator can use. The specifications have four parts: an order of morpheme classes, a lexicon, the morphemes and their allomorphs (affixes, clitics, stem buffers), and regular-expression substitutions for diacritic changes. Each allomorph sets and requires conditions (for example, "transitive" or "object clitic"), and an "else" term expresses complementary distribution. Stem buffers let one stem stand for a family of forms, so a defective verb needs only one stem plus a condition. The authors test the framework on Modern Standard Arabic and Egyptian verbs. Only 4 of 60 forms match between the two varieties with diacritics, and 38 of 60 without them, which shows how much of the variation is in the diacritics. The paper reports that the models give better coverage than the public analysers it compares with.

Ideas

• I1. Stem buffers for weak roots: list one stem for a verb whose surface forms alternate (third radical w/y, middle or final weak, geminates) and put the alternation in a buffer class with a condition term, instead of listing each surface stem. This is the cleanest way to write the weak-verb paradigms in Hebrew and Aramaic without multiplying lexicon entries.
• I2. Allomorph conditions with a reason: every allomorph sets and requires named conditions, so a rejected form can say which condition failed. Write the pronominal and suffix allomorphs of Hebrew and Aramaic this way, so that Mova's "why" can be read off the table.
• I3. Marked, ordered rewrites: mark the vowels that may be elided (the paper uses a "!" mark) and apply a short ordered list of rules, followed by a clean-up that removes the marks. This gives deterministic vowel reduction in the unpointed mode of tools/hbo and in the Aramaic phonology notes.
• I4. A paradigm table as a test: write out the verbal paradigm for a few verbs in every variety, count which forms match with and without diacritics, and keep the table as a regression test for the generator. The paper's 4-of-60 and 38-of-60 counts show what such a table reveals.

For Mova

I1 and I2 go into The weak verbs and The Noun, the Adjective and the Numeral for suffix allomorphy, and into the lexicon of tools/hbo. I3 applies to Script and phonology and to the unpointed mode of tools/hbo. I4 is a regression test for The verb paradigms. The paper's approach is a middle ground between tables and rules, which fits Mova's requirement that each decision can say why.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/2022.sigmorphon-1.10

@inproceedings{habash2022morphotactic,
  title = {{Morphotactic Modeling in an Open-source Multi-dialectal Arabic Morphological Analyzer and Generator}},
  author = {Habash, Nizar and Marzouk, Reham and Khairallah, Christian and Khalifa, Salam},
  booktitle = {Proceedings of the 19th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology},
  year = {2022},
  doi = {10.18653/v1/2022.sigmorphon-1.10},
  url = {https://doi.org/10.18653/v1/2022.sigmorphon-1.10}
}

References

31 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Imad A. Al-Sughaiyer and Ibrahim A. Al-Kharashi. 2004. Arabic morphological analysis techniques: A comprehensive survey. Journal of the American Society for Information Science and Technology, 55(3):189–213. find
2. Mohamed Altantawy, Nizar Habash, and Owen Rambow. 2011. Fast Yet Rich Morphological Analysis. In Proceedings of the International Workshop on Finite-State Methods and Natural Language Processing (FSMNLP), Blois, France. find
3. Ramy Baly, Hazem Hajj, Nizar Habash, Khaled Bashir Shaban, and Wassim El-Hajj. 2017. A sentiment treebank and morphologically enriched recursive deep models for effective sentiment analysis in Arabic. ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP), 16(4):23. find
4. Kenneth Beesley. 1996. Arabic Finite-State Morphological Analysis and Generation. In Proceedings of the International Conference on Computational Linguistics (COLING), pages 89–94, Copenhagen, Denmark. find
5. Kenneth Beesley, Tim Buckwalter, and Stuart Newton. 1989. Two-Level Finite-State Analysis of Arabic Morphology. In Proceedings of the Seminar on Bilingual Computing in Arabic and English. find
6. Mohamed Boudchiche, Azzeddine Mazroui, Mohamed Ould Abdallahi Ould Bebah, Abdelhak Lakhouaja, and Abderrahim Boudlal. 2017. AlKhalil Morpho Sys 2: A robust Arabic morpho-syntactic analyzer. Journal of King Saud University - Computer and Information Sciences, 29(2):141–146. find
7. Barli Bram. 2012. Three models of English morphology. LLT Journal: A Journal on Language and Language Teaching, 15(1):179–185. find
8. Tim Buckwalter. 2002. Buckwalter Arabic morphological analyzer version 1.0. Linguistic Data Consortium (LDC) catalog number LDC2002L49, ISBN 1-58563-257-0. find
9. Tim Buckwalter. 2004. Buckwalter Arabic Morphological Analyzer Version 2.0. LDC catalog number LDC2004L02, ISBN 1-58563-324-0. find
10. Noam Chomsky and Morris Halle. 1968. The sound pattern of English. New York: Harper and Row. find
11. Mona Diab, Nizar Habash, Owen Rambow, and Ryan Roth. 2013. LDC Arabic treebanks and associated corpora: Data divisions manual. arXiv preprint arXiv:1309.5652. ↗
12. David Graff, Mohamed Maamouri, Basma Bouziri, Sondos Krouna, Seth Kulick, and Tim Buckwalter. 2009. Standard Arabic Morphological Analyzer (SAMA) Version 3.1. Linguistic Data Consortium LDC2009E73. find
13. Nizar Habash. 2004. Large Scale Lexeme Based Arabic Morphological Generation. In Proceedings of Traitement Automatique des Langues Naturelles (TALN), pages 271–276, Fez, Morocco. find
14. Nizar Habash, Fadhl Eryani, Salam Khalifa, Owen Rambow, Dana Abdulrahim, Alexander Erdmann, Reem Faraj, Wajdi Zaghouani, Houda Bouamor, Nasser Zalmout, Sara Hassan, Faisal Al shargi, Sakhar Alkhereyf, Basma Abdulkareem, Ramy Eskander, Mohammad Salameh, and Hind Saddiki. 2018. Unified guidelines and resources for Arabic dialect orthography. In Proceedings of the Language Resources and Evaluation Conference (LREC), Miyazaki, Japan. find
15. Nizar Habash, Ramy Eskander, and Abdelati Hawwari. 2012. A Morphological Analyzer for Egyptian Arabic. In Proceedings of the Workshop of the Special Interest Group on Computational Morphology and Phonology (SIGMORPHON), pages 1–9, Montréal, Canada. find our card
16. Nizar Habash and Owen Rambow. 2006. MAGEAD: A morphological analyzer and generator for the Arabic dialects. In Proceedings of the International Conference on Computational Linguistics and the Conference of the Association for Computational Linguistics (COLING-ACL), pages 681–688, Sydney, Australia. find our card
17. Nizar Habash, Abdelhadi Soudi, and Tim Buckwalter. 2007. On Arabic Transliteration. In A. van den Bosch and A. Soudi, editors, Arabic Computational Morphology: Knowledge-based and Empirical Methods, pages 15–22. Springer, Netherlands. find
18. Nizar Y Habash. 2010. Introduction to Arabic natural language processing. Morgan & Claypool Publishers. find
19. Charles F Hockett. 1954. Two models of grammatical description. Word, 10(2-3):210–234. find
20. Mans Hulden and Younes Samih. 2012. Conversion of procedural morphologies to finite-state morphologies: a case study of Arabic. In Proceedings of the 10th International Workshop on Finite State Methods and Natural Language Processing, pages 70–74. find our card
21. Go Inoue, Salam Khalifa, and Nizar Habash. 2022. Morphosyntactic tagging with pre-trained language models for Arabic and its dialects. In Findings of the Association for Computational Linguistics: ACL 2022, pages 1708–1719, Dublin, Ireland. Association for Computational Linguistics. find
22. George Kiraz. 1994. Multi-tape Two-level Morphology: A Case study in Semitic Non-Linear Morphology. In Proceedings of the International Conference on Computational Linguistics (COLING), pages 180– 186, Kyoto, Japan. find our card
23. Mohamed Maamouri, Ann Bies, Tim Buckwalter, and Wigdan Mekki. 2004. The Penn Arabic Treebank: Building a Large-Scale Annotated Arabic Corpus. In Proceedings of the International Conference on Arabic Language Resources and Tools, pages 102– 109, Cairo, Egypt. find
24. Mohamed Maamouri, Ann Bies, Seth Kulick, Dalila Tabessi, and Sondos Krouna. 2012. Egyptian Arabic Treebank DF Parts 1-8 V2.0 - LDC catalog numbers LDC2012E93, LDC2012E98, LDC2012E89, LDC2012E99, LDC2012E107, LDC2012E125, LDC2013E12, LDC2013E21. find
25. Ossama Obeid, Nasser Zalmout, Salam Khalifa, Dima Taji, Mai Oudah, Bashar Alhafni, Go Inoue, Fadhl Eryani, Alexander Erdmann, and Nizar Habash. 2020. CAMeL tools: An open source python toolkit for Arabic natural language processing. In Proceedings of The 12th Language Resources and Evaluation Conference, pages 7022–7032, Marseille, France. European Language Resources Association. find
26. Arfath Pasha, Mohamed Al-Badrashiny, Mona Diab, Ahmed El Kholy, Ramy Eskander, Nizar Habash, Manoj Pooleery, Owen Rambow, and Ryan Roth. 2014. MADAMIRA: A fast, comprehensive tool for morphological analysis and disambiguation of Arabic. In Proceedings of the Language Resources and Evaluation Conference (LREC), pages 1094–1101, Reykjavik, Iceland. find our card
27. Otakar Smrž. 2007. Functional Arabic Morphology. Formal System and Implementation. Ph.D. thesis, Charles University in Prague, Prague, Czech Republic. find
28. Dima Taji, Salam Khalifa, Ossama Obeid, Fadhl Eryani, and Nizar Habash. 2018. An Arabic Morphological Analyzer and Generator with Copious Features. In Proceedings of the Fifteenth Workshop on Computational Research in Phonetics, Phonology, and Morphology (SIGMORPHON), pages 140–150. find our card
29. Wajdi Zaghouani, Behrang Mohit, Nizar Habash, Ossama Obeid, Nadi Tomeh, Alla Rozovskaya, Noura Farra, Sarah Alkuhlani, and Kemal Oflazer. 2014. Large Scale Arabic Error Annotation: Guidelines and Framework. In Proceedings of the Language Resources and Evaluation Conference (LREC), Reykjavik, Iceland. find
30. Nasser Zalmout and Nizar Habash. 2017. Don’t throw those morphological analyzers away just yet: Neural morphological disambiguation for Arabic. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 704–713, Copenhagen, Denmark. find our card
31. Nasser Zalmout and Nizar Habash. 2019. Joint diacritization, lemmatization, normalization, and fine-grained morphological tagging. arXiv preprint arXiv:1910.02267. ↗ our card

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/lingbulk-2022-habash-morphotactic-modeling-open-source-multi/