A Morphological Analyzer for Egyptian Arabic

What it is about

The paper presents CALIMA, a morphological analyzer for Egyptian Arabic (EGY), built by extending an older lexicon, the Egyptian Colloquial Arabic Lexicon (ECAL, about 66K entries). Four steps: a small set of hand-written finite-state rules inserts diacritics into the undiacritized entries; about 4,600 rules map the ECAL tags to the LDC's EGY tag set and to CODA, a conventional orthography for dialects; the analysis tables are converted to the SAMA format, with complex prefixes, stems and suffixes and their compatibility; then the tables are extended with extra clitics and with non-CODA spellings, which map back to CODA in the output. The result has about 100K stems over 36K lemmas and 48 million analysable word forms. On a blind, manually annotated set of 3,300 words, the correct answer is among the analyses for 84.1% of words, against 57.4% for ECAL and 63.7% for SAMA 3.1 (MSA). Merged with an extended SAMA, the coverage reaches 92.1%. In a review of 100 failures, 30% were gold-annotation errors, 22% non-CODA spellings, 20% incomplete paradigms, and 21% near-misses on case or mood.

Ideas

• I1. Fill gaps in a paradigm by analogy: when a lemma lacks some inflected stems, copy them from a lemma of the same class that has them (the paper's planned step for hollow verbs). For Hebrew, apply it to the weak-verb classes (I-nun, III-aleph, III-he, geminate) in the lexicon of tools/hbo, and measure on held-out forms that were removed on purpose.
• I2. Map non-standard spellings to the canonical form only inside the analysis: accept the variant, return the canonical analysis, and keep the variant list as a table. For unpointed Hebrew, this covers the plene and defective spellings of the same word; it fits the unpointed mode of tools/hbo.
• I3. Error analysis as a fixed set of categories: gold error, orthography, incomplete paradigm, near-miss on case or mood. Sample 100 misses of tools/hbo against MACULA and sort them into these classes, to show which fix would pay most.

For Mova

I1 goes into The weak verbs and the lexicon of tools/hbo. I2 goes into 1. Script and pointing and the unpointed mode of tools/hbo. I3 is an evaluation step for tools/hbo and tools/macula compare, recorded in 13. Biblical Hebrew in Mova: the tools.

Cite

The paper itself, not this card. BibTeX from the library's record of the work.

@inproceedings{habash2012morphological,
  title = {{A Morphological Analyzer for Egyptian Arabic}},
  author = {Habash, Nizar and Eskander, Ramy and Hawwari, Abdelati},
  booktitle = {Proceedings of the Twelfth Meeting of the Special Interest Group on Computational Morphology and Phonology},
  year = {2012},
  url = {https://aclanthology.org/W12-2301/}
}

References

29 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ernest T. Abdel-Massih, Zaki N. Abdel-Malek, and El-Said M. Badawi. 1979. A Reference Grammar of Egyptian Arabic. Georgetown University Press. find
2. Hitham Abo Bakr, Khaled Shaalan, and Ibrahim Ziedan. 2008. A Hybrid Approach for Converting Written Egyptian Colloquial Dialect into Diacritized Arabic. In The 6th International Conference on Informatics and Systems, INFOS2008. Cairo University. find
3. Imad Al-Sughaiyer and Ibrahim Al-Kharashi. 2004. Arabic Morphological Analysis Techniques: A Comprehensive Survey. Journal of the American Society for Information Science and Technology, 55(3):189– 213. find
4. Mohamed Altantawy, Nizar Habash, and Owen Rambow. 2011. Fast Yet Rich Morphological Analysis. In Proceedings of the 9th International Workshop on Finite-State Methods and Natural Language Processing (FSMNLP 2011), Blois, France. find
5. Kenneth Beesley, Tim Buckwalter, and Stuart Newton. 1989. Two-Level Finite-State Analysis of Arabic Morphology. In Proceedings of the Seminar on Bilingual Computing in Arabic and English, page n.p. find
6. Tim Buckwalter. 2004. Buckwalter arabic morphological analyzer version 2.0. LDC catalog number LDC2004L02, ISBN 1-58563-324-0. find
7. Tim Buckwalter. 2007. Issues in Arabic Morphological Analysis. In A. van den Bosch and A. Soudi, editors, Arabic Computational Morphology: Knowledge-based and Empirical Methods. Springer. find
8. Mark W. Cowell. 1964. A Reference Grammar of Syrian Arabic. Georgetown University Press. find
9. Mona Diab, Kadri Hacioglu, and Daniel Jurafsky, 2007. Arabic Computational Morphology: Knowledge-based and Empirical Methods, chapter Automated Methods for Processing Arabic Text: From Tokenization to Base Phrase Chunking. Springer. find
10. Wallace Erwin. 1963. A Short Reference Grammar of Iraqi Arabic. Georgetown University Press. find
11. Hassan Gadalla, Hanaa Kilany, Howaida Arram, Ashraf Yacoub, Alaa El-Habashi, Amr Shalaby, Krisjanis Karins, Everett Rowson, Robert MacIntyre, Paul Kingsbury, David Graff, and Cynthia McLemore. 1997. CALLHOME Egyptian Arabic Transcripts. In Linguistic Data Consortium, Philadelphia. find
12. Hassan Gadalla. 2000. Comparative Morphology of Standard and Egyptian Arabic. LINCOM EUROPA. find
13. David Graff, Mohamed Maamouri, Basma Bouziri, Sondos Krouna, Seth Kulick, and Tim Buckwalter. 2009. Standard Arabic Morphological Analyzer (SAMA) Version 3.1. Linguistic Data Consortium LDC2009E73. find
14. Nizar Habash and Owen Rambow. 2005. Arabic Tokenization, Part-of-Speech Tagging and Morphological Disambiguation in One Fell Swoop. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL’05), pages 573–580, Ann Arbor, Michigan. find our card
15. Nizar Habash and Owen Rambow. 2006. MAGEAD: A Morphological Analyzer and Generator for the Arabic Dialects. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, pages 681–688, Sydney, Australia. find our card
16. Nizar Habash, Abdelhadi Soudi, and Tim Buckwalter. 2007. On Arabic Transliteration. In A. van den Bosch and A. Soudi, editors, Arabic Computational Morphology: Knowledge-based and Empirical Methods. Springer. find
17. Nizar Habash, Mona Diab, and Owen Rabmow. 2012. Conventional Orthography for Dialectal Arabic. In Proceedings of the Language Resources and Evaluation Conference (LREC), Istanbul. find
18. Nizar Habash. 2007. Arabic Morphological Representations for Machine Translation. In Antal van den Bosch and Abdelhadi Soudi, editors, Arabic Computational Morphology: Knowledge-based and Empirical Methods. Kluwer/Springer. find
19. Nizar Habash. 2010. Introduction to Arabic Natural Language Processing. Morgan & Claypool Publishers. find
20. Clive Holes. 2004. Modern Arabic: Structures, Functions, and Varieties. Georgetown Classics in Arabic Language and Linguistics. Georgetown University Press. find
21. H. Kilany, H. Gadalla, H. Arram, A. Yacoub, A. El-Habashi, and C. McLemore. 2002. Egyptian Colloquial Arabic Lexicon. LDC catalog number LDC99L22. find
22. George Anton Kiraz. 2000. Multi-Tiered Nonlinear Morphology Using Multi-Tape Finite Automata: A Case Study on Syriac and Arabic. Computational Linguistics, 26(1):77–105. find
23. Mohamed Maamouri, Ann Bies, Tim Buckwalter, Mona Diab, Nizar Habash, Owen Rambow, and Dalila Tabessi. 2006. Developing and Using a Pilot Dialectal Arabic Treebank. In Proceedings of the Language Resources and Evaluation Conference (LREC), Genoa, Italy. find
24. Mohamed Maamouri, Ann Bies, Seth Kulick, Dalila Tabessi, and Sondos Krouna. 2012a. Egyptian Arabic Treebank Pilot. find
25. Mohamed Maamouri, Sondos Krouna, Dalila Tabessi, Nadia Hamrouni, and Nizar Habash. 2012b. Egyptian Arabic Morphological Annotation Guidelines. find
26. Emad Mohamed, Behrang Mohit, and Kemal Oflazer. 2012. Annotating and Learning Morphological Segmentation of Egyptian Colloquial Arabic. In Proceedings of the Language Resources and Evaluation Conference (LREC), Istanbul. find
27. Jason Riesa and David Yarowsky. 2006. Minimally Supervised Morphological Segmentation with Applications to Machine Translation. In Proceedings of the 7th Conference of the Association for Machine Translation in the Americas (AMTA06), pages 185–192, Cambridge,MA. find
28. Wael Salloum and Nizar Habash. 2011. Dialectal to Standard Arabic Paraphrasing to Improve Arabic-English Statistical Machine Translation. In Proceedings of the First Workshop on Algorithms and Resources for Modelling of Dialects and Language Varieties, pages 10–21, Edinburgh, Scotland. find
29. Otakar Smrž. 2007. Functional Arabic Morphology. Formal System and Implementation. Ph.D. thesis, Charles University in Prague, Prague, Czech Republic. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/lingbulk-2012-habash-morphological-analyzer-egyptian-arabic/