Mova0.5 pre-release⌕
☰
home / docs / papers / Semitic languages / Morphological Analysis and Disambiguation for Gulf Arabic: The Interplay between Resources and Methods

Morphological Analysis and Disambiguation for Gulf Arabic: The Interplay between Resources and Methods

What it is about

This is the first full morphological analysis and disambiguation system for Gulf Arabic (Emirati data). The analyser for Gulf Arabic is not written by hand: it is built by paradigm completion from the training split of the Annotated Gumar Corpus (about 202,000 tokens, eight novel excerpts). Two hand-built analysers, for MSA and Egyptian, are used as optional extra sources. The disambiguator is a neural joint tagger (Zalmout and Habash 2019), used in three setups: no analyser, analyser candidates embedded in the input, and embedding plus re-ranking of the candidates. The split is made inside each novel (80/10/10), so every style is in each split. The abstract's main finding: when resources are very small, analysers help full disambiguation; as the data grow, their value falls. Raw text was normalized to CODA first, with a word error rate of 24.9% against raw spelling.

Ideas

For Mova

I1 is a method for languages/tmr/ and languages/jpa/ (morphology sections) once an annotated sample exists. I2 is an experiment for tools/hbo and gparse; the measure is the same split under three setups. I3 is the split rule for any treebank test in treebanks/hbo/ptnk, and it applies to cv decisions, not to the SVAMP test.

Cite

The paper itself, not this card. BibTeX from the library's record of the work.

@inproceedings{khalifa2020morphological,
  title = {{Morphological Analysis and Disambiguation for Gulf Arabic: The Interplay between Resources and Methods}},
  author = {Khalifa, Salam and Zalmout, Nasser and Habash, Nizar},
  booktitle = {Proceedings of the Twelfth Language Resources and Evaluation Conference},
  year = {2020},
  url = {https://aclanthology.org/2020.lrec-1.480/}
}

References

63 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abandah, G. A., Graves, A., Al-Shagoor, B., Arabiyat, A., Jamour, F., and Al-Taee, M. (2015). Automatic diacritization of Arabic text using recurrent neural networks. International Journal on Document Analysis and Recognition (IJDAR), 18(2):183–197. find
  2. Abdelali, A., Darwish, K., Durrani, N., and Mubarak, H. (2016). Farasa: A Fast and Furious Segmenter for Arabic. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), pages 11–16, San Diego, California. find our card
  3. Al-Sabbagh, R. and Girju, R. (2012). A supervised POS tagger for written Arabic social networking corpora. In Jeremy Jancsary, editor, Proceedings of the Conference on Natural Language Processing (KONVENS), pages 39–52. ÖGAI. Main track: oral presentations. find
  4. Albogamy, F. and Ramsay, A. (2015). POS tagging for Arabic tweets. In Proceedings of the International Conference Recent Advances in Natural Language Processing, pages 1–8, Hissar, Bulgaria, September. INCOMA Ltd. Shoumen, BULGARIA. find
  5. Alharbi, R., Magdy, W., Darwish, K., Abdelali, A., and Mubarak, H. (2018). Part-of-speech tagging for Arabic gulf dialect using bi-lstm. In Proceedings of the Language Resources and Evaluation Conference (LREC), Miyazaki, Japan. find
  6. Alkuhlani, S. and Habash, N. (2011). A Corpus for Modeling Morpho-Syntactic Agreement in Arabic: Gender, Number and Rationality. In Proceedings of the Conference of the Association for Computational Linguistics (ACL), Portland, Oregon, USA. find our card
  7. Alshargi, F., Dibas, S., Alkhereyf, S., Faraj, R., Abdulkareem, B., Yagi, S., Kacha, O., Habash, N., and Rambow, O. (2019). Morphologically annotated corpora for seven Arabic dialects: Taizi, sanaani, najdi, jordanian, syrian, iraqi and Moroccan. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 137– 147, Florence, Italy, August. Association for Computational Linguistics. find our card
  8. Baly, R., Hajj, H., Habash, N., Shaban, K. B., and El-Hajj, W. (2017). A sentiment treebank and morphologically enriched recursive deep models for effective sentiment analysis in arabic. ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP), 16(4):23. find
  9. Bojanowski, P., Grave, E., Joulin, A., and Mikolov, T. (2016). Enriching word vectors with subword information. arXiv preprint arXiv:1607.04606. ↗
  10. Bouamor, H., Habash, N., Salameh, M., Zaghouani, W., Rambow, O., Abdulrahim, D., Obeid, O., Khalifa, S., Eryani, F., Erdmann, A., and Oflazer, K. (2018). The MADAR Arabic Dialect Corpus and Lexicon. In Proceedings of the Language Resources and Evaluation Conference (LREC), Miyazaki, Japan. find
  11. Boudlal, A., Lakhouaja, A., Mazroui, A., Meziane, A., Bebah, M., and Shoul, M. (2010). Alkhalil Morpho Sys1: A morphosyntactic analysis system for Arabic texts. In Proceedings of the International Arab Conference on Information Technology, pages 1–6. find
  12. Buckwalter, T. (2004). Buckwalter Arabic Morphological Analyzer Version 2.0. LDC catalog number LDC2004L02, ISBN 1-58563-324-0. find
  13. Darwish, K. and Mubarak, H. (2016). Farasa: A new fast and accurate Arabic word segmenter. In Proceedings of the Language Resources and Evaluation Conference (LREC), Portorož, Slovenia. find
  14. Darwish, K., Mubarak, H., Abdelali, A., and Eldesouki, M. (2017). Arabic pos tagging: Don’t abandon feature engineering just yet. In Proceedings of the Workshop for Arabic Natural Language Processing (WANLP), pages 130–137, Valencia, Spain. find
  15. Darwish, K., Mubarak, H., Abdelali, A., Eldesouki, M., Samih, Y., Alharbi, R., Attia, M., Magdy, W., and Kallmeyer, L. (2018). Multi-dialect Arabic pos tagging: A CRF approach. In Proceedings of the Language Resources and Evaluation Conference (LREC), Miyazaki, Japan. find
  16. Diab, M., Hacioglu, K., and Jurafsky, D. (2004). Automatic Tagging of Arabic Text: From Raw Text to Base Phrase Chunks. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), pages 149–152, Boston, MA. find
  17. Diab, M., Hacioglu, K., and Jurafsky, D., (2007). Arabic Computational Morphology: Knowledge-based and Empirical Methods, chapter Automated Methods for Processing Arabic Text: From Tokenization to Base Phrase Chunking. Springer Netherlands, kluwer/springer edition. find
  18. Duh, K. and Kirchhoff, K. (2005). POS tagging of dialectal Arabic: a minimally supervised approach. In Proceedings of the Workshop on Computational Approaches to Semitic Languages (CASL), pages 55–62, Ann Arbor, Michigan. find
  19. El Kholy, A. and Habash, N. (2012). Orthographic and morphological processing for English–Arabic statistical machine translation. Machine Translation, 26(1-2):25– 45. find
  20. Erdmann, A., Khalifa, S., Oudah, M., Habash, N., and Bouamor, H. (2019). A little linguistics goes a long way: Unsupervised segmentation with limited language specific guidance. In Proceedings of the 16th Workshop on Computational Research in Phonetics, Phonology, and Morphology, pages 113–124, Florence, Italy, August. Association for Computational Linguistics. find
  21. Eryani, F., Habash, N., Bouamor, H., and Khalifa, S. (2020). A Spelling Correction Corpus for Multiple Arabic Dialects. In Proceedings of the Language Resources and Evaluation Conference (LREC), Marseille, France. find
  22. Eskander, R., Habash, N., and Rambow, O. (2013a). Automatic extraction of morphological lexicons from morphologically annotated corpora. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1032–1043, Seattle, Washington, USA. find
  23. Eskander, R., Habash, N., Rambow, O., and Tomeh, N. (2013b). Processing spontaneous orthography. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), pages 585–595, Atlanta, Georgia. find
  24. Eskander, R., Habash, N., Rambow, O., and Pasha, A. (2016). Creating resources for Dialectal Arabic from a single annotation: A case study on Egyptian and Levantine. In Proceedings of the International Conference on Computational Linguistics (COLING), pages 3455– 3465, Osaka, Japan. find
  25. Graff, D., Maamouri, M., Bouziri, B., Krouna, S., Kulick, S., and Buckwalter, T. (2009). Standard Arabic Morphological Analyzer (SAMA) Version 3.1. Linguistic Data Consortium LDC2009E73. find
  26. Guzmán, F., Bouamor, H., Baly, R., and Habash, N. (2016). Machine translation evaluation for Arabic using morphologically-enriched embeddings. In Proceedings of the International Conference on Computational Linguistics (COLING), pages 1398–1408, Osaka, Japan. find
  27. The COLING 2016 Organizing Committee. find
  28. Habash, N. and Rambow, O. (2005). Arabic tokenization, part-of-speech tagging and morphological disambiguation in one fell swoop. In Proceedings of the Conference of the Association for Computational Linguistics (ACL), pages 573–580, Ann Arbor, Michigan. find our card
  29. Habash, N. and Rambow, O. (2006). MAGEAD: A morphological analyzer and generator for the Arabic dialects. In Proceedings of the International Conference on Computational Linguistics and the Conference of the Association for Computational Linguistics (COLING-ACL), pages 681–688, Sydney, Australia. find our card
  30. Habash, N., Soudi, A., and Buckwalter, T. (2007). On Arabic Transliteration. In A. van den Bosch et al., editors, Arabic Computational Morphology: Knowledge-based and Empirical Methods, pages 15–22. Springer, Netherlands. find
  31. Habash, N., Rambow, O., and Roth, R. (2009). MADA+TOKAN: A toolkit for Arabic tokenization, diacritization, morphological disambiguation, POS tagging, stemming and lemmatization. In Khalid Choukri et al., editors, Proceedings of the International Conference on Arabic Language Resources and Tools, Cairo, Egypt. The MEDAR Consortium. find
  32. Habash, N., Eskander, R., and Hawwari, A. (2012). A Morphological Analyzer for Egyptian Arabic. In Proceedings of the Workshop of the Special Interest Group on Computational Morphology and Phonology (SIG-MORPHON), pages 1–9, Montréal, Canada. find our card
  33. Habash, N., Roth, R., Rambow, O., Eskander, R., and Tomeh, N. (2013). Morphological Analysis and Disambiguation for Dialectal Arabic. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), Atlanta, Georgia. find
  34. Habash, N., Eryani, F., Khalifa, S., Rambow, O., Abdulrahim, D., Erdmann, A., Faraj, R., Zaghouani, W., Bouamor, H., Zalmout, N., Hassan, S., shargi, F. A., Alkhereyf, S., Abdulkareem, B., Eskander, R., Salameh, M., and Saddiki, H. (2018). Unified guidelines and resources for Arabic dialect orthography. In Proceedings of the Language Resources and Evaluation Conference (LREC), Miyazaki, Japan. find
  35. Habash, N. (2007). Arabic Morphological Representations for Machine Translation. In Antal van den Bosch et al., editors, Arabic Computational Morphology: Knowledge-based and Empirical Methods. Kluwer/Springer. find
  36. Halabi, N. (2016). Modern standard Arabic phonetics for speech synthesis. Ph.D. thesis, University of Southampton. find
  37. Hamdi, A., Nasr, A., Habash, N., and Gala, N. (2015). POS-tagging of Tunisian dialect using standard Arabic resources and tools. In Proceedings of the Second Workshop on Arabic Natural Language Processing, pages 59– 68, Beijing, China, July. Association for Computational Linguistics. find
  38. Heigold, G., Genabith, J. v., and Neumann, G. (2016). Scaling character-based morphological tagging to fourteen languages. In Proceedings of the International Conference on Big Data (Big Data), pages 3895–3902. find
  39. Inoue, G., Shindo, H., and Matsumoto, Y. (2017). Joint Prediction of Morphosyntactic Categories for Fine-Grained Arabic Part-of-Speech Tagging Exploiting Tag Dictionary Information. In Proceedings of the Conference on Computational Natural Language Learning (CoNLL), pages 421–431, Vancouver, Canada. find our card
  40. Jarrar, M., Habash, N., Akra, D., and Zalmout, N. (2014). Building a Corpus for Palestinian Arabic: A Preliminary Study. In Proceedings of the Workshop for Arabic Natural Language Processing (WANLP), pages 18–27, Doha, Qatar. find
  41. Khalifa, S., Habash, N., Abdulrahim, D., and Hassan, S. (2016a). A Large Scale Corpus of Gulf Arabic. In Proceedings of the Language Resources and Evaluation Conference (LREC), Portorož, Slovenia. find
  42. Khalifa, S., Zalmout, N., and Habash, N. (2016b). Yamama: Yet another multi-dialect Arabic morphological analyzer. In Proceedings of the International Conference on Computational Linguistics (COLING), pages 223– 227, Osaka, Japan. find
  43. Khalifa, S., Hassan, S., and Habash, N. (2017). A Morphological Analyzer for Gulf Arabic Verbs. In Proceedings of the Workshop for Arabic Natural Language Processing (WANLP), Valencia, Spain. find
  44. Khalifa, S., Habash, N., Eryani, F., Obeid, O., Abdulrahim, D., and Kaabi, M. A. (2018). A morphologically annotated corpus of emirati Arabic. In Proceedings of the Language Resources and Evaluation Conference (LREC), Miyazaki, Japan. find
  45. Khoja, S. (2001). APT: Arabic Part-of-Speech Tagger. In Proceedings of NAACL Student Research Workshop (SRW), pages 20–26, Pittsburgh. find
  46. Kilany, H., Gadalla, H., Arram, H., Yacoub, A., El-Habashi, A., and McLemore, C. (2002). Egyptian Colloquial Arabic Lexicon. LDC catalog number LDC99L22. find
  47. Maamouri, M., Bies, A., Buckwalter, T., and Mekki, W. (2004). The Penn Arabic Treebank: Building a Large-Scale Annotated Arabic Corpus. In Proceedings of the International Conference on Arabic Language Resources and Tools, pages 102–109, Cairo, Egypt. find
  48. Maamouri, M., Graff, D., Bouziri, B., Krouna, S., Bies, A., and Kulick, S. (2010). Standard Arabic morphological analyzer (sama) version 3.1. Linguistic Data Consortium, Catalog No.: LDC2010L01. find
  49. Maamouri, M., Bies, A., Kulick, S., Tabessi, D., and Krouna, S. (2012). Egyptian Arabic Treebank DF Parts 1-8 V2.0 - LDC catalog numbers LDC2012E93, LDC2012E98, LDC2012E89, LDC2012E99, LDC2012E107, LDC2012E125, LDC2013E12, LDC2013E21. find
  50. Mohit, B., Rozovskaya, A., Habash, N., Zaghouani, W., and Obeid, O. (2014). The first QALB shared task on automatic text correction for Arabic. In Proceedings of the Workshop for Arabic Natural Language Processing (WANLP), pages 39–47, Doha, Qatar. find
  51. Obeid, O., Khalifa, S., Habash, N., Bouamor, H., Zaghouani, W., and Oflazer, K. (2018). MADARi: A Web Interface for Joint Arabic Morphological Annotation and Spelling Correction. In Proceedings of the Language Resources and Evaluation Conference (LREC), Miyazaki, Japan. find
  52. Pasha, A., Al-Badrashiny, M., Diab, M., Kholy, A. E., Eskander, R., Habash, N., Pooleery, M., Rambow, O., and Roth, R. (2014). Madamira: A fast, comprehensive tool for morphological analysis and disambiguation of Arabic. In Proceedings of the Language Resources and Evaluation Conference (LREC), pages 1094–1101, Reykjavik, Iceland. find our card
  53. Roth, R., Rambow, O., Habash, N., Diab, M., and Rudin, C. (2008). Arabic morphological tagging, diacritization, and lemmatization using lexeme models and feature ranking. In Proceedings of the Conference of the Association for Computational Linguistics (ACL), Columbus, Ohio. find our card
  54. Sadat, F. and Habash, N. (2006). Combination of Arabic Preprocessing Schemes for Statistical Machine Translation. In Proceedings of the International Conference on Computational Linguistics and the Conference of the Association for Computational Linguistics (COLING-ACL), pages 1–8, Sydney, Australia. find
  55. Salloum, W. and Habash, N. (2014). ADAM: Analyzer for Dialectal Arabic Morphology. Journal of King Saud University - Computer and Information Sciences, 26(4):372–378. find
  56. Samih, Y., Attia, M., Eldesouki, M., Abdelali, A., Mubarak, H., Kallmeyer, L., and Darwish, K. (2017a). A neural architecture for dialectal Arabic segmentation. In Proceedings of the Workshop for Arabic Natural Language Processing (WANLP), pages 46–54, Valencia, Spain. find our card
  57. Samih, Y., Eldesouki, M., Attia, M., Darwish, K., Abdelali, A., Mubarak, H., and Kallmeyer, L. (2017b). Learning from relatives: Unified dialectal Arabic segmentation. In Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), pages 432– 441, Vancouver, Canada, August. Association for Computational Linguistics. find
  58. Shen, Q., Clothiaux, D., Tagtow, E., Littell, P., and Dyer, C. (2016). The role of context in neural morphological disambiguation. In Proceedings of the International Conference on Computational Linguistics (COLING), pages 181–191, Osaka, Japan. find
  59. The COLING 2016 Organizing Committee. find
  60. Watson, D., Zalmout, N., and Habash, N. (2018). Utilizing character and word embeddings for text normalization with sequence-to-sequence models. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 837–843, Brussels, Belgium, October-November. Association for Computational Linguistics. find
  61. Zalmout, N. and Habash, N. (2017). Don’t throw those morphological analyzers away just yet: Neural morphological disambiguation for Arabic. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 704–713, Copenhagen, Denmark. find our card
  62. Zalmout, N. and Habash, N. (2019). Joint diacritization, lemmatization, normalization, and fine-grained morphological tagging. arXiv preprint arXiv:1910.02267. ↗ our card
  63. Zribi, I., Ellouze Khemakhem, M., and Hadrich Belguith, L. (2013). Morphological analysis of Tunisian dialect. In Proceedings of the Sixth International Joint Conference on Natural Language Processing, pages 992–996, Nagoya, Japan, October. Asian Federation of Natural Language Processing. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/lingbulk-2020-khalifa-morphological-analysis-disambiguation-gulf-arabic/