Noise-Robust Morphological Disambiguation for Dialectal Arabic

What it is about

The paper builds a morphological disambiguator for Egyptian Arabic, where spelling is unstandardised and most words have many analyses (about 24 per word, against about 12 in MSA). The core is a Bi-LSTM tagger that takes, for each word, its word embedding, a character embedding (LSTM or CNN) and the sum of the embeddings of the candidate tags from an analyser. Noise is handled on the vector level only: a narrow embedding window (2) is compared with a wide one (5), and an embedding-space mapping pulls rare variants towards frequent canonical neighbours, scored by cosine plus a learned weighted edit distance. On the blind test set the best system reaches 78.4 percent full analysis accuracy against 77.3 for the MADAMIRA baseline (about 5 percent relative error reduction), with POS accuracy 93.6 against 91.8. Narrow windows and LSTM character encoders win in every setting.

Ideas

• I1. Two embedding windows as two feature sets: a narrow-window context (syntactic) and a wide-window context (topical) give different information. For any statistical step in Mova that chooses among candidate analyses, try a narrow co-occurrence context first and measure it against the wide one.
• I2. Summed candidate-tag vectors: when a token has several analyses, score each candidate by the sum of its tag weights and pick the best, with the tag weights learned from the gold tags. The analyses stay in the table; the model only ranks them.
• I3. Edit-distance-weighted normalisation: learn substitution and deletion weights from pairs of candidate variants within edit distance two, and map a rare form to its most frequent close neighbour only as a vector-level lookup. Use it for OCR variants in tools/ocrfix, and for unpointed variants in tools/hbo.
• I4. Gold-error sampling: the authors checked 100 errors per task and found that many were gold-annotation errors (about 48 percent of POS errors). Make this a standing step: before a rule is blamed, sample its errors and label each as gold error or system error.

For Mova

I1 and I2 belong in tools/hbo, where the segmenter gives several splits or analyses per token and a small model ranks them. I3 goes to tools/ocrfix (OCR correction of printed editions) and to the unpointed mode of tools/hbo. I4 is a measurement protocol for treebanks/ar and for any gold set Mova trains on. The Egyptian data itself is not in Mova's scope.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/n18-1087

@inproceedings{zalmout2018noise,
  title = {{Noise-Robust Morphological Disambiguation for Dialectal Arabic}},
  author = {Zalmout, Nasser and Erdmann, Alexander and Habash, Nizar},
  booktitle = {Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)},
  year = {2018},
  doi = {10.18653/v1/n18-1087},
  url = {https://doi.org/10.18653/v1/n18-1087}
}

References

50 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. tagging for twitter: Annotation, features, and experiments. In ACL-HLT ’11: Short Papers -Volume 2. Association for Computational Linguistics, Stroudsburg, PA, USA, HLT ’11, pages 42– 47. http://dl.acm.org/citation.cfm? id=2002736.2002747. ↗
2. Ahmed Abdelali, Kareem Darwish, Nadir Durrani, and Hamdy Mubarak. 2016. Farasa: A fast and furious segmenter for Arabic. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations. San Diego, California, pages 11– 16. http://www.aclweb.org/anthology/ N16-3003. ↗ our card
3. Gustavo Aguilar, Suraj Maharjan, Adrian Pastor López Monroy, and Thamar Solorio. 2017. A multi-task approach for named entity recognition in social media data. In Proceedings of the 3rd Workshop on Noisy User-generated Text. pages 148–153. find
4. Rania Al-Sabbagh and Roxana Girju. 2010. Mining the Web for the Induction of a Dialectical Arabic Lexicon. In LREC. Valetta, Malta. find
5. Peter F Brown, Peter V Desouza, Robert L Mercer, Vincent J Della Pietra, and Jenifer C Lai. 1992. Class-based n-gram models of natural language. Computational linguistics 18(4):467–479. find
6. Pradeep Dasigi and Mona Diab. 2011. Codact: Towards identifying orthographic variants in dialectal arabic. In Proceedings of the International Joint Conference on Natural Language Processing. Chiang Mai, Thailand. find
7. Leon Derczynski, Alan Ritter, Sam Clark, and Kalina Bontcheva. 2013. Twitter part-of-speech tagging for all: Overcoming sparse and noisy data. In Proceedings of the International Conference Recent Advances in Natural Language Processing RANLP 2013. INCOMA Ltd. Shoumen, BULGARIA, pages 198–206. http://www.aclweb.org/anthology/R13-1026. ↗
8. Mona Diab, Nizar Habash, Owen Rambow, and Ryan Roth. 2013. LDC Arabic treebanks and associated corpora: Data divisions manual. arXiv preprint arXiv:1309.5652 . Seppo Enarvi and Mikko Kurimo. 2016. Theanolm - an extensible toolkit for neural network language modeling. CoRR abs/1605.00942. http://arxiv.org/abs/1605.00942. ↗
9. Ramy Eskander, Nizar Habash, Owen Rambow, and Nadi Tomeh. 2013. Processing Spontaneous Orthography. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT). Atlanta, GA. find
10. Jennifer Foster, Özlem Çetinoǧlu, Joachim Wagner, Joseph Le Roux, Stephen Hogan, Joakim Nivre, Deirdre Hogan, and Josef Van Genabith. 2011. #hardtoparse: Pos tagging and parsing the twitterverse. In Proceedings of the 5th AAAI Conference on Analyzing Microtext. AAAI Press, AAAIWS’11-05, pages 20–25. http://dl.acm.org/ citation.cfm?id=2908630.2908634. ↗
11. Kevin Gimpel, Nathan Schneider, Brendan O’Connor, Dipanjan Das, Daniel Mills, Jacob Eisenstein, Michael Heilman, Dani Yogatama, Jeffrey Flanigan, and Noah A. Smith. 2011. find
12. Part-of-speech Yoav Goldberg. 2016. A primer on neural network models for natural language processing. J. Artif. Intell. Res.(JAIR) 57:345–420. find
13. David Graff, Mohamed Maamouri, Basma Bouziri, Sondos Krouna, Seth Kulick, and Tim Buckwalter. 2009. Standard Arabic Morphological Analyzer (SAMA) Version 3.1. Linguistic Data Consortium LDC2009E73. find
14. Nizar Habash, Mona Diab, and Owen Rambow. 2012a. Conventional Orthography for Dialectal Arabic: Principles and Guidelines – Egyptian Arabic. Technical Report CCLS-12-02, Columbia University Center for Computational Learning Systems. find
15. Nizar Habash, Fadhl Eryani, Salam Khalifa, Owen Rambow, Dana Abdulrahim, Alexander Erdmann, Reem Faraj, Wajdi Zaghouani, Houda Bouamor, Nasser Zalmout, Sara Hassan, Faisal Al-Shargi, Sakhar Alkhereyf, Basma Abdulkareem, Ramy Eskander, Mohammad Salameh, and Hind Saddiki. 2018. Unified guidelines and resources for arabic dialect orthography. In Proceedings of the International Conference on Language Resources and Evaluation (LREC). find
16. Nizar Habash, Ramy Eskander, and Abdelati Hawwari. 2012b. A Morphological Analyzer for Egyptian Arabic. In Proceedings of the Twelfth Meeting of the Special Interest Group on Computational Morphology and Phonology. Montréal, Canada, pages 1–9. find our card
17. Nizar Habash and Owen Rambow. 2005. Arabic Tokenization, Part-of-Speech Tagging and Morphological Disambiguation in One Fell Swoop. In Proceedings of the 43rd Annual Meeting of the ACL. Ann Arbor, Michigan, pages 573–580. find our card
18. Nizar Habash and Owen Rambow. 2007. Arabic Diacritization through Full Morphological Tagging. In Proceedings of the 8th Meeting of the North American Chapter of the Association for Computational Linguistics/Human Language Technologies Conference (HLT-NAACL07). find our card
19. Nizar Habash, Owen Rambow, and Ryan Roth. 2009. MADA+TOKAN: A toolkit for Arabic tokenization, diacritization, morphological disambiguation, POS tagging, stemming and lemmatization. In Khalid Choukri and Bente Maegaard, editors, Proceedings of the Second International Conference on Arabic Language Resources and Tools. The MEDAR Consortium. find
20. Nizar Habash, Ryan Roth, Owen Rambow, Ramy Eskander, and Nadi Tomeh. 2013a. Morphological Analysis and Disambiguation for Dialectal Arabic. In Proceedings of the 2013 Conference of NAACL-HLT. Atlanta, GA. find
21. Nizar Habash, Ryan Roth, Owen Rambow, Ramy Eskander, and Nadi Tomeh. 2013b. find
22. Morphological Mohamed Maamouri, Ann Bies, Tim Buckwalter, and Wigdan Mekki. 2004. The Penn Arabic Treebank: Building a Large-Scale Annotated Arabic Corpus. In NEMLAR Conference on Arabic Language Resources and Tools. Cairo, Egypt, pages 102–109. Analysis and Disambiguation for Dialectal Arabic. In Proceedings of NAACL-HLT. Atlanta, Georgia, pages 426–432. http://www.aclweb.org/ anthology/N13-1044. ↗
23. Nizar Habash, Abdelhadi Soudi, and Tim Buckwalter. 2007. On Arabic Transliteration. In A. van den Bosch and A. Soudi, editors, Arabic Computational Morphology: Knowledge-based and Empirical Methods, Springer. find
24. Mohamed Maamouri, Sondos Krouna, Dalila Tabessi, Nadia Hamrouni, and Nizar Habash. 2012. Egyptian Arabic Morphological Annotation Guidelines. find
25. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 26, Curran Associates, Inc., pages 3111–3119. find
26. Georg Heigold, Guenter Neumann, and Josef van Genabith. 2017. An extensive empirical evaluation of character-based morphological tagging for 14 languages. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers. Association for Computational Linguistics, pages 505–513. http://www.aclweb.org/ anthology/E17-1048. ↗
27. Emad Mohamed, Behrang Mohit, and Kemal Oflazer. 2012. Annotating and Learning Morphological Segmentation of Egyptian Colloquial Arabic. In Proceedings of the Language Resources and Evaluation Conference (LREC). Istanbul. find
28. Go Inoue, Hiroyuki Shindo, and Yuji Matsumoto. 2017. Joint prediction of morphosyntactic categories for fine-grained arabic part-of-speech tagging exploiting tag dictionary information. In Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017). pages 421– 431. find our card
29. Olutobi Owoputi, Brendan O’Connor, Chris Dyer, Kevin Gimpel, Nathan Schneider, and Noah A. Smith. 2013. Improved part-of-speech tagging for online conversational text with word clusters. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, pages 380–390. http://www.aclweb.org/ anthology/N13-1039. ↗
30. Mustafa Jarrar, Nizar Habash, Faeq Alrimawi, Diyam Akra, and Nasser Zalmout. 2016. Curras: an annotated corpus for the Palestinian Arabic dialect. Language Resources and Evaluation pages 1–31. https://doi.org/10.1007/ s10579-016-9370-7. ↗
31. Arfath Pasha, Mohamed Al-Badrashiny, Ahmed El Kholy, Ramy Eskander, Mona Diab, Nizar Habash, Manoj Pooleery, Owen Rambow, and Ryan Roth. 2014. MADAMIRA: A Fast, Comprehensive Tool for Morphological Analysis and Disambiguation of Arabic. In In Proceedings of LREC. Reykjavik, Iceland. find our card
32. Anna Jørgensen, Dirk Hovy, and Anders Søgaard. 2015. Challenges of studying and processing dialects in social media. In Proceedings of the Workshop on Noisy User-generated Text. pages 9–18. find
33. Salam Khalifa, Nizar Habash, Dana Abdulrahim, and Sara Hassan. 2016a. A Large Scale Corpus of Gulf Arabic. In Proceedings of the International Conference on Language Resources and Evaluation (LREC). Portorož, Slovenia. find
34. Slav Petrov and Ryan McDonald. 2012. Overview of the 2012 shared task on parsing the web. In Notes of the First Workshop on Syntactic Analysis of Non-Canonical Language (SANCL). volume 59. find
35. Salam Khalifa, Nasser Zalmout, and Nizar Habash. 2016b. Yamama: Yet another multi-dialect arabic morphological analyzer. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: System Demonstrations. pages 223–227. find
36. Barbara Plank, Anders Søgaard, and Yoav Goldberg. 2016. Multilingual part-of-speech tagging with bidirectional long short-term memory models and auxiliary loss. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Association for Computational Linguistics, pages 412–418. https://doi.org/10.18653/ v1/P16-2067. ↗
37. Diederik P. Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. CoRR abs/1412.6980. http://arxiv.org/abs/ 1412.6980. ↗
38. Radim Řehůřek and Petr Sojka. 2010. Software Framework for Topic Modelling with Large Corpora. In Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks. ELRA, Valletta, Malta, pages 45–50. http://is.muni.cz/ publication/884893/en. ↗
39. Xiaohua Liu, Shaodian Zhang, Furu Wei, and Ming Zhou. 2011. Recognizing named entities in tweets. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies - Volume 1. Association for Computational Linguistics, Stroudsburg, PA, USA, HLT ’11, pages 359– 367. http://dl.acm.org/citation.cfm? id=2002472.2002519. ↗
40. Alan Ritter, Sam Clark, Mausam, and Oren Etzioni. 2011. Named entity recognition in tweets: An experimental study. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Stroudsburg, PA, USA, EMNLP ’11, pages 1524–1534. http://dl.acm.org/ citation.cfm?id=2145432.2145595. ↗
41. Ryan Roth, Owen Rambow, Nizar Habash, Mona Diab, and Cynthia Rudin. 2008. Arabic morphological tagging, diacritization, and lemmatization using lexeme models and feature ranking. In ACL 2008: The Conference of the Association for Computational Linguistics; Companion Volume, Short Papers. Association for Computational Linguistics, Columbus, Ohio. find our card
42. Houda Saadane and Nizar Habash. 2015. A Conventional Orthography for Algerian Arabic. In ANLP Workshop 2015. page 69. find
43. Wael Salloum and Nizar Habash. 2014. ADAM: Analyzer for Dialectal Arabic Morphology. Journal of King Saud University-Computer and Information Sciences 26(4):372–378. find
44. Vivek Kumar Rangarajan Sridhar. 2015. Unsupervised text normalization using distributed representations of words and phrases. In Proceedings of the 1st Workshop on Vector Space Modeling for Natural Language Processing. pages 8–16. find
45. Andrew Trask, David Gilmore, and Matthew Russell. 2015. Modeling order in neural word embeddings at scale. arXiv preprint arXiv:1506.02338 . ↗
46. Lifu Tu, Kevin Gimpel, and Karen Livescu. 2017. Learning to embed words in context for syntactic tasks. In Proceedings of the 2nd Workshop on Representation Learning for NLP. pages 265–275. find
47. Houcemeddine Turki, Emad Adel, Tariq Daouda, and Nassim Regragui. 2016. A Conventional Orthography for Maghrebi Arabic. In Proceedings of the International Conference on Language Resources and Evaluation (LREC). Portorož, Slovenia. find
48. Rob van der Goot, Barbara Plank, and Malvina Nissim. 2017. To normalize, or not to normalize: The impact of normalization on part-of-speech tagging. In Proceedings of the 3rd Workshop on Noisy User-generated Text. Association for Computational Linguistics, Copenhagen, Denmark, pages 31– 39. http://www.aclweb.org/anthology/ W17-4404. ↗
49. Nasser Zalmout and Nizar Habash. 2017. Don’t throw those morphological analyzers away just yet: Neural morphological disambiguation for arabic. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Copenhagen, Denmark, pages 704–713. https://www.aclweb.org/anthology/D17-1073. ↗ our card
50. I. Zribi, R. Boujelbane, A. Masmoudi, M. Ellouze Khmekhem, L. Hadrich Belguith, and N. Habash. 2014. A Conventional Orthography for Tunisian Arabic. In In Proceedings of LREC. Reykjavik, Iceland. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/naacl-2018-zalmout-noise-robust-morphological-disambiguation-dialectal/