Don't Throw Those Morphological Analyzers Away Just Yet: Neural Morphological Disambiguation for Arabic

What it is about

A morphological analyzer gives several readings for each Arabic word when taken out of context, and the task is to choose the right one in context. The authors train Bi-LSTM taggers for 14 non-lexical features (POS, case, number, gender, voice, mood, state, aspect, person, and the clitics) on Penn Arabic Treebank parts 1 to 3, with about 503,000 training words. The inputs are word embeddings plus affix features, with dictionary tags as an option. The lexical features (lemma and diacritized form) come from LSTM language models. Each analysis is scored by how many features it shares with the taggers, and the weights are tuned by Downhill Simplex on a held-out slice of training data. Full-analysis accuracy rises from 85.6% (MADAMIRA) to 90.0%, a 4.4-point gain; on out-of-vocabulary words it rises from 66.3% to 76.9%.

Ideas

• I1. Rank the candidate analyses of a word by a weighted feature-match score: each candidate gets one point per feature that agrees with a per-feature tagger, with the weights tuned on held-out data. For Mova, the candidate splits of tools/hbo and the readings of macula dict can be ranked this way. The weights are a small table, so the choice stays deterministic and can be explained.
• I2. Keep the affix information as a rule. The paper's fixed three-character prefixes and suffixes give most of the gain without any lexicon. The Hebrew analogue is a table of proclitics (the conjunction ו, the prepositions ב, כ, ל, מ) read as features, not as a lookup.
• I3. Report the errors per feature, with the out-of-vocabulary split, as the paper does. Mark case and voice as the hard cases: they depend on syntax, so test them after gparse in prag, not inside the segmenter.

For Mova

I1 and I2 go into tools/hbo: candidate ranking after the segmenter, with macula dict as the source of readings. I3 is the report format for the tests of tools/macula, and a syntax test in prag. The neural taggers are not reused; the ranking and the per-feature report are.

Topics: Part-of-speech tagging

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/d17-1073

@inproceedings{zalmout2017don,
  title = {{Don't Throw Those Morphological Analyzers Away Just Yet: Neural Morphological Disambiguation for Arabic}},
  author = {Zalmout, Nasser and Habash, Nizar},
  booktitle = {Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing},
  year = {2017},
  doi = {10.18653/v1/d17-1073},
  url = {https://doi.org/10.18653/v1/d17-1073}
}

References

27 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Seppo Enarvi and Mikko Kurimo. 2016. Theanolm - an extensible toolkit for neural network language modeling. CoRR, abs/1605.00942. ↗
2. Gheith A. Abandah, Alex Graves, Balkees Al-Shagoor, Alaa Arabiyat, Fuad Jamour, and Majid Al-Taee. 2015. Automatic diacritization of Arabic text using recurrent neural networks. International Journal on Document Analysis and Recognition (IJDAR), 18(2):183–197. find
3. Francisco Guzmán, Houda Bouamor, Ramy Baly, and Nizar Habash. 2016. Machine translation evaluation for Arabic using morphologically-enriched embeddings. In Proceedings of COLING 2016, the International Conference on Computational Linguistics, pages 1398–1408, Osaka, Japan. find
4. Nizar Habash and Owen Rambow. 2005. Arabic Tokenization, Part-of-Speech Tagging and Morphological Disambiguation in One Fell Swoop. In Proceedings of the 43rd Annual Meeting of the ACL, pages 573–580, Ann Arbor, Michigan. find our card
5. Franz Josef Och. 1999. An efficient method for determining bilingual word classes. In Proceedings of the Ninth Conference on European Chapter of the Association for Computational Linguistics, EACL ’99, pages 71–76, Stroudsburg, PA, USA. find
6. Nizar Habash, Ryan Roth, Owen Rambow, Ramy Eskander, and Nadi Tomeh. 2013. Morphological analysis and disambiguation for dialectal Arabic. In Proceedings of NAACL-HLT, pages 426–432, Atlanta, Georgia. find
7. Franz Josef Och and Hermann Ney. 2003. A systematic comparison of various statistical alignment models. Computational Linguistics. find
8. Robert Parker, David Graff, Ke Chen, Junbo Kong, and Kazuaki Maeda. 2011. Arabic Gigaword Fifth Edition. LDC catalog number No. LDC2011T11, ISBN 1-58563-595-2. find
9. Nizar Habash, Abdelhadi Soudi, and Tim Buckwalter. 2007. On Arabic Transliteration. In A. van den Bosch and A. Soudi, editors, Arabic Computational Morphology: Knowledge-based and Empirical Methods. Springer. find
10. Arfath Pasha, Mohamed Al-Badrashiny, Ahmed El Kholy, Ramy Eskander, Mona Diab, Nizar Habash, Manoj Pooleery, Owen Rambow, and Ryan Roth. 2014. MADAMIRA: A Fast, Comprehensive Tool for Morphological Analysis and Disambiguation of Arabic. In Proceedings of LREC, Reykjavik, Iceland. find our card
11. Nizar Y Habash. 2010. Introduction to Arabic natural language processing, volume 3. Morgan & Claypool Publishers. find
12. Georg Heigold, Josef van Genabith, and Günter Neumann. 2016. Scaling character-based morphological tagging to fourteen languages. In 2016 IEEE International Conference on Big Data (Big Data), pages 3895–3902. find
13. Arfath Pasha, Mohammad Al-Badrashiny, Mona Diab, Nizar Habash, Manoj Pooleery, and Owen Rambow. 2013. MADAMIRA v1.0 User Manual. http:// www.nizarhabash.com/publications/ MADAMIRA-UserManual.pdf. ↗
14. Zhiheng Huang, Wei Xu, and Kai Yu. 2015. Bidirectional LSTM-CRF models for sequence tagging. arXiv preprint arXiv:1508.01991. ↗
15. Marek Rei, Gamal KO Crichton, and Sampo Pyysalo. 2016. Attending to characters in neural sequence labeling models. arXiv preprint arXiv:1611.04361. ↗
16. Go Inoue, Hiroyuki Shindo, and Yuji Matsumoto. 2017. Joint prediction of morphosyntactic categories for fine-grained Arabic part-of-speech tagging exploiting tag dictionary information. In Proceedings of the 21st SIGNLL Conference on Computational Natural Language Learning (CoNLL), Vancouver, Canada. find our card
17. Ryan Roth, Owen Rambow, Nizar Habash, Mona Diab, and Cynthia Rudin. 2008. Arabic morphological tagging, diacritization, and lemmatization using lexeme models and feature ranking. In ACL 2008: The Conference of the Association for Computational Linguistics, Columbus, Ohio. find our card
18. Anas Shahrour, Salam Khalifa, and Nizar Habash. 2015. Improving Arabic diacritization through syntactic analysis. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1309–1315, Lisbon, Portugal. find our card
19. Salam Khalifa, Nasser Zalmout, and Nizar Habash. 2016. Yamama: Yet another multi-dialect Arabic morphological analyzer. In Proceedings of the International Conference on Computational Linguistics (COLING): System Demonstrations, pages 223– 227, Osaka, Japan. find
20. Diederik P. Kingma and Jimmy Ba. 2014. A method for stochastic optimization. abs/1412.6980. ↗
21. Qinlan Shen, Daniel Clothiaux, Emily Tagtow, Patrick Littell, and Chris Dyer. 2016. The role of context in neural morphological disambiguation. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 181–191, Osaka, Japan. Adam: CoRR, Matthieu Labeau, Kevin Löser, Alexandre Allauzen, and Rue John von Neumann. 2015. Non-lexical neural architecture for fine-grained pos tagging. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 232–237. find
22. Andreas Stolcke. 2002. SRILM - an Extensible Language Modeling Toolkit. In Proceedings of the International Conference on Spoken Language Processing (ICSLP), volume 2, pages 901–904, Denver, CO. find
23. Mohamed Maamouri, Ann Bies, Tim Buckwalter, and Wigdan Mekki. 2004. The Penn Arabic Treebank: Building a Large-Scale Annotated Arabic Corpus. In NEMLAR Conference on Arabic Language Resources and Tools, pages 102–109, Cairo, Egypt. find
24. Peilu Wang, Yao Qian, Frank K. Soong, Lei He, and Hai Zhao. 2015. Part-of-speech tagging with bidirectional long short-term memory recurrent neural network. CoRR, abs/1510.06168. ↗
25. Eray Yildiz, Caglar Tirkaz, H. Bahadir Sahin, Mustafa Tolga Eren, and Ozan Sonmez. 2016. A morphology-aware network for morphological disambiguation. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, AAAI’16, pages 2863–2869. AAAI Press. find
26. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. CoRR, abs/1301.3781. ↗
27. J.A. Nelder and R. Mead. 1965. A simplex method for function minimization. The Computer Journal, 7:308–313. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2017-zalmout-don-t-throw-those-morphological/