MADAMIRA: A Fast, Comprehensive Tool for Morphological Analysis and Disambiguation of Arabic

What it is about

MADAMIRA merges two older Arabic tools, MADA (morphology and disambiguation) and AMIRA (tokenization, tagging, chunking), into one Java system. The pipeline is: preprocessing, a list of all analyses per word from SAMA (MSA) or CALIMA (Egyptian), feature prediction by linear SVMs (closed-class features) and language models (lemma and diacritics), ranking of the analyses by agreement with those predictions, then tokenization in 11 schemes, base-phrase chunking and named-entity tags. It runs as a stand-alone tool or as a server. On blind test sets of about 25K MSA words and 20K Egyptian words, its full-match accuracy is within 0.2 points of MADA for MSA (84.1 against 84.3) and within 0.6 for Egyptian. The paper reports speed-ups of 16 to 21 times for MSA and 14 to 19 times for Egyptian, against the older tools. The licence is non-commercial research only.

Ideas

• I1. One analysis, several token views: from the single chosen analysis, produce a set of named tokenization schemes (all clitics split; only conjunctions and question particles split; a surface form with the article kept; a form with the POS tags). For Hebrew, give tools/hbo the same kind of named schemes, so its output can be compared with the MACULA ATB-like tokenization in macula compare.
• I2. Keep a ranked list of analyses with a score per analysis, and let each downstream step (lemma, POS, gender) vote for one of them. Output TOP or ALL, as MADAMIRA does. This gives the explain output of tools/hbo a list of alternatives, each with the reason for its rank.
• I3. Report five separate scores on the same blind set: diacritized form, lemma, POS, full morphological match, and tokenization. Use the same five for every tools/hbo run, so a change in one step is not hidden inside one total.

For Mova

I1 goes into tools/hbo (the segmenter output) and 13. Biblical Hebrew in Mova: the tools. I2 goes into the explain output of tools/hbo and the ranking step of gparse use. I3 is the evaluation table for tools/hbo, with the scores kept in the repository's test notes. The tool itself is research-only, so no code or model is taken from it.

Cite

The paper itself, not this card. BibTeX from the library's record of the work.

@inproceedings{pasha2014madamira,
  title = {{MADAMIRA: A Fast, Comprehensive Tool for Morphological Analysis and Disambiguation of Arabic}},
  author = {Pasha, Arfath and Al-Badrashiny, Mohamed and Diab, Mona and Kholy, Ahmed El and Eskander, Ramy and Habash, Nizar and Pooleery, Manoj and Rambow, Owen and Roth, Ryan},
  booktitle = {Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)},
  year = {2014},
  url = {https://aclanthology.org/L14-1479/}
}

References

10 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Al-Sabbagh, R. and Girju, R. (2012). A supervised POS tagger for written Arabic social networking corpora. In EGY MADA-ARZ MADAMIRA v0.4 v1.0 83.8 83.2 87.8 87.8 91.8 92.4 77.5 77.3 98.9 99.2 96.6 97.6 96.5 97.4 420.2 1013.4 44.9 – 389.1 844.1 find
2. Jancsary, J., editor, Proceedings of KONVENS 2012, pages 39–52. ÖGAI, September. Main track: oral presentations. find
3. Buckwalter, T. (2004). Buckwalter Arabic Morphological Analyzer Version 2.0. LDC catalog number LDC2004L02, ISBN 1-58563-324-0. find
4. Buckwalter, T. (2007). Issues in Arabic Morphological Analysis. In van den Bosch, A. and Soudi, A., editors, Arabic Computational Morphology: Knowledge-based and Empirical Methods. Springer. find
5. Diab, M., Hacioglu, K., and Jurafsky, D. (2007). Automated Methods for Processing Arabic Text: From Tokenization to Base Phrase Chunking. In van den Bosch, A. and Soudi, A., editors, Arabic Computational Morphology: Knowledge-based and Empirical Methods. Kluwer/Springer. find
6. Diab, M., Habash, N., Rambow, O., and Roth, R. (2013). LDC Arabic Treebanks and Associated Corpora: Data Divisions Manual. Technical Report CCLS-13-02, Center for Computational Learning Systems, Columbia University. find
7. Fan, R.-E., Chang, K.-W., Hsieh, C.-J., Wang, X.-R., and Lin, C.-J. (2008). Liblinear: A library for large linear classification. Journal of Machine Learning Research, 9:1871–1874. find
8. Graff, D., Maamouri, M., Bouziri, B., Krouna, S., Kulick, S., and Buckwalter, T. (2009). Standard Arabic Morphological Analyzer (SAMA) Version 3.1. Linguistic Data Consortium LDC2009E73. find
9. Habash, N. and Rambow, O. (2005). Arabic Tokenization, Part-of-Speech Tagging and Morphological Disambiguation in One Fell Swoop. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL’05), pages 573–580, Ann Arbor, Michigan. find our card
10. Habash, N. and Rambow, O. (2006). MAGEAD: A Morphological Analyzer and Generator for the Arabic Dialects. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, pages 681–688, Sydney, Australia. find our card

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/lingbulk-2014-pasha-madamira-fast-comprehensive-tool-morphological/