An Open-Source Finite State Morphological Transducer for Modern Standard Arabic
What it is about
Attia et al. build AraComLex, an open-source finite-state morphological analyzer for Modern Standard Arabic, compiled with Foma from a lemma-based lexc lexicon and about 130 alteration rules. They grow a seed lexicon of 10,799 lemmas to 30,587 lemmas. The additions come from a billion-word corpus pre-annotated with MADA, and from SAMA 3.1. They drop SAMA entries with no web hits or fewer than ten corpus occurrences. A small multilayer perceptron predicts continuation classes, humanness and transitivity for new lemmas. Broken plurals are extracted by Levenshtein distance on glosses (1,965 of 2,266 candidates pass pattern validation). Against SAMA, coverage is about 1 point lower (85.73% against 86.95% on the semi-literary text), but the analyses per word fall to 2.08 from 5.30.
Ideas
- I1. Corpus-frequency filter for a seed lexicon: keep a lemma only if it has at least one web hit and at least ten corpus occurrences. Run this per language layer (biblical, mishnaic) with counts from the corpora Mova already holds, so that layers are not mixed.
- I2. Report ambiguity next to coverage: the average number of analyses per token, as the paper does. Add it to every
tools/hboandtools/macularesult, on the same held-out text. - I3. A lemma with alteration rules rather than a list of every stem. For Aramaic, the verb and noun chapters can list lemmas and generate the forms with a few replace rules, and the result can be checked against the Aramaic texts the chapters use.
- I4. Predict humanness and transitivity for new lemmas from frequency statistics, as a suggestion only. The rule that uses the class stays in Mova and decides the outcome.
For Mova
I1 goes into The lexicon and The lexicon of Biblical Aramaic. I2 is a reporting rule for tools/hbo and tools/macula. I3 goes into The verb in Biblical Aramaic and The noun in Biblical Aramaic. I4 is an idea for prag and the lexicon of world, kept as a suggestion that the rules check.
Cite
@inproceedings{attia2011open,
title = {{An Open-Source Finite State Morphological Transducer for Modern Standard Arabic}},
author = {Attia, Mohammed and Pecina, Pavel and Toral, Antonio and Tounsi, Lamia and Genabith, Josef van},
booktitle = {Proceedings of the 9th International Workshop on Finite State Methods and Natural Language Processing},
year = {2011},
url = {https://aclanthology.org/W11-4417/}
}References
- Atkins, B. T. S. and Rundell, M. 2008. The Oxford Guide to Practical Lexicography. Oxford University Press. find
- Attia, M. 2006. An Ambiguity-Controlled Morphological Analyzer for Modern Standard Arabic Modelling Finite State Networks. In: Challenges of Arabic for NLP/MT Conference, The British Computer Society, London, UK. find
- Beesley, K. R. 1998. Arabic Morphological Analysis on the Internet. In: The 6th International Conference and Exhibition on Multilingual Computing, Cambridge, UK. find
- Beesley, K. R. 2001. Finite-State Morphological Analysis and Generation of Arabic at Xerox Research: Status and Plans in 2001. In: The ACL 2001 Workshop on Arabic Language Processing: Status and Prospects, Toulouse, France. find
- Beesley, K. R., and Karttunen, L. 2003. Finite State Morphology: CSLI studies in computational linguistics. Stanford, Calif.: Csli. find
- Bin-Muqbil, M. 2006. Phonetic and Phonological Aspects of Arabic Emphatics and Gutturals. Ph.D. thesis in the University of WisconsinMadison. find
- Buckwalter, T. 2004. Buckwalter Arabic Morphological Analyzer (BAMA) Version 2.0. Linguistic Data Consortium (LDC) catalogue number LDC2004L02, ISBN1-58563-324-0. find
- Dichy, J., and Farghaly, A. 2003. Roots & Patterns vs. Stems plus Grammar-Lexis Specifications: on what basis should a multilingual lexical database centred on Arabic be built? In: The MT-Summit IX workshop on Machine Translation for Semitic Languages, New Orleans. find
- Elgibali, A. and Badawi, E. M. 1996. Understanding Arabic: Essays in Contemporary Arabic Linguistics in Honor of El-Said M. Badawi. American University in Cairo Press, Egypt. find
- Fischer, W. 1997. Classical Arabic. In: The Semitic Languages. London: Routledge. find
- Haykin, S. 1998. Neural Networks: A Comprehensive Foundation (2 ed.). Prentice Hall. find
- Hulden, M. 2009. Foma: a finite-state compiler and library. In: Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics (EACL ’09). Stroudsburg, PA, USA. find
- Kiraz, G. A. 2001. Computational Nonlinear Morphology: With Emphasis on Semitic Languages. Cambridge University Press. find
- Maamouri, M., Graff, D., Bouziri, B., Krouna, S., and Kulick, S. 2010. LDC Standard Arabic Morphological Analyzer (SAMA) v. 3.1. LDC Catalog No. LDC2010L01. ISBN: 1-58563-555-3. find
- Parker, R., Graff, D., Chen, K., Kong, J., and Maeda, K. 2009. Arabic Gigaword Fourth Edition. LDC Catalog No. LDC2009T30. ISBN: 1-58563-532-4. find
- Roth, R., Rambow, O., Habash, N., Diab, M., and Rudin, C. 2008. Arabic Morphological Tagging, Diacritization, and Lemmatization Using Lexeme Models and Feature Ranking. In: Proceedings of Association for Computational Linguistics (ACL), Columbus, Ohio. find our card
- Sinclair, J. M. (ed.). 1987. Looking Up: An Account of the COBUILD Project in Lexical Computing. London: Collins. find
- Stetkevych, J. 1970. The modern Arabic literary language: lexical and stylistic developments. Publications of the Center for Middle Eastern Studies, No. 6. Chicago and London: University of Chicago Press. find
- Van Mol, M. 2003. Variation in Modern Standard Arabic in Radio News Broadcasts, A Synchronic Descriptive Investigation in the use of complementary Particles. Leuven, OLA 117. find
- Watson, J. 2002. The Phonology and Morphology of Arabic, New York: Oxford University Press. find
- Wehr, H. and Cowan, J. M. 1976. Dictionary of Modern Written Arabic, pp. VII-XV. Ithaca, N.Y.: Spoken Language Services. find
This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2011-attia-open-source-finite-state-morphological/