Identifying Broken Plurals in Unvowelised Arabic Text
What it is about
Broken plurals (formed by changing the consonant pattern of the singular, not by adding a suffix) cannot be undone by affix stripping, so a light stemmer leaves them as separate stems. The authors compare four ways of detecting them in unvowelised text. The first is simple matching of 39 grammar-book patterns after light stemming, which gives high recall (99.5%) and low precision (13.8%) on data set 2. The second restricts the pattern letters by hand, which lifts precision to about 50%. The third is a decision tree (C4.5) per pattern. The fourth is a dictionary of about 3,580 broken-plural stems, built from an 18.5-million-word corpus and checked by a linguist. The dictionary gives precision 86.9% and F 92.5 on data set 2, and with affix refinement 92.3% and F 96.0. A light stemmer with broken-plural recognition, indexing "stem + BP", beats full-word, root and basic-stem indexing in a 50-query retrieval test (p < .01).
Ideas
- I1. A table of restricted templates with exceptions: write each broken-plural pattern as fixed letters plus a set of allowed letters in the meta-letter slots, and a short list of known exceptions (the paper's verbal nominalisations with a t-infix). It is rule-based, readable, and can be measured on its own. Candidate place: the noun-class tables in The Noun, the Adjective and the Numeral, and the plural classes in The noun in Biblical Aramaic.
- I2. A bootstrapped lexicon: run a cheap, high-recall rule over a large corpus, hand-correct the output once, and keep the corrected list as a lookup table. The paper's 3,580-stem list shows the cost is small once the cheap rule exists. For Mova, the same step can build lemma lists from the MACULA-derived lexicon in
tools/macula dict. - I3. A retrieval measure: a 50-query test with relevance judgments, comparing full word, root and lemma indexes, as the paper does. Mova has no search test of this kind yet; it would measure whether lemma conflation helps before any stemmer is built.
For Mova
I1 is a candidate test for the noun chapters: The Noun, the Adjective and the Numeral for Ancient Hebrew plural classes, and The noun in Biblical Aramaic for Aramaic. I2 uses tools/macula dict as the seed. I3 is a measure, not a rule; it is only worth running if Mova gets a retrieval component. Arabic itself is not a Mova language yet.
Cite
@inproceedings{goweder2004identifying,
title = {{Identifying Broken Plurals in Unvowelised Arabic Text}},
author = {Goweder, Abduelbaset and Poesio, Massimo and Roeck, Anne De and Reynolds, Jeff},
booktitle = {Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing},
year = {2004},
url = {https://aclanthology.org/W04-3232/}
}References
- Abuleil, Saleem and Evens, Martha W. (1998). “Discovering Lexical Information by Tagging Arabic Newspaper Text.” Computational Approaches to Semitic Languages, Proceedings of the Workshop. find
- Abu-Salem, Hani; Al-Omari, Mahmoud; and Evens, Martha W. (1999). “Stemming Methodologies over Individual Query Words for an Arabic Information Retrieval System.” JASIST, 50(6):524-529. find
- Ali , N. (1988). “Computers and the Arabic Language.” Cairo, Egypt: Al-khat Publishing Press, Ta’reep. find
- Al-Kharashi, I. and Evens, Martha W. (1994). “Comparing words, stems and roots as index terms in an Arabic Information retrieval system.” JASIST, 45(8):548-560. find
- Baeza-Yates, Ricardo and Ribeiro-Neto, Berthier (1999). “Modern Information Retrieval”. Addison-Wesley, ACM Press. find
- Beesley, K. R. (1996) “Arabic finite-state morphological analysis and generation.” In COLING-96: Proceedings of the 16th international conference on Computational Linguistics, vol. 1, pp. 89--94. find
- Boudelaa , Sami; Gaskell, M. Gareth (2002). “A re-examination of the default system for Arabic plurals.” Psychology Press Ltd, vol. 17, pp. 321-343, 2002. find
- Cowan, David (1958). “Modern Literary Arabic.” Cambridge University Press. find
- Goweder, Abduelbaset and De Roeck, Anne (2001). “Assessment of a Significant Arabic Corpus.” ACL 2001. Arabic language Processing. pp. 73-79, 2001. find
- Haywood, J. A. and Nahmad, H. M. (1976). “A new Arabic Grammar of the written language.” Lund Humphries London. find
- Hmeidi, Ismail; Kanaan, Ghassan; and Evens, Martha (1997). “Design and Implementation of Automatic Indexing for Information Retrieval with Arabic Documents.” Journal of the American Society for Information Science. 48(10) (pp. 867-881), 1997. find
- Idrissi, Ali (1997). “Plural Formation in Arabic.” In Current issues in Linguistic Theory, Perspectives on Arabic Linguistics X. Edited by Mushira Eid and Robert Ratcliffe. Vol 153, pp 123-145. find
- Khoja, S. and Garside, R. (1999) “Stemming Arabic text.” Computing Department, Lancaster University, Lancaster, United Kingdom. http://www.comp.lancs.ac.uk/computing/users/khoja/ste mmer.ps Kiraz, G. (1996a). Analysis of the Arabic broken plural and diminutive. In Proceedings of the 5th International Conference and Exhibition on Multi-Lingual Computing. Cambridge. ↗
- Larkey, L. S. and Connell, M. E. (2001) “Arabic information retrieval at UMass in TREC-10.” In TREC 2001. Gaithersburg: NIST, 2001. find
- Larkey, L.; Ballesteros, L.; and Connell, M.E (2002). “Improving Stemming for Arabic Information Retrieval: Light Stemming and Co-occurrence Analysis.” In SIGIR’02, August 11-15, 2002, Tampere, Finland, pp 275–282, 2002. find
- Manning, Christopher D. and Schutze, Hinrich (1999). “Foundations of Statistical Natural Language Processing.” McCarthy, John J.; and Prince, Alan S (1990). “Foot and Word in Prosodic Morphology: The Arabic Broken Plural.” Natural Language and Linguistic Theory 8, 209–282. find
- Quinlan, J. R. (1993). “C4.5: Programs for Machine Learning.” San Mateo: Morgan Kaufmann. find our card
This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2004-goweder-identifying-broken-plurals-unvowelised-arabic/