Building a Shallow Arabic Morphological Analyser in One Day

What it is about

The paper describes Sebawai, a shallow analyser that returns candidate roots for an Arabic word and is learned from word-root pairs. The pairs were produced by an existing analyser (ALPNET) on 9,606 words of traditional text and on 270,000 news words. For each pair the system extracts the prefix, the suffix and the stem template, counts them, and ranks candidate roots by the product of the three probabilities, with Witten-Bell smoothing that drops items seen only once. Two-letter stems and weak letters are handled by generating substitute stems, checked against a root list. The system was built in under twelve hours, in about 200 lines of Perl. On the 9,606-word set it gave a correct top root for 86.4% of the words it analysed (a manual check put this nearer 93%). In retrieval on the Zad collection, indexing by its top root gave a mean average precision of 0.45, against 0.34 for ALPNET's top root, while indexing by its top two roots fell to 0.29.

Ideas

• I1. Learn the prefix, suffix and template tables from the analyses of tools/hbo with counts, and smooth by dropping once-seen items. The paper's ranking multiplies three independent probabilities, which it admits is wrong for illegal combinations; measure how many analyses change when the smoothing is added.
• I2. Keep one root by default: the second root hurt retrieval badly. For any root-based index in Mova, return the top analysis and report a second only when the score gap is small; test on a retrieval set.
• I3. For defective Hebrew roots, generate the variants of a short stem (doubled last letter, weak letter in the middle or at the ends) and keep only those that exist in the root list of the lexicon.

For Mova

I1 goes into the segmenter tables (lex, pre) of tools/hbo. I2 is a rule for any root index in tools/hbo and a retrieval measure. I3 goes into The weak verbs, with the lexicon in The lexicon as the filter.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.3115/1118637.1118643

@inproceedings{darwish2002building,
  title = {{Building a Shallow Arabic Morphological Analyser in One Day}},
  author = {Darwish, Kareem},
  booktitle = {Proceedings of the ACL-02 Workshop on Computational Approaches to Semitic Languages},
  year = {2002},
  doi = {10.3115/1118637.1118643},
  url = {https://doi.org/10.3115/1118637.1118643}
}

References

14 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

2. Al-Kharashi, Ibrahim and Martha Evens, “Comparing Words, Stems, and Roots as Index Terms in an Arabic Information Retrieval.” JASIS. 45 (8): 548-560, 1994. find
3. Abu-Salem, Hani, Mahmoud Al-Omari, and Martha Evens, “Stemming Methodologies Over Individual Query Words for Arabic Information Retrieval.” JASIS. 50 (6): 524-529, 1999. find
4. Hmeidi, Ismail, Ghassan Kanaan, and Martha Evens, “Design and Implementation of Automatic Indexing for Information Retrieval with Arabic Documents.” JASIS. 48 (10): 867-881, 1997. find
5. Abdul-Al-Aal, Abdul-Monem, An-Nahw Ashamil. Maktabat Annahda Al-Masriya, Cairo, Egypt, 1987. find
6. Ahmed, Mohamed Attia, “A Large-Scale Computational Processor of the Arabic Morphology, and Applications.” A Master’s Thesis, Faculty of Engineering, Cairo University, Cairo, Egypt, 2000. find
7. Koskenniemi, Kimmo, “Two Level Morphology: A General Computational Model for Word-form Recognition and Production.” Publication No. 11, Dept. of General Linguistics, University of Helsinki, Helsinki, 1983. find
8. Beesley, Kenneth, Tim Buckwalter, and Stuart Newton, “Two-Level Finite-State Analysis of Arabic Morphology.” Proceedings of the Seminar on Bilingual Computing in Arabic and English, Cambridge, England, 1989. find
9. Beesley, Kenneth, “Arabic Finite-State Morphological Analysis and Generation.” COLING-96, 1996. find
10. Kiraz, George, “Arabic Computation Morphology in the West.” Proceedings of the 6th International Conference and Exhibition on Multi-lingual Computing, Cambridge, 1998. find
12. Goldsmith, John, “Unsupervised Learning of the Morphology of a Natural Language.” http://humanities.uchicago.edu/faculty/golds mith/, 2000. ↗
13. Al-Areeb Electronic Publishers, LLC. 16013 Malcolm Dr., Laurel, MD 20707, USA find
15. Jurafsky, Daniel and James Martin, “Speech and Language Processing.” Prentice Hall, New Jersey, 2000. find
16. Antworth, Evan L, “PC-KIMMO: a two-level processor for morphological analysis.“ Occasional Publications in Academic Computing No. 16. Dallas, TX: Summer Institute of Linguistics, 1990. find
17. Antworth, Evan L, “PC-KIMMO User’s Guide.” www.sil.org/pckimmo/v2/doc/guide.html, 1995 find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2002-darwish-building-shallow-arabic-morphological-analyser/