A Morphologically Sensitive Clustering Algorithm for Identifying Arabic Roots

What it is about

The paper groups Arabic words that share a root, for indexing in retrieval, with no dictionary. It starts from Adamson and Boreham's bigram Dice score with single-link clustering and adds: light stemming to strip obvious affixes; word-boundary blanks; a "cross" bigram that joins the letters on either side of a weak letter; per-bigram weights (0.25 for weak letters, 0.5 for suspected affixes, 1 for the rest); and a Jaccard score in place of Dice. It was tested on five hand-clustered samples of unedited newspaper Arabic, from 232 to 750 words, with the three development sets used for tuning. On the held-out fifth set (314 words, a different domain) multi-word clusters containing infixes were found at 72.72%, against 9.09% for the baseline, and the abstract reports accuracy up to 94.06%. Single-word clusters stayed as clean as the baseline. The authors say the data are small and that scale is untested.

Ideas

• I1. Use the weighting scheme for unknown Hebrew words: lower the weight of weak consonants (aleph, waw, yod, and he of lamed-he verbs) and mark the prefix letters (ה ו ב כ ל מ ש) as suspect, in a similarity score between an unknown unpointed token and the known lemmas in tools/hbo.
• I2. The cross bigram for Hebrew verbs that lose a first or middle radical (pe-yod, ayin-waw): add the bigram that joins the neighbours of the weak letter, and measure on the verb forms in MACULA.
• I3. A held-out test: cluster a few hundred Hebrew forms by hand into root groups, count the recall of multi-word groups that contain weak or infixed forms, and run it once per release. Recall is the measure, as in the paper.

For Mova

I1 and I2 apply to the fallback that guesses the lemma of an unknown word in tools/hbo, with the lexicon in The lexicon. I3 is a held-out test set for that fallback.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.3115/1075218.1075244

@inproceedings{roeck2000morphologically,
  title = {{A Morphologically Sensitive Clustering Algorithm for Identifying Arabic Roots}},
  author = {Roeck, Anne N. De and Al-Fares, Waleed},
  booktitle = {Proceedings of the 38th Annual Meeting of the Association for Computational Linguistics},
  year = {2000},
  doi = {10.3115/1075218.1075244},
  url = {https://doi.org/10.3115/1075218.1075244}
}

References

21 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Adams, E. (1991) A Study of Trigrams and their feasibility as Index Terms in a full text Information Retrieval System. PhD Thesis, George Washington University, USA. find
2. Adamson, George W. and J. Boreham (1974) The use of an association measure based on character structure to identify semantically related pairs of words and document titles. Information Storage and Retrieval,. Vol 10, pp 253-260 find
3. Al-Fedaghi Sabah S. and Fawaz Al-Anzi (1989) A new algorithm to generate Arabic root-pattern forms. Proceedings of the 11th National Computer Conference, King Fahd University of Petroleum & Minerals, Dhahran, Saudi Arabia., pp04-07 find
4. Al-Kharashi, I. and M. Evens (1994) Comparing words, stems, and roots as Index terms in an Arabic Information Retrieval system. Journal of the American Society for Information Science, 45/8, pp. 548-560 find
5. Al-Najem, Salah R. (1998). An Explanation of Computational Arabic Morphology. DATR Documentation Report, University of Sussex. find
6. Al-Raya (1997) Newspaper. Quatar. find
7. Al-Shalabi, R. and M. Evens (1998) A Computational Morphology System for Arabic. Proceedings of COLING-ACL, New Brunswick, NJ. find
8. Al-Watan (2000) Newspaper. Qatar. find
9. Beesley, K.B. (1996) Arabic Finite-State Morphological Analysis and Generation. Proceedings of COLING-96, pp 89-94. find
10. Beesley, K.B. (1998) Arabic Morphological Analysis on the Internet. Proceedings of the 6 th International Conference and Exhibition on Multi-Lingual Computing, Cambridge. find
11. El-Sadany, T. and M. Hashish (1989) An Arabic morphological system. IBM System Journal, 28/4 find
12. Harman, D. (1991) How effective is suffixing? Journal of the American Society for Information Science, 42/1, pp 7-15. find
13. Hmeidi, I., Kanaan, G. and M. Evens (1997) Design and Implementation of Automatic Indexing for Information Retrieval with Arabic Documents. Journal of the American Society for Information Science, 48/10, pp. 867-881. find
14. Kiraz, G. (1994) Multi-tape two-level Morphology: a case study in Semitic non-linear morphology. Proceedings of COLING-94, pp180-186. find our card
15. Lovins, J.B. (1968) Development of a Stemming Algorithm. Mechanical Translation and Computational Linguistics, 11/1. find
16. Popovic, M. and P. Willet (1992) The effectiveness of stemming for natural language access to Sloven textual data. Journal of the American Society for Information Science, 43/5, pp. 384-390. find
17. Porter, M.F. (1980) An Algorithm for suffix stripping. Program, 14 /3, pp 130-137 find
18. Stalls, B. and Knight, K. (1998) Translating names and technical terms in Arabic text. Proceedings of COLING-ACL, New Brunswick, NJ, 1998 find
19. van Rijsbergen, C. J. (1979) Information Retrieval. Butterworths, London. find
20. Robertson, A. and Willett, P.(1992) Searching for historical word-forms in a database of 17 th- century English text using spelling-correction methods. 15 th Annual International Conference SIGIR. find
21. Ubu-Salem H., Al-Omari M., and M. Evens (1999) Stemming methodologies over individual query words for an Arabic information retrieval system. Journal of the American Society for Information Science. 50/6, pp 524-529. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2000-roeck-morphologically-sensitive-clustering-algorithm-identifying/