A Comparative Study of Root-Based and Stem-Based Approaches for Measuring the Similarity between Arabic Words

What it is about

The authors ask how the choice of word normalisation changes the similarity between Arabic words in a distributional model. They build Latent Semantic Analysis (word-by-paragraph co-occurrence matrix, SVD, reduction to a few hundred dimensions) over two preprocessing schemes: a root-based one (Khoja's morphological analyser, which extracts a triliteral root) and a stem-based one (Larkey's light stemmer, which only strips frequent affixes). Word similarity is then measured with four functions: Euclidean distance, cosine, Jaccard and Pearson correlation. The abstract reports that the stem-based approach gives the better similarity, because root extraction often merges words with unrelated meanings. A root-extractor error is shown as an example: a word meaning "organisations" is reduced to a root that means "he was thirsty".

Ideas

• I1. Make a root-versus-lemma test for Hebrew: for every root shared by two or more lemmas in the macula dict Torah dictionary, check the glosses and count pairs with unrelated meanings. The count measures the over-grouping that Froud describe for Arabic, and it tells us how far a root key can be trusted in world before we build on it.
• I2. Run the same comparison in Mova with a deterministic co-occurrence table (BTreeMap) built from a treebank: similarity keyed by root against similarity keyed by lemma, using cosine and Jaccard. Report both runs and check they are identical before any claim.
• I3. Their error example (a noun mis-rooted to a verb of thirst) is a useful negative case for any root table built for Hebrew: a root must match the stem's consonant order and its gloss, not only its letters.

For Mova

I1 and I3 are checks for the root notes in The lexicon and macula dict. I2 is a measure for world (similarity of words), to be run on treebanks/hbo/ptnk first. The paper's LSA itself is not needed: Mova uses frames, not a vector space.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. arXiv 1212.3634

@misc{lachkar2012comparative,
  title = {{A Comparative Study of Root-Based and Stem-Based Approaches for Measuring the Similarity between Arabic Words}},
  author = {Lachkar;, Hanane Froud; Abdelmonaim and Ouatik, Said Alaoui},
  year = {2012},
  eprint = {1212.3634},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1212.3634}
}

References

22 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Bellot, P. and M. El-Bèze. Clustering by means of Unsupervised Decision Trees or Hierarchical and K-Means-like Algorithm. Proc. of RIAO 2000, 344–363. find
2. H.Ghwanmeh (2007). Applying Clustering of Hierarchical K-Means-like Algorithm on Arabic Language. International Journal of Information Technology IJIT 3, 168–172. find
3. G.Salton and M.McGill (1983). Introduction to Modern Information McGraw-Hill. find
4. H.Schutze and C. Silverstein (1997). Projections for Efficient Document Clustering. In Proceedings of SIGIR’97, 74–81. find
5. H.Froud, A.Lachkar, S. Ouatik, and R.Benslimane (2010). Towards a Modern Arabic Information Retrieval System: Arabic Documents Clustering using Singular Value Decomposition. Journées Doctorales en Technologies de l’Information et de la Communication JDTIC10,Fès. Morocco. find
6. C.Bellissens, P. and G.Denhière (2004). Deux Modèles Vectoriels de la Mémoire Sémantique : Description, Théorie et Derspectives. Le Langage et l’Homme. find
7. H.Froud, A.Lachkar, S. Ouatik, and R.Benslimane (2010). Stemming and Similarity Measures for Arabic Documents Clustering. 5th International Symposium on I/V Communications and Mobile Networks ISIVC, IEEE Xplore. find
8. C.Barrière and S.Zaki (2001). Révision d’une Hypothèse Fausse pour les Mesures de Similarité des Mots dans les Corpus. find
9. T. LANDAUER and S. DUMAIS (1997). A Solution to Plato’s Problem: The Latent Semantic Analysis Theory of the Acquisition, Induction, and Representation of Knowledge. Psychological Review 104, 211–240. find
10. S.Khoja and R.Garside (1999). Stemming Arabic Text. Computing Department, Lancaster University, Lancaster, http://www.comp.lancs.ac.uk/computing/users/khoja/stemmer.ps. ↗
11. L.S.Larkey, L.Ballesteros, & M.E.Connell. (2002). Improving Stemming for Arabic Information Retrieval: Light Stemming and Co-occurrence Analysis. The 25th Annual International Conference on Research and Development in Information Retrieval (SIGIR 2002), Tampere, Finland, August 11-15, 2002, (pp. 275-282). find
12. A.Huang, “Similarity Measures for Text Document Clustering”, NZCSRSC 2008, April 2008, Christchurch, New Zealand.. find
13. B. Yates and R. Neto.”Modern Information Retrieval”. ADDISON-WESLEY, New York, 1999. find
14. B. Larsen and C. Aone.” Fast and Effective Text Mining using Linear-time Document Clustering”. In Proceedings of the Fifth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 1999. find
15. H.Froud, A.Lachkar, and S.Ouatik. (2012). Stemming for Arabic Words Similarity Measures based on Latent Semantic Analysis Model. The 3rd International Conference on Multimedia Computing and Systems (ICMCS'12), May 10-12,. Tangier, Morocco, IEEE Xplore. find
16. R. Duwairi, M. Al-Refai, and N. Khasawneh, “Stemming Versus Light Stemming as Feature Selection Techniques for Arabic Text Categorization”. Innovations in Information Technology, 2007. IIT '07. 4th International Conference, 446 – 450. find
17. Al-Fedaghi S. and F. Al-Anzi. “A new algorithm to generate Arabic root-pattern forms”. In proceedings of the 11th national Computer Conference and Exhibition. PP 391-400. March 1989. Retrieval. New York: find
18. Al-Shalabi R. and M. Evens. “A computational morphology system for Arabic”. In Workshop on Computational Approaches to Semitic Languages, COLING-ACL98. August 1998. find
19. Aljlayl M. and O. Frieder. “On Arabic search: improving the retrieval effectiveness via a light temming approach”. In ACM CIKM 2002 International Conference on Information and Knowledge Management, McLean, VA, USA. PP 340-347. 2002. find
20. Larkey L., and M. E. Connell. “Arabic information retrieval at UMass in TREC-10”. Proceedings of TREC 2001, Gaithersburg: NIST. 2001. find
21. Chen A. and F. Gey. “Building an Arabic Stemmer for Information Retrieval”. In Proceedings of the 11th Text Retrieval Conference (TREC 2002), National Institute of Standards and Technology. 2002. find
22. Abu-Hamdiyyah, Mohammad.2000. “The Qur'An: An Introduction” Authors Miss. Hanane Froud Phd Student in Laboratory of Information Science and Systems, ECOLE NATIONALE DES SCIENCES APPLIQUÉES, University Sidi Mohamed Ben Abdellah (USMBA), Fez, Morocco. She has also presented different papers at different National and International conferences. Pr. Abdelmonaime LACHKAR received his PhD degree from the USMBA, Morocco in 2004 in computer science; He is a Professor in the Head of Computer Science and Engineering E.N.S.A, University Sidi Mohamed Ben Abdellah (USMBA), Fez, Morocco. His current research interests include Arabic Text Mining Applications: Arabic Web Document Clustering and Categorization. Arabic Information and Retrieval Systems, Arabic Text Summarization, ect …, Image Indexing and Retrieval, 3D ShapeI Indexing and Retrieval in large 3D Objects DataBases, Colour Image Segmentation, Unsupervised clustering, Cluster Validity Index, on-line and off-line Arabic and Latin handwritten recognition, and Medical Image Applications. Pr. Said Alaoui Ouatik is working as Professor in Department of Computer Science, Faculty of Science Dhar EL Mahraz (FSDM), Fez, Morocco. His research interests include high-dimensional indexing and content-based retrieval, Arabic Document Categorization. 2D/3D Shapes Indexing and Retrieval in large 3D Objects DataBase. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/langstream-2012-froud-comparative-study-root-based-stem/