Mova0.5 pre-release⌕
☰
home / docs / papers / Semitic languages / Restoring Arabic vowels through omission-tolerant dictionary lookup

Restoring Arabic vowels through omission-tolerant dictionary lookup

What it is about

Arabic is written mostly without its short vowels and other diacritics (about 3% of words carry any), so a word form can stand for several vocalised words. The authors build Arabic-Unitex, a lexical resource: a hand-built lexicon of about 76,000 fully vowelised lemmas, inflected by finite-state transducers into about 6 million forms, with grammars for clitic agglutination. They write two dozen typographic rules that describe which vowels may be omitted, and use them for an omission-tolerant lookup that returns every fully vowelised candidate for a written form. The lookup runs at about 5,000 words per second, and the resource is said to cover more than 99% of words in popular newspapers. The same rules give a spell checker that flags a misplaced vowel. The paper also proposes a pattern-and-root view of Semitic morphology, in which patterns take precedence over roots.

Ideas

For Mova

I1 goes into tools/hbo (unpointed mode) and is described in 1. Script and pointing as the lookup rule for points that are optional in the text. I2 is a candidate for The Noun, the Adjective and the Numeral and The verb, tested against macula dict. I3 is a measurement for The lexicon.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.1007/s10579-019-09464-6 · arXiv 1905.04051

@misc{neme2019restoring,
  title = {{Restoring Arabic vowels through omission-tolerant dictionary lookup}},
  author = {Neme, Alexis Amid and Paumier, Sebastien},
  year = {2019},
  doi = {10.1007/s10579-019-09464-6},
  eprint = {1905.04051},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.1007/s10579-019-09464-6}
}

References

63 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abandah, G.A., Graves, A., Al-Shagoor, B., Arabiyat, A., Jamour, F., Al-Taee, M., 2015. Automatic diacritization of Arabic text using recurrent neural networks. Int. J. Doc. Anal. Recogn. 18, 183–197. http://dx.doi.org/10.1007/s10032-015-0242-2. ↗
  2. Abdel-Nour, Jabbour (2006). Dictionnaire Abdel-Nour al-Mufassal Arabe-Français. Dar El-Ilm Lil-Malayin. 10th edition.2034 pages, 3 columns. find
  3. Al-Bawab, M., Mrayati, M., Alam, Y.M., Al-Tayyan, M.H. (1994). A computerized morphosyntactic system of Arabic. In The Arabian Journal of Science and Engineering, 19, 461-480. Published by KFUPM, Saudi Arabia. find
  4. Al-Ghalāyini, Mustafa (2007). “Jāmi3 al-durūs al-’arabiyah” (A university grammar textbook). 1st edition 1912. Dar El Fikr Printers-Publishers, Beirut. 3 volumes, 570 pages. In Arabic. find
  5. Al-Jar, Khalil (1973).Al-mu’jam al-’arabiy al-Hadith. find
  6. Larousse. 1973, 53,500 lexical entries.1300 pages, 2 columns, in Arabic. find
  7. Altantawy, Mohamed; Habash, Nizar; Rambow, Owen (2011). Fast Yet Rich Morphological Analysis. In Proceedings of the 9th International Workshop on Finite State Methods and Natural Language Processing (FSMNLP), pages 116-124. find
  8. Alqahtani, Sawsan; Diab, Mona; Zaghouani, Wajdi (2018). ARLEX: A Large Scale Comprehensive Lexical Inventory for Modern Standard Arabic in the 3rd Workshop on Open-Source Arabic Corpora and Processing Tools, Miyazaki, Japan (OSACT 3, May-2018) pages 1-7. find
  9. Altantawy, Mohamed; Habash, Nizar; Rambow, Owen; Saleh, Ibrahim (2010). Morphological Analysis and Generation of Arabic Nouns: A Morphemic Functional Approach. In Proceedings of the Language Resource and Evaluation Conference (LREC), Malta, pages 851-858. find our card
  10. Attia, M. 2006. An Ambiguity-Controlled Morphological Analyzer for Modern Standard Arabic Modelling Finite State Networks. In: Challenges of Arabic for NLP/MT Conference, The British Computer Society, London, UK. find
  11. Attia, Mohammed, Pavel Pecina, Lamia Tounsi, Antonio Toral, Josef van Genabith. (2011). An Open-Source Finite State Morphological Transducer for Modern Standard Arabic. International Workshop on Finite State Methods and Natural Language Processing (FSMNLP). Blois, France. find our card
  12. Attia., M., Yaseen., M., Choukri., K. (2005). Specifications of the Arabic Written Corpus produced within the NEMLAR project, www.NEMLAR.org. find
  13. Azmi, Aqil, Reham S Almajed. 2015. A survey of automatic Arabic diacritization techniques. Natural Language Engineering, 21, pp 477–495. doi:10.1017/S1351324913000284 ↗
  14. Beesley, Kenneth. 1990. Finite-state description of Arabic morphology. In Proceedings of the Second Cambridge Conference on Bilingual Computing in Arabic and English, September 5-7. No pagination. find
  15. Beesley, Kenneth R. 1991. Computer analysis of Arabic morphology: A two-level approach with detours. In Bernard Comrie and Mushira Eid, editors, Perspectives on Arabic Linguistics III: Papers from the Third Annual Symposium on Arabic Linguistics, pages 155-172. John Benjamins, Amsterdam. Read originally at the Third Annual Symposium on Arabic Linguistics, University of Utah, Salt Lake City, Utah, 3-4 March 1989. find
  16. Beesley, Kenneth R. (1996). Arabic finite state morphological analysis and generation. In Proceedings of the International Conference on Computational Linguistics (COLING), Copenhagen, Center for Sprogteknologi, volume 1, pages 89-94. find
  17. Beesley, Kenneth R.. 1998a. Arabic morphological analysis on the Internet. In ICEMCO-g8, Cambridge, April 17-18. Centre for Middle Eastern Studies. Proceedings of the 6th International Conference and Exhibition on Multi-56 lingual Computing. Paper number 3.1.1; no pagination. find
  18. Beesley, Kenneth R. 1998b. Arabic stem morphotactics via finite-state intersection. Paper presented at the 12th Symposium on Arabic Linguistics, Arabic Linguistic Society, 6-7 March, 1998, Champaign, IL. find
  19. Beesley, Kenneth R. 1998c. Consonant spreading in Arabic stems. In COLING'98. find
  20. Beesley, Kenneth R. 1998d. Constraining separated morphotactic dependencies in finitestate grammars. In FSMNLP-98, Bilkent. find
  21. Beesley, Kenneth R. (2001). Finite-State Morphological Analysis and Generation of Arabic at Xerox Research: Status and Plans in 2001. In Proceedings of the ACL/EACL Workshop ‘Arabic Language Processing: Status and Prospects’, pages 1-8. find
  22. Beesley, Kenneth R., Lauri Karttunen, (2003). Finite State Morphology, CLSI Studies in Computional Linguistics, 509 pages. find
  23. Ben Mesmia F., Friburger N., Haddar K. and Maurel D. 2015. Arabic Named Entity Recognition Process using Transducer Cascade and Arabic Wikipedia. Proceedings of Recent Advances in Natural Language Processing, pp 48–54, Hissar, Bulgaria. find
  24. Boudchiche, M., Mazroui, A., Ould Abdallahi Ould Bebah, M., Lakhouaja, A., Boudlal, A., 2016. AlKhalil Morpho Sys 2: A robust Arabic morpho-syntactic analyzer. J. King Saud Univ. - Comput. Inf. Sci. http://dx.doi.org/10.1016/j.jksuci.2016.05.002 ↗
  25. Boudchiche, M., Mazroui, A., Ould Abdallahi Ould Bebah, M., Lakhouaja, A., Boudlal, A., 2014, L’Analyseur Morphosyntaxique AlKhalil, Morpho Sys 2, DOI: 10.13140/RG.2.1.4280.0085 ↗
  26. Boudlal, A., Lakhouaja, A., Mazroui, A., Meziane, A., Ould Abdallahi Ould Bebah, M., Shoul, M., 2010. Alkhalil Morpho SYS1: a morphosyntactic analysis system for arabic texts. In: International Arab Conference on Information Technology. Benghazi, Libya, pp. 1–6. find
  27. Buckwalter Arabic Morphological Analyzer Version 1.0. (2002). LDC Catalog No.: LDC2002349. find
  28. Buckwalter, T. (2004). Buckwalter Arabic Morphological Analyser Version 2.0. Linguistic Data Consortium (LDC) Catalog Number LDC2004L02, ISBN 1-58563-324-0. find
  29. Chennoufi, A., Mazroui, A., 2016. Morphological, syntactic and diacritics rules for automatic diacritization of Arabic sentences. Journal of King Saud University - Computer and Information Sciences. find
  30. Debili, F., Achour, H. (1998). Voyellation automatique de l’arabe. Actes du Workshop on Computational Approaches To Semitic Languages, Université de Montréal. find
  31. Debili, F., Souissi, E. (1998). Etiquetage grammatical de l’arabe voyellé ou non. In Proceedings of the Workshop on Computational Approaches to Semitic Languages, Stroudsburg. find
  32. Debili, F., Achour, H., Souissi, E. (2002). La langue arabe et l’ordinateur: de l’étiquetage grammatical à la voyellation automatique. Correspondances de l’IRMC, N°71, Tunis. find
  33. Gal Y., (2002). An HMM approach to vowel restoration in Arabic and Hebrew. In ACL-02 Workshop on Computational Approaches to Semitic Languages. find our card
  34. Fairon, C., Paumier, S., Watrin, P. (2005). Can we parse without tagging? 2nd Language & Technology Conference (LTC'05), 2005, Poznan, Poland. 2nd Language & Technology Conference (LTC'05), pp.473-477, 2005 find
  35. Habash, N. and Rambow O. (2005). Arabic Tokenization, Part-of-speech Tagging and Morphological Disambiguation in One Fell Swoop. In Proceedings of the Conference of the American Association for Computational Linguistics, New York. find our card
  36. Habash, Nizar; Rambow, Owen (2006). MAGEAD: A Morphological Analyzer and Generator for the Arabic Dialects. In Proceedings of the International Conference on Computational Linguistics and Annual Meeting of the Association for Computational Linguistics (COLING-ACL), Sydney, Australia, pages 681–688. find our card
  37. Habash N. and Rambow O., (2007). Arabic Diacritization through Full Morphological Tagging, In Proceedings of the North American chapter of the Association for Computational Linguistics (NAACL), Rochester, New York. find our card
  38. Habash N. (2010) Introduction to Arabic natural language processing. Synthesis lectures on human language technologies. Morgan & Claypool, San Rafael. doi:http://dx.doi.org/10.2200/S00277ED1V01Y201008HLT010 ↗
  39. Hamdi, Ahmed (2012), Apport de la diacritisation dans l’analyse morphosyntaxique de l’arabe; 247–254. find
  40. Hamed Osama, Torsten Zesch, (2017). A Survey and Comparative Study of Arabic Diacritization Tools. In JLCL, volume 32, Number 1, http://jlcl.org/content/5-allissues/1-Heft1-2017/Heft1-2017.pdf . ↗
  41. Kirchhoff K. and Vergyri D., (2005). Cross-dialectal data sharing for acoustic modeling in Arabic speech recognition. Speech Communication, 46(1):37–51, May. find
  42. Krstev, Cvetana, Stanković Ranka, Vitas Duško. 2018. Knowledge and Rule-Based Diacritic Restoration in Serbian. In Proceedings of the Third International Conference Computational Linguistics in Bulgaria, CLIB-2018:41-51. find
  43. Maamouri Mohamed, Ann Bies, and Seth Kulick. 2006. Diacritization: A challenge to Arabic treebank annotation and parsing. In Proceedings of the British Computer Society Arabic NLP/MT Conference, London, UK, October. find
  44. Maamouri Mohamed, Seth Kulick, Ann Bies 2008. Diacritic Annotation in the Arabic Treebank and its Impact on Parser Evaluation. . In Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC 2008), Marrakech, Morocco, May 28-30, 2008. find our card
  45. Maamouri, Mohamed, et al. LDC Standard Arabic Morphological Analyzer (SAMA) Version 3.1 LDC2010L01. Web Download. Philadelphia: Linguistic Data Consortium, 2010. find
  46. Maamouri, M., Bies, A. & Buckwalter, T. (2004). The Penn Arabic treebank: Building a largescale annotated Arabic corpus. In NEMLAR Conference on Arabic Language Resources and Tools, Cairo, Egypt. find
  47. Maamouri, Mohamed, Ann Bies, Tim Buckwalter, and Wigdan Mekki. The Penn Arabic Treebank: Building a Large-Scale Annotated Arabic Corpus, 2004. find
  48. Mubarak, Hamdy, Kareem Darwish. 2014. "Automatic Correction of Arabic Text: a Cascaded Approach". Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (NLP). find
  49. Neme, Alexis, Laporte Éric (2013). Pattern-and-root inflectional morphology: the Arabic broken plural. Language Sciences. http://dx.doi.org/10.1016/j.langsci.2013.06.002 ↗ our card
  50. Neme, Alexis (2011). A lexicon of Arabic verbs constructed on the basis of Semitic taxonomy and using finite-state transducers. In Proceedings of the International Workshop on Lexical Resources (WoLeR) at ESSLLI. find
  51. Neme, Alexis Amid (2014). Why Microsoft Arabic Spell checker is ineffective, Linguistica Communicatio, http://www.al-erfan.com/, 2014, Arabic Language in Information Technology, 16, pp.55. <http://www.al-erfan.com/> ↗
  52. Paumier, Sébastien. (2016). Unitex – User manuel 3.1RC, University of Marne-la-Vallée. find
  53. Pasha Arfath, Mohamed Al-Badrashiny, Mona Diab, Ahmed El Kholy, Ramy Eskander, Nizar Habash, Manoj Pooleery, Owen Rambow, Ryan Roth (2014). MADAMIRA: A Fast, Comprehensive Tool for Morphological Analysis and Disambiguation of Arabic, Proceedings of the Ninth International Conference on Language Resources and Evaluation LREC, May 2014. find our card
  54. Revuz, Dominique (1992): Minimization of acyclic deterministic automata in linear time. Theoretical Computer Science 92:1, Elsevier 181-189 find
  55. Sak, H., Güngör, T. & Saraçlar M. (2011). Resources for Turkish morphological processing. find
  56. Lang Resources & Evaluation (2011) 45: 249. https://doi.org/10.1007/s10579-010-9128-6 ↗
  57. Shaalan, K., Allam A., Gomah A. (2003). Towards automatic spell checking for Arabic. Conference on Language Engineering. find
  58. Shaalan, Khaled, Samih, Younes, Attia, Mohammed, Pecina, Pavel, & van Genabith, Josef (2012). Shaalan, Khaled,Younes Samih, Mohammed Attia, Pavel Pecina, and Josef van Genabith. (2012). Arabic Word Generation and Modelling for Spell Checking. Language Resources and Evaluation (LREC). Istanbul, Turkey. Pages: 719-725 find
  59. Smrz, Otakar. (2007). ElixirFM — Implementation of Functional Arabic Morphology. In Computational Approaches to Semitic Languages, ACL 2007, Prague. find our card
  60. Traboulsi, Hayssam. 2009. Arabic named entity extraction: A local grammar-based approach. In Proceedings of the International Multi-conference on Computer Science and Information Technology (IMCSIT 2009), pages 139–143, Mragowo. find
  61. Wintner S. (2008). Strengths and weaknesses of finite-state technology: a case study morphological grammar development. Nat Lang Eng 14(4):457–469. doi:http://dx.doi.org/10.1017/S1351324907004676 ↗
  62. Nasser Zalmout and Nizar Habash (2017). Don't Throw Those Morphological Analyzers Away Just Yet: Neural Morphological Disambiguation for Arabic. Conference on Empirical Methods in Natural Language Processing, Proceedings pages 715–724 Copenhagen, September, 2017 find our card
  63. Zitouni I., Sorensen J. S., and Sarikaya R., (2006), “Maximum entropy based restoration of Arabic diacritics”. In Proceedings of ACL’06 find our card

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/langstream-2019-neme-restoring-arabic-vowels-through-omission/