Arabic Diacritization Using Morphologically Informed Character-Level Model

What it is about

The paper restores Arabic short vowels with a character-level recurrent model, and adds the morpheme boundaries that a separate segmenter predicts. The segmenter is a character BiLSTM with a CRF layer and needs no list of prefixes, suffixes or stems. It is trained on Farasa's output for Arabic Wikipedia, so Farasa is its teacher. The diacritizer predicts core-word diacritics and case endings in one pass, with 15 labels. Training data are a proprietary MSA corpus of 7.2 million tokens, a subset of Tashkeela for Classical Arabic, and fully vowelized Moroccan and Tunisian Bible translations. The reported word-level error rates are 3.4% for MSA, 5.4% for Classical Arabic, and competitive numbers for the dialects. The segmenter reaches 98.4% on MSA and 93.1% on dialect. A hand check of 50 random errors per task shows that segmentation mistakes are the largest class.

Ideas

• I1. Boundaries as input, not as guesses: give the vowel restorer the segment boundaries as extra symbols, so each morpheme is vowelized in its own context. Test on pointed MACULA forms with and without the boundaries.
• I2. Error taxonomy: adopt the paper's classes (segmentation mistake, partial segmentation, named entity or foreign word, both readings correct, error in the reference) as the label set of every error report for tools/hbo and tools/langseg.
• I3. Table against learned segmentation: build a held-out Hebrew set where the pre and lex tables have no entry for a form, and count how often the table route fails. This says whether the table segmenter needs a learned fallback, and how big it must be.

For Mova

I1 goes into tools/hbo (unpointed mode) and the vowel logic of Phonology of Biblical Hebrew. I2 is the report format for tools/hbo and tools/langseg. I3 is a test for the segmenter tables in tools/hbo, measured on MACULA.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.63317/5pgzxaektewi

@inproceedings{elmallah2024arabic,
  title = {{Arabic Diacritization Using Morphologically Informed Character-Level Model}},
  author = {Elmallah, Muhammad Morsy and Reda, Mahmoud and Darwish, Kareem and El-Sheikh, Abdelrahman and Elneima, Ashraf Hatim and Aljubran, Murtadha and Alsaeed, Nouf and Mohammed, Reem and Al-Badrashiny, Mohamed},
  booktitle = {Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)},
  year = {2024},
  doi = {10.63317/5pgzxaektewi},
  url = {https://doi.org/10.63317/5pgzxaektewi}
}

References

33 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Gheith A Abandah, Alex Graves, Balkees Al-Shagoor, Alaa Arabiyat, Fuad Jamour, and Majid Al-Taee. 2015. Automatic diacritization of arabic text using recurrent neural networks. International Journal on Document Analysis and Recognition (IJDAR), 18:183–197. find
2. Ahmed Abdelali, Kareem Darwish, Nadir Durrani, and Hamdy Mubarak. 2016. Farasa: A fast and furious segmenter for arabic. In Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: Demonstrations, pages 11–16. find our card
3. Ahmed Abdelali, Hamdy Mubarak, Younes Samih, Sabit Hassan, and Kareem Darwish. 2020. Arabic dialect identification in the wild. arXiv preprint arXiv:2005.06557. ↗
4. Mahmoud Al-Ayyoub, Aya Nuseir, Kholoud Alsmearat, Yaser Jararweh, and Brij Gupta. 2018. Deep learning for arabic nlp: A survey. Journal of computational science, 26:522–531. find
5. Nouf Alabbasi, Mohamed Al-Badrashiny, Maryam Aldahmani, Ahmed AlDhanhani, Abdullah Saleh Alhashmi, Fawaghy Ahmed Alhashmi, Khalid Al Hashemi, Rama Emad Alkhobbi, Shamma T Al Maazmi, Mohammed Ali Alyafeai, et al. 2022. Gulf arabic diacritization: Guidelines, initial dataset, and results. In Proceedings of the The Seventh Arabic Natural Language Processing Workshop (WANLP), pages 356–360. find
6. Badr AlKhamissi, Muhammad ElNokrashy, and Mohamed Gabr. 2020. Deep diacritization: Efficient hierarchical recurrence for improved Arabic diacritization. In Proceedings of the Fifth Arabic Natural Language Processing Workshop, pages 38–48, Barcelona, Spain (Online). Association for Computational Linguistics. find
7. Manar M Almanea. 2021. Automatic methods and neural networks in arabic texts diacritization: a comprehensive survey. IEEE Access, 9:145012–145032. find
8. Aqil M Azmi and Reham S Almajed. 2015. A survey of automatic arabic diacritization techniques. Natural Language Engineering, 21(3):477–495. find
9. Yonatan Belinkov and James Glass. 2015. Arabic diacritization with recurrent neural networks. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2281–2285. find our card
10. Kareem Darwish, Ahmed Abdelali, Hamdy Mubarak, and Mohamed Eldesouki. 2021. Arabic diacritic recovery using a feature-rich bilstm model. Transactions on Asian and Low-Resource Language Information Processing, 20(2):1–18. find
11. Kareem Darwish, Ahmed Abdelali, Hamdy Mubarak, Younes Samih, and Mohammed Attia. 2018a. Diacritization of moroccan and tunisian arabic dialects: A crf approach. OSACT, 3:62. find
12. Kareem Darwish and Hamdy Mubarak. 2016. Farasa: A new fast and accurate arabic word segmenter. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 1070–1074. find
13. Kareem Darwish, Hamdy Mubarak, and Ahmed Abdelali. 2017. Arabic diacritization: Stats, rules, and hacks. In Proceedings of the third Arabic natural language processing workshop, pages 9–17. find our card
14. Kareem Darwish, Hamdy Mubarak, Ahmed Abdelali, Mohamed Eldesouki, Younes Samih, Randah Alharbi, Mohammed Attia, Walid Magdy, and Laura Kallmeyer. 2018b. Multi-dialect arabic pos tagging: A crf approach. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Paris, France. European Language Resources Association (ELRA). find
15. Yousif A El-Imam. 2004. Phonetization of arabic: rules and algorithms. Computer Speech & Language, 18(4):339–373. find
16. T El-Sadany and M Hashish. 1988. Semi-automatic vowelization of arabic verbs. In 10th NC Conference, Jeddah, Saudi Arabia. find
17. Ali Fadel, Ibraheem Tuffaha, Mahmoud Al-Ayyoub, et al. 2019. Arabic text diacritization using deep neural networks. In 2019 2nd international conference on computer applications & information security (ICCAIS), pages 1–7. IEEE. find
18. Ya’akov Gal. 2002. An hmm approach to vowel restoration in arabic and hebrew. In Proceedings of the ACL-02 workshop on Computational approaches to semitic languages. find our card
19. Nizar Habash, Ramy Eskander, and Abdelati Hawwari. 2012. A morphological analyzer for egyptian arabic. In Proceedings of the twelfth meeting of the special interest group on computational morphology and phonology, pages 1–9. find our card
20. Nizar Habash and Owen Rambow. 2007. Arabic diacritization through full morphological tagging. In Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Companion Volume, Short Papers, pages 53–56. find our card
21. Osaama Hamed and Torsten Zesch. 2017. A survey and comparative study of arabic diacritization tools. Journal for Language Technology and Computational Linguistics, 32(1):27–47. find
22. Mohamed Maamouri, Ann Bies, Tim Buckwalter, and Wigdan Mekki. 2004. The penn arabic treebank: Building a large-scale annotated arabic corpus. In NEM-LAR conference on Arabic language resources and tools, volume 27, pages 466–467. Cairo. find
23. Hamdy Mubarak, Ahmed Abdelali, Kareem Darwish, Mohamed Eldesouki, Younes Samih, and Hassan Sajjad. 2019a. A system for diacritizing four varieties of arabic. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP): System Demonstrations, pages 217–222. find
24. Hamdy Mubarak, Ahmed Abdelali, Hassan Sajjad, Younes Samih, and Kareem Darwish. 2019b. Highly effective arabic diacritization using sequence to sequence modeling. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 2390–2395. find
25. Rani Nelken and Stuart Shieber. 2005. Arabic diacritization using weighted finite-state transducers. In Proceedings of the 2005 ACL Workshop on Computational Approaches to Semitic Languages. Association for Computational Linguistics. find our card
26. Hatem M Noaman, Shahenda S Sarhan, and MAA Rashwan. 2018. A hybrid approach for automatic morphological diacritizationof arabic text. find
27. Ossama Obeid, Nasser Zalmout, Salam Khalifa, Dima Taji, Mai Oudah, Bashar Alhafni, Go Inoue, Fadhl Eryani, Alexander Erdmann, and Nizar Habash. 2020. Camel tools: An open source python toolkit for arabic natural language processing. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 7022–7032. find
28. Arfath Pasha, Mohamed Al-Badrashiny, Mona T Diab, Ahmed El Kholy, Ramy Eskander, Nizar Habash, Manoj Pooleery, Owen Rambow, and Ryan Roth. 2014. Madamira: A fast, comprehensive tool for morphological analysis and disambiguation of arabic. In Lrec, volume 14, pages 1094–1101. find our card
29. Mohsen AA Rashwan, Ahmad A Al Sallab, Hazem M Raafat, and Ahmed Rafea. 2015. Deep learning framework with confused sub-set resolution architecture for automatic arabic diacritization. IEEE/ACM Transactions on Audio, Speech, And Language Processing, 23(3):505–516. find
30. Ryan Roth, Owen Rambow, Nizar Habash, Mona Diab, and Cynthia Rudin. 2008. Arabic morphological tagging, diacritization, and lemmatization using lexeme models and feature ranking. In Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics on Human Language Technologies: Short Papers, pages 117–120. find our card
31. Jie Yang and Yue Zhang. 2018. Ncrf++: An open-source neural sequence labeling toolkit. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. find
32. Taha Zerrouki and Amar Balla. 2017. Tashkeela: Novel corpus of arabic vocalized texts, data for auto-diacritization systems. Data in brief, 11:147–151. find
33. Imed Zitouni, Jeffrey Sorensen, and Ruhi Sarikaya. 2006. Maximum entropy based restoration of arabic diacritics. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, pages 577–584. find our card

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/coling-2024-elmallah-arabic-diacritization-morphologically-informed-character/