Maximum Entropy Based Restoration of Arabic Diacritics

What it is about

Diacritics (short vowels, case endings, shadda, sukun) are missing from written Arabic, and restoring them is needed for speech and text-to-speech. The authors treat restoration as sequence labelling: each character gets a diacritic label, chosen by a maximum-entropy model with a Viterbi-style search over the last few labels. The features are character n-grams around the character (window of seven), word trigrams, segment-based features from a 98%-accurate segmenter (prefixes, stem, suffixes, and whether the character starts or ends a segment), and POS tags from a 96%-accurate tagger; the two previous labels are also features. The corpus is the diacritized Penn Arabic Treebank part 3, split by date (288,000 training words, 52,000 devtest words). The best system reaches a diacritic error rate (DER) of 5.1%, a segment error rate of 8.5% and a word error rate (WER) of 17.3%; without case endings the figures are 2.2%, 4.0% and 7.2%. Lexical features alone give a WER of 25.1%, and adding segment features cuts it by about a quarter.

Ideas

• I1. Restoration as labelling with segment features: for unpointed Hebrew, the vowel and dagesh labels of each consonant can be predicted from character n-grams around it and from the prefix, stem and suffix boundaries that the segmenter in tools/hbo already produces. The model is a table of counts with a deterministic search, so it stays inside the Rust mechanics; compare it first with the pure lexical version.
• I2. A date-based split: the paper keeps the devtest after the training period, so no document leaks into both sets. For Hebrew, hold out whole books or whole periods (for example one biblical book, or a late book with Mishnaic traits) and keep that split fixed across runs.
• I3. A separate first step for gemination: the paper restores shadda first and the other diacritics second, and reports both settings. For Hebrew, a dagesh (forte or lene) classifier as the first step, with a measured effect on the vowel step, fits the chapter on script and phonology.

For Mova

I1 is a model for the unpointed mode of tools/hbo, and I3 is part of Phonology of Biblical Hebrew and 1. Script and pointing. I2 is an evaluation rule for tools/hbo and for the held-out discipline of the release; no test set is opened for tuning.

Topics: Part-of-speech tagging

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.3115/1220175.1220248

@inproceedings{zitouni2006maximum,
  title = {{Maximum Entropy Based Restoration of Arabic Diacritics}},
  author = {Zitouni, Imed and Sorensen, Jeffrey S. and Sarikaya, Ruhi},
  booktitle = {Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics},
  year = {2006},
  doi = {10.3115/1220175.1220248},
  url = {https://doi.org/10.3115/1220175.1220248}
}

References

22 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. M. Afify, S. Abdou, J. Makhoul, L. Nguyen, and B. Xiang. 2004. The BBN RT04 BN Arabic System. In RT04 Workshop, Palisades NY. find
2. A. Berger, S. Della Pietra, and V. Della Pietra. 1996. A maximum entropy approach to natural language processing. Computational Linguistics, 22(1):39–71. find
3. T. Buckwalter. 2002. Buckwalter Arabic morphological analyzer version 1.0. Technical report, Linguistic Data Consortium, LDC2002L49 and ISBN 1-58563-257-0. find
4. Stanley F. Chen and Joshua Goodman. 1999. An empirical study of smoothing techniques for language modeling. computer speech and language. Computer Speech and Language, 4(13):359–393. find
5. Stanley Chen and Ronald Rosenfeld. 2000. A survey of smoothing techniques for me models. IEEE Trans. on Speech and Audio Processing. find
6. F. Debili, H. Achour, and E. Souissi. 2002. De l’etiquetage grammatical a‘ la voyellation automatique de l’arabe. Technical report, Correspondances de l’Institut de Recherche sur le Maghreb Contemporain 17. find
7. Y. El-Imam. 2003. Phonetization of arabic: rules and algorithms. Computer Speech and Language, 18:339– 373. find
8. T. El-Sadany and M. Hashish. 1988. Semi-automatic vowelization of Arabic verbs. In 10th NC Conference, Jeddah, Saudi Arabia. find
9. O. Emam and V. Fisher. 2004. A hierarchical approach for the statistical vowelization of Arabic text. Technical report, IBM patent filed, DE9-2004-0006, US patent application US2005/0192809 A1. find
10. R. Florian, H. Hassan, A. Ittycheriah, H. Jing, N. Kambhatla, X. Luo, N Nicolov, and S Roukos. 2004. A statistical model for multilingual entity detection and tracking. In Proceedings of HLT-NAACL 2004, pages 1–8. find
11. Y. Gal. 2002. An HMM approach to vowel restoration in Arabic and Hebrew. In ACL-02 Workshop on Computational Approaches to Semitic Languages. find our card
12. Joshua Goodman. 2002. Sequential conditional generalized iterative scaling. In Proceedings of ACL’02. find
13. K. Kirchhoff and D. Vergyri. 2005. Cross-dialectal data sharing for acoustic modeling in Arabic speech recognition. Speech Communication, 46(1):37–51, May. find
14. John Lafferty, Andrew McCallum, and Fernando Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In ICML. find
15. Y.-S. Lee, K. Papineni, S. Roukos, O. Emam, and H. Hassan. 2003. Language model based Arabic word segmentation. In Proceedings of the ACL’03, pages 399–406. find
16. Andrew McCallum, Dayne Freitag, and Fernando Pereira. 2000. Maximum entropy markov models for information extraction and segmentation. In ICML. find
17. Rani Nelken and Stuart M. Shieber. 2005. Arabic diacritization using weighted finite-state transducers. In ACL-05 Workshop on Computational Approaches to Semitic Languages, pages 79–86, Ann Arbor, Michigan. find our card
18. Adwait Ratnaparkhi. 1996. A maximum entropy model for part-of-speech tagging. In Conference on Empirical Methods in Natural Language Processing. find
19. M. Tayli and A. Al-Salamah. 1990. Building bilingual microcomputer systems. Communications of the ACM, 33(5):495–505. find
20. D. Vergyri and K. Kirchhoff. 2004. Automatic diacritization of Arabic for acoustic modeling in speech recognition. In COLING Workshop on Arabic-script Based Languages, Geneva, Switzerland. find
21. Tong Zhang, Fred Damerau, and David E. Johnson. 2002. Text chunking based on a generalization of Winnow. Journal of Machine Learning Research, 2:615– 637. find
22. Imed Zitouni, Jeff Sorensen, Xiaoqiang Luo, and Radu Florian. 2005. The impact of morphological stemming on Arabic mention detection and coreference resolution. In Proceedings of the ACL Workshop on Computational Approaches to Semitic Languages, pages 63– 70, Ann Arbor, June. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2006-zitouni-maximum-entropy-based-restoration-arabic/