Mova0.5 pre-release⌕
☰
home / docs / papers / Ancient Hebrew / Restoring Hebrew Diacritics Without a Dictionary

Restoring Hebrew Diacritics Without a Dictionary

What it is about

Most Modern Hebrew is written unpointed, and dotting (adding niqqud) needs expertise or a paid service; the state of the art, Dicta's Nakdan, uses morphological analysis, dictionaries and several networks. The authors ask how far one simple model can go with plain pointed text. NAKDIMON is a two-layer character BiLSTM with three linear heads: shin/sin dot, dagesh/mappiq, and the other vowel marks, decoded greedily without any lexicon. Training uses pre-modern pointed texts (about 1.8M tokens), an automatically dotted set of stories, and about 327K tokens of modern text dotted with Dicta and hand-corrected. A new modern test set of 20K tokens from eleven sources is released. NAKDIMON is close to Morfix on character accuracy but below it on word accuracy; its errors are often unreadable vowel sequences that a reader notices, and it does well on foreign names.

Ideas

For Mova

I1 and I2 apply to the unpointed mode of tools/hbo, to any restoration step, and to 1. Script and pointing. I3 is a test-set design for tools/hbo and, for Modern Hebrew, for 12. Mova and Modern Hebrew: where it stands, what it would need. The character BiLSTM is not a Mova component; the Mova version of the three-head design is rules with a measured error table.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/2022.findings-naacl.75

@inproceedings{gershuni2022restoring,
  title = {{Restoring Hebrew Diacritics Without a Dictionary}},
  author = {Gershuni, Elazar and Pinter, Yuval},
  booktitle = {Findings of the Association for Computational Linguistics: NAACL 2022},
  year = {2022},
  doi = {10.18653/v1/2022.findings-naacl.75},
  url = {https://doi.org/10.18653/v1/2022.findings-naacl.75}
}

References

25 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Gheith Abandah, Alex Graves, Balkees Al-Shagoor, Alaa Arabiyat, Fuad Jamour, and Majid Al-Taee. 2015. Automatic diacritization of Arabic text using recurrent neural networks. International Journal on Document Analysis and Recognition (IJDAR), 18:183–197. ↗
  2. Salim Abu-Rabia. 2001. The role of vowels in reading semitic scripts: Data from Arabic and Hebrew. Reading and Writing, 14(1-2):39–59. ↗
  3. Yonatan Belinkov and James Glass. 2015. Arabic diacritization with recurrent neural networks. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2281– 2285, Lisbon, Portugal. Association for Computational Linguistics. find our card
  4. Shlomo Bentin and Ram Frost. 1987. Processing lexical ambiguity and visual word recognition in a deep orthography. Memory & Cognition, 15(1):13–23. find
  5. Su Lin Blodgett, Solon Barocas, Hal Daumé III, and Hanna Wallach. 2020. Language (technology) is power: A critical survey of “bias” in NLP. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5454– 5476, Online. Association for Computational Linguistics. ↗
  6. Ya’akov Gal. 2002. An HMM approach to vowel restoration in Arabic and Hebrew. In Proceedings of the ACL-02 Workshop on Computational Approaches to Semitic Languages, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics. find our card
  7. Yoav Goldberg and Michael Elhadad. 2010. Easy-first dependency parsing of modern Hebrew. In Proceedings of the NAACL HLT 2010 First Workshop on Statistical Parsing of Morphologically-Rich Languages, pages 103–107, Los Angeles, CA, USA. Association for Computational Linguistics. find our card
  8. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735– 1780. ↗
  9. Dror Kamir, Naama Soreq, and Yoni Neeman. 2002. A comprehensive NLP system for modern standard Arabic and modern Hebrew. In Proceedings of the ACL-02 Workshop on Computational Approaches to Semitic Languages, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics. find our card
  10. Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. ↗
  11. Leonid Kontorovich. 2001. Problems in Semitic NLP: Hebrew vocalization using HMMs. In Problems in Semitic NLP, NIPS Workshop on Machine Learning Methods for Text and Images. find
  12. Amit Moryossef, Roee Aharoni, and Yoav Goldberg. 2019. Filling gender & number gaps in neural machine translation with black-box context injection. In Proceedings of the First Workshop on Gender Bias in Natural Language Processing, pages 49–54, Florence, Italy. Association for Computational Linguistics. find
  13. Hamdy Mubarak, Ahmed Abdelali, Hassan Sajjad, Younes Samih, and Kareem Darwish. 2019. Highly effective Arabic diacritization using sequence to sequence modeling. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 2390–2395, Minneapolis, Minnesota. Association for Computational Linguistics. find
  14. Jakub Náplava, Milan Straka, and Jana Straková. 2021. Diacritics restoration using bert with analysis on czech language. arXiv preprint arXiv:2105.11408. ↗
  15. Jakub Náplava, Milan Straka, Pavel Straňák, and Jan Hajič. 2018. Diacritics restoration using neural networks. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Association (ELRA). find
  16. Barbara Plank, Anders Søgaard, and Yoav Goldberg. 2016. Multilingual part-of-speech tagging with bidirectional long short-term memory models and auxiliary loss. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 412–418, Berlin, Germany. Association for Computational Linguistics. Copenhagen, Denmark. Association for Computational Linguistics. find
  17. Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2017. Men also like shopping: Reducing gender bias amplification using corpus-level constraints. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2979–2989, Copenhagen, Denmark. Association for Computational Linguistics. find
  18. Danny Shacham and Shuly Wintner. 2007. Morphological disambiguation of Hebrew: A case study in classifier combination. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), pages 439– 447, Prague, Czech Republic. Association for Computational Linguistics. find our card
  19. Avi Shmidman, Shaltiel Shmidman, Moshe Koppel, and Yoav Goldberg. 2020. Nakdan: Professional Hebrew diacritizer. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 197–203, Online. Association for Computational Linguistics. find our card
  20. Leslie N Smith. 2017. Cyclical learning rates for training neural networks. In 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), pages 464–472. IEEE. find
  21. Lukas Stankevičius, Mantas Lukoševičius, Jurgita Kapočiūtė-Dzikienė, Monika Briedienė, and Tomas Krilavičius. 2022. Correcting diacritics and typos with ByT5 transformer model. arXiv e-prints. find
  22. Eran Tomer. 2012. Automatic Hebrew Text Vocalization. Ben-Gurion University of the Negev, Faculty of Natural Sciences, Department of Computer Science. find
  23. Reut Tsarfaty, Shoval Sadde, Stav Klein, and Amit Seker. 2019. What’s wrong with Hebrew NLP? and how to make it right. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP): System Demonstrations, pages 259–264, Hong Kong, China. Association for Computational Linguistics. find our card
  24. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems, pages 5998–6008. find
  25. Nasser Zalmout and Nizar Habash. 2017. Don’t throw those morphological analyzers away just yet: Neural morphological disambiguation for Arabic. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 704–713, Dicta – reported / reproduced New test set (§2.3) OOV System DEC CHA WOR VOC DEC CHA WOR VOC WOR VOC Baselines M AJ M OD M AJ A LL 84.93 91.67 75.94 86.29 68.10 79.43 69.63 81.19 88.04 93.79 81.22 90.01 76.14 84.87 77.10 86.19 N/A N/A N/A N/A Lexicalized S NOPI M ORFIX D ICTA 87.81 94.91 97.53 78.96 / 79.92 90.32 / 91.29 95.12 / 95.71 66.41 / 66.57 80.90 / 82.24 88.23 / 89.23 70.35 86.48 90.66 91.29 96.84 97.95 85.84 94.92 96.77 76.45 90.38 94.11 78.91 92.39 94.92 40.83 63.91 76.21 42.39 69.20 77.66 Unlexicalized N AKDIMON 0 95.78 N AKDIMON 92.59 79.00 83.01 94.59 97.91 91.70 96.37 84.94 89.75 87.54 91.64 47.05 57.46 50.96 62.06 find our card

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/findings-2022-gershuni-restoring-hebrew-diacritics-without-dictionary/