MenakBERT — Hebrew Diacriticizer

What it is about

The task is to add niqqud to plain Hebrew. Following Nakdimon, the marks are split into three decisions per letter: the shin/sin dot, the dagesh, and the vowel. The authors fine-tune TavBERT, a character-level BERT pretrained on 3.1 billion Hebrew words, with three output heads, on Nakdimon's data (2.6 million premodern and 0.3 million modern characters), trained in chronological order so the model ends on modern text. On Nakdimon's test set it scores 98.82% decision accuracy and 94.12% word accuracy, close to Dicta's rule-assisted Nakdan (98.94 / 95.83) and above Nakdimon. Errors include verb/adjective homographs, the definite article hidden in the vowel of a prefixed preposition (וּבָרְכוּשׁ vs וּבִרְכוּשׁ), and foreign names; a model fine-tuned for dotting also transfers to POS tagging.

Ideas

I1. The three-way split of the pointing (shin/sin dot, dagesh, vowel) is a clean way to describe what the marks do: use it in the script chapter, and in any Mova check of pointed text count errors per category.
I2. The hidden article: after בְּ/כְּ/לְ the article shows only in the prefix's vowel and the following dagesh. Check on the Torah trees that every such case gets Definite=Def, and state the rule (pataḥ/qameṣ + dagesh after the prefix = article) in the noun chapter.
I3. The four scores DEC/CHA/WOR/VOC (decision, character, word, and "pronounced right") are the standard for a vocaliser; reuse them if Mova ever vocalises Mishnaic or unpointed text.

For Mova

Mova's Torah is already pointed, so the model itself is not needed. I1 goes to 1. Script and pointing; I2 is a check in tools/macula (FEATS) and a point in 04-noun.md (the article); I3 is the scoring for any future vocaliser, described in 13-mova-tools.md.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. arXiv 2410.02417

@misc{cohen2024menakbert,
  title = {{MenakBERT — Hebrew Diacriticizer}},
  author = {Cohen, Ido and Gidron, Jacob and Pinto, Idan},
  year = {2024},
  eprint = {2410.02417},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2410.02417}
}

References

8 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

Ya’akov Gal. 2002. An HMM approach to vowel restoration in Arabic and Hebrew. In Proceedings of the ACL-02 Workshop on Computational Approaches to Semitic Languages, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics. find our card
Leonid Kontorovich. 2001. Problems in semitic nlp: Hebrew vocalization using hmms. find
Avi Shmidman, Shaltiel Shmidman, Moshe Koppel, and Yoav Goldberg. 2020. Nakdan: Professional Hebrew diacritizer. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 197– 203, Online. Association for Computational Linguistics. find our card
Elazar Gershuni, Yuval Pinter. 2021. Restoring Hebrew Diacritics Without a Dictionary (Nakdimon). find our card
Omri Keren, Tal Avinari, Reut Tsarfaty, Omer Levy. 2022. Breaking Character: Are Subwords Good Enough for MRLs After All? find
Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova. 2018. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S. find
Weld, Luke Zettlemoyer, Omer Levy. 2020. SpanBERT: Improving Pre-training by Representing and Predicting Spans. find
Shoval Sade, Amit Seker, and Reut Tsarfaty. 2018. The Hebrew Universal Dependency treebank: Past present and future Amit Seker and Reut Tsarfaty. 2020. A pointer network architecture for joint morphological segmentation and tagging find our card

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/hbo-2024-cohen-menakbert-hebrew-diacriticizer/