ReNikud: Audio-Supervised Hebrew Grapheme-to-Phoneme Conversion

What it is about

Modern Hebrew is written without most vowels, so pronunciation needs either diacritics or a direct grapheme-to-phoneme model. The authors take pronunciation from unlabelled audio. Two ASR systems, one for Hebrew text and one for IPA, run on about 1,700 hours of parliamentary speech. A finite-state aligner then maps each written letter to a (consonant, vowel, stress) triplet. Digraphs with a geresh, silent matres lectionis and the patah-gnuva vowel each get a rule. After filtering, 1.52M aligned sentences remain. The model is a character encoder with three heads, one per phonetic slot, under constrained decoding: each letter may take only its legal sounds, and each word exactly one stress. On the new MILIM benchmark (1,653 sentences, 3,110 target words, eleven categories) the model reaches word error 24.3% and character error 6.6%, against 35.6% and 10.6% for Phonikud. Native speakers prefer its informal readings (mean +0.87 on a seven-point scale). For diacritization with 1k sentences, the word error is 23.6%, against 37.4% for DictaBERT.

Ideas

I1. Constrained decoding as a rule table: for each letter, list its legal sounds (ב gives b or v, ט and ת both give t) and enforce exactly one stress per word. The paper's ablation is the measure: unconstrained decoding gives 24.9% word error, constrained 24.3%.
I2. Write the alignment rules for Hebrew spelling: geresh digraphs (the base letter takes the foreign sound and the geresh is skipped), and silent letters ו, י, ה, א that map to a null consonant. Put these as a small table in the grammar.
I3. Build a hand-written MILIM-style set for Hebrew pronunciation with its own categories (homographs, minimal stress pairs, acronyms, penultimate stress, colloquial). The paper notes that its sentences were produced by an LLM, which may inflate scores. Ours should be written and checked by hand.

For Mova

I1 and I2 go into 3. Phonology and 2. Script and spelling. For ancient spelling, the same silent-letter table is a check for 1. Script and pointing. I3 is a test for 3. Phonology, written by hand and kept held out. The paper's benchmark is not carried into the build.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. arXiv 2606.20179

@misc{melichov2026renikud,
  title = {{ReNikud: Audio-Supervised Hebrew Grapheme-to-Phoneme Conversion}},
  author = {Melichov, Maxim and Kolani, Yakov and Alper, Morris},
  year = {2026},
  eprint = {2606.20179},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2606.20179}
}

References

20 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

P. T. Daniels and W. Bright, The world’s writing systems. Oxford University Press, 1996. find
V. Pratap, A. Tjandra, B. Shi, P. Tomasello, A. Babu, S. Kundu, A. Elkahky, Z. Ni, A. Vyas, M. Fazel-Zarandi et al., “Scaling speech technology to 1,000+ languages,” Journal of Machine Learning Research, vol. 25, no. 97, pp. 1–52, 2024. find
Y. Kolani, M. Melichov, C. Calev, and M. Alper, “Phonikud: Hebrew grapheme-to-phoneme conversion for real-time text-to-speech,” arXiv preprint arXiv:2506.12311, 2025. ↗ our card
A. Aharoni, “Vocalization of modern hebrew,” in Encyclopedia of Hebrew Language and Linguistics, G. Khan, S. Bolozky, S. E. Fassberg, G. A. Rendsburg, A. D. Rubin, O. Schwarzwald, and T. Zewi, Eds. Leiden: Brill, 2013, vol. 3, pp. 944–951. find
H. Neudecker, “Vocalization of modern hebrew and colloquial pronunciation,” in Encyclopedia of Hebrew Language and Linguistics, G. Khan, S. Bolozky, S. E. Fassberg, G. A. Rendsburg, A. D. Rubin, O. Schwarzwald, and T. Zewi, Eds. Leiden: Brill, 2013, vol. 3, pp. 951–953. find
E. Gershuni and Y. Pinter, “Restoring hebrew diacritics without a dictionary,” in Findings of the Association for Computational Linguistics: NAACL 2022, 2022, pp. 1010–1018. find our card
S. Shmidman, A. Shmidman, and M. Koppel, “Dictabert: A state-ofthe-art bert suite for modern hebrew,” 2023. find
J. Zhu, C. Zhang, and D. Jurgens, “Byt5 model for massively multilingual grapheme-to-phoneme conversion,” in Proc. Interspeech 2022, 2022, pp. 446–450. find
ivrit.ai, “ivrit-ai/whisper-large-v3-turbo,” https://huggingface.co/ivrit-ai/whisper-large-v3-turbo, 2025. ↗
O. Sharoni, R. Shenberg, and E. Cooper, “Saspeech: A hebrew single speaker dataset for text to speech and voice conversion,” in Proc. Interspeech, 2023. find
ivrit.ai, “ivrit-ai/crowd-recital,” https://huggingface.co/datasets/ivrit-ai/crowd-recital, 2025. ↗
Y. Marmor, A. Zulti, D. Krongauz, A. Gabet, Y. Snapir, Y. Lifshitz, and E. Segal, “Voxknesset: A large-scale longitudinal hebrew speech dataset for aging speaker modeling,” 2026. [Online]. Available: https://arxiv.org/abs/2603.01270 ↗ our card
S. Sun, K. Richmond, and H. Tang, “Improving seq2seq tts frontends with transcribed speech audio,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 31, pp. 1940–1952, 2023. find
S. Sun and K. Richmond, “Acquiring pronunciation knowledge from transcribed speech audio via multi-task learning,” 2024. [Online]. Available: https://arxiv.org/abs/2409.09891 ↗
M. S. Ribeiro, G. Comini, and J. Lorenzo-Trueba, “Improving grapheme-to-phoneme conversion by learning pronunciations from speech recordings,” arXiv preprint arXiv:2307.16643, 2023. ↗
J. Route, S. Hillis, I. C. Etinger, H. Zhang, and A. W. Black, “Multimodal, multilingual grapheme-to-phoneme conversion for low-resource languages,” in Proceedings of the 2nd Workshop on Deep Learning Approaches for Low-Resource NLP (DeepLo 2019), 2019, pp. 192–201. find
H. Gao, M. Hasegawa-Johnson, and C. D. Yoo, “G2pu: grapheme-to-phoneme transducer with speech units,” in ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2024, pp. 10 061–10 065. find
C.-J. Li, K. Chang, S. Bharadwaj, E. Yeo, K. Choi, J. Zhu, D. Mortensen, and S. Watanabe, “Powsm: A phonetic open whisper-style speech foundation model,” arXiv preprint arXiv:2510.24992, 2025. ↗
S. Shatnawi, S. Alqahtani, and H. Aldarmaki, “Automatic restoration of diacritics for speech data sets,” 2024. [Online]. Available: https://arxiv.org/abs/2311.10771 ↗
A. Ghannam, N. Alharthi, F. Alasmary, K. Al Tabash, S. Sadah, and L. Ghouti, “Abjad ai at nadi 2025: Catt-whisper: Multimodal diacritic restoration using text and speech representations,” pp. 757–761, 2025. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/langstream-2026-melichov-renikud-audio-supervised-hebrew-grapheme/