ivrit.ai: A Comprehensive Dataset of Hebrew Speech for AI Research and Development

What it is about

The paper introduces ivrit.ai, a Hebrew speech dataset of about 3,300 hours from more than a thousand speakers. The audio comes from podcasts, lectures, conversations and lessons, and each contributor signed permission for use under the project's licence. It is released in three forms: raw audio; audio after voice activity detection (VAD), which cuts it into short segments of a few seconds to a minute; and a partly transcribed part. The transcripts were made automatically with Whisper-small and count about 2.8 million utterances and about 30 million words. The authors compare the dataset with other Hebrew speech resources (SASPEECH, HUJI, CoSIH, MaTaCOp, CMU, MMS, Whisper) and argue that most of them are small, non-commercial or read speech. The paper gives no word error rate for the automatic transcripts.

Ideas

• I1. Use the transcripts as a large, noisy, spoken-register Modern Hebrew text, to measure how far Mova's Modern Hebrew rules hold on speech: sentence length, clitic use, and the share of colloquial forms. Use it only as an unlabelled sample; the Whisper transcripts are not gold.
• I2. Measure transcript noise before any use: sample utterances, hand-check them, and report a word error rate. The same deterministic correction approach as tools/ocrfix (rules over a lexicon) could then be tested on the transcripts, with its gain measured.
• I3. Use the segment length distribution from VAD (a few seconds to a minute) as a prior when Mova has to cut long spoken text into sentence-sized units.

For Mova

I1 is a corpus sample for 1. Overview: Modern Hebrew and 7. Syntax (spoken versus written register). I2 is a measurement for tools/ocrfix-style correction, placed in 2. Script and spelling. I3 is a note for 12. Mova and Modern Hebrew: where it stands, what it would need, since Mova does not read speech yet.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. arXiv 2307.08720

@misc{marmor2023ivrit,
  title = {{ivrit.ai: A Comprehensive Dataset of Hebrew Speech for AI Research and Development}},
  author = {Marmor, Yanir and Misgav, Kinneret and Lifshitz, Yair},
  year = {2023},
  eprint = {2307.08720},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2307.08720}
}

References

44 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Rodrigo Alarcon, Lourdes Moreno, and Paloma Martínez. 2021. Lexical simplification system to improve web accessibility. IEEE Access, 9:58755– 58767. find
2. Anne H Anderson, Miles Bader, Ellen Gurman Bard, Elizabeth Boyle, Gwyneth Doherty, Simon Garrod, Stephen Isard, Jacqueline Kowtko, Jan McAllister, Jim Miller, et al. 1991. The hcrc map task corpus. Language and speech, 34(4):351–366. find
3. Junyi Ao, Rui Wang, Long Zhou, Chengyi Wang, Shuo Ren, Yu Wu, Shujie Liu, Tom Ko, Qing Li, Yu Zhang, et al. 2021. Speecht5: Unified-modal encoder-decoder pre-training for spoken language processing. arXiv preprint arXiv:2110.07205. ↗
4. Jacob Azogui, Anat Lerner, and Vered Silber-Varod. 2016. The open university of israel map task corpus (matacop). find
5. Alan W Black. 2019. Cmu wilderness multilingual speech dataset. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 5971–5975. IEEE. find
6. Alexandra Canavan, David Graff, and George Zipperlen. 1997. Callhome american english speech. Linguistic Data Consortium. find
7. Avihay Chriqui and Inbal Yahav. 2022. Hebert and hebemo: A hebrew bert model and a tool for polarity analysis and emotion recognition. INFORMS Journal on Data Science, 1(1):81–95. ↗ our card
8. Heidi Christensen, Yoshihiko Gotoh, and Steve Renals. 2001. Punctuation annotation using statistical prosody models. find
9. Ann Clifton, Aasish Pappu, Sravana Reddy, Yongze Yu, Jussi Karlgren, Ben Carterette, and Rosie Jones. 2020. The spotify podcast dataset. arXiv preprint arXiv:2004.04270. ↗
10. Jia Cui, Xiaodong Cui, Bhuvana Ramabhadran, Janice Kim, Brian Kingsbury, Jonathan Mamou, Lidia Mangu, Michael Picheny, Tara N Sainath, and Abhinav Sethy. 2013. Developing speech recognition systems for corpus indexing under the iarpa babel program. In 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, pages 6753–6757. IEEE. find
11. John W Du Bois, Wallace L Chafe, Charles Meyer, Sandra A Thompson, and Nii Martey. 2000. Santa barbara corpus of spoken american english. CD-ROM. Philadelphia: Linguistic Data Consortium. find
12. Marcello Federico and Gareth JF Jones. 2004. The clef 2003 cross-language spoken document retrieval track. In Comparative Evaluation of Multilingual Information Access Systems: 4th Workshop of the Cross-Language Evaluation Forum, CLEF 2003, Trondheim, Norway, August 21-22, 2003, Revised Selected Papers 4, pages 646–652. Springer. find
13. J Garofolo, Jonathan Fiscus, and Audrey Le. 2004a. 2002 rich transcription broadcast news and conversational telephone speech. Linguistic Data Consortium. find
14. John S Garofolo. 1993. Timit acoustic phonetic continuous speech corpus. find
15. Linguistic Data Consortium, 1993. find
16. John S Garofolo, Cedric GP Auzanne, Ellen M Voorhees, et al. 2000. The trec spoken document retrieval track: A success story. NIST SPECIAL PUB-LICATION SP, 500(246):107–130. find
17. John S Garofolo, Christophe Laprun, Martial Michel, Vincent M Stanford, and Elham Tabassi. 2004b. The nist meeting room pilot corpus. In LREC. find
18. Eylon Guetta, Avi Shmidman, Shaltiel Shmidman, Cheyn Shmuel Shmidman, Joshua Guedalia, Moshe Koppel, Dan Bareket, Amit Seker, and Reut Tsarfaty. 2022. Large pre-trained models with extra-large vocabularies: A contrastive analysis of hebrew bert models and a new one to outperform them all. arXiv preprint arXiv:2211.15199. ↗ our card
19. Yoichiro Hasebe. 2015. Design and implementation of an online corpus of presentation transcripts of ted talks. Procedia-Social and Behavioral Sciences, 198:174–182. find
20. Charles T Hemphill, John J Godfrey, and George R Doddington. 1990. The atis spoken language systems pilot corpus. In Speech and Natural Language: Proceedings of a Workshop Held at Hidden Valley, Pennsylvania, June 24-27, 1990. find
21. Amr Hendy, Mohamed Abdelrehim, Amr Sharaf, Vikas Raunak, Mohamed Gabr, Hitokazu Matsushita, Young Jin Kim, Mohamed Afify, and Hany Hassan Awadalla. 2023. How good are gpt models at machine translation? a comprehensive evaluation. arXiv preprint arXiv:2302.09210. ↗
22. Shlomo Izre’el, Benjamin Hary, and Giora Rahav. 2001. Designing cosih: the corpus of spoken israeli hebrew. International Journal of Corpus Linguistics, 6(2):171–197. find
23. Siddique Latif, Aun Zaidi, Heriberto Cuayahuitl, Fahad Shamshad, Moazzam Shoukat, and Junaid Qadir. 2023. Transformers in speech processing: A survey. arXiv preprint arXiv:2303.11607. ↗
24. Shuo Liu, Adria Mallol-Ragolta, Emilia Parada-Cabaleiro, Kun Qian, Xin Jing, Alexander Kathan, Bin Hu, and Bjoern W Schuller. 2022. Audio self-supervised learning: A survey. Patterns, 3(12). find
25. Reza Lotfian and Carlos Busso. 2017. Building naturalistic emotionally balanced speech corpus by retrieving emotional speech from existing podcast recordings. IEEE Transactions on Affective Computing, 10(4):471–483. find
26. Bei Luo, Raymond YK Lau, Chunping Li, and Yain-Whar Si. 2022. A critical review of state-of-the-art chatbot designs and applications. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 12(1):e1434. find
27. James Manyika. 2023. An overview of bard: an early experiment with generative ai. find
28. Michal Marmorstein and Nadav Matalon. 2022. The huji corpus of spoken hebrew: An interaction-oriented design of a corpus. find
29. Jörgen Valk and Tanel Alumäe. 2021. Voxlingua107: a dataset for spoken language recognition. In 2021 IEEE Spoken Language Technology Workshop (SLT), pages 652–658. IEEE. find
30. Changhan Wang, Morgane Riviere, Ann Lee, Anne Wu, Chaitanya Talnikar, Daniel Haziza, Mary Williamson, Juan Pino, and Emmanuel Dupoux. 2021. Voxpopuli: A large-scale multilingual speech corpus for representation learning, semi-supervised learning and interpretation. arXiv preprint arXiv:2101.00390. ↗
31. Luz Martinez-Lucas, Mohammed Abdelwahab, and Carlos Busso. 2020. The msp-conversation corpus. Interspeech 2020. find
32. DeLiang Wang and Jitong Chen. 2018. Supervised speech separation based on deep learning: An overview. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 26(10):1702–1726. find
33. Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur. 2015. Librispeech: an asr corpus based on public domain audio books. In 2015 IEEE international conference on acoustics, speech and signal processing (ICASSP), pages 5206–5210. IEEE. find
34. Anne Wu, Changhan Wang, Juan Pino, and Jiatao Gu. 2020. Self-supervised representations improve end-to-end speech translation. arXiv preprint arXiv:2006.12124. ↗
35. Vineel Pratap, Andros Tjandra, Bowen Shi, Paden Tomasello, Arun Babu, Sayani Kundu, Ali Elkahky, Zhaoheng Ni, Apoorv Vyas, Maryam Fazel-Zarandi, et al. 2023. Scaling speech technology to 1,000+ languages. arXiv preprint arXiv:2305.13516. ↗
36. Vineel Pratap, Qiantong Xu, Anuroop Sriram, Gabriel Synnaeve, and Ronan Collobert. 2020. Mls: A large-scale multilingual dataset for speech research. arXiv preprint arXiv:2012.03411. ↗
37. Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2023. Robust speech recognition via large-scale weak supervision. In International Conference on Machine Learning, pages 28492–28518. PMLR. find
38. Konstantinos I Roumeliotis and Nikolaos D Tselikas. 2023. Chatgpt and open-ai models: A preliminary review. Future Internet, 15(6):192. find
39. Amit Seker, Elron Bandel, Dan Bareket, Idan Brusilovsky, Refael Shaked Greenfeld, and Reut Tsarfaty. 2021. Alephbert: A hebrew large pre-trained language model to start-off your hebrew nlp application with. arXiv preprint arXiv:2104.04052. ↗ our card
40. Orian Sharoni, Roee Shenberg, and Erica Cooper. 2023. Saspeech: A hebrew single speaker dataset for text to speech and voice conversion. In Proc. Interspeech 2023, page To Appear. find
41. Vered Silber-Varod, Ingo Siegert, Oliver Jokish, Yamini Sinha, and Nitza Geri. 2021. A cross-language study of speech recognition systems for english, german, and hebrew. Online Journal of Applied Knowledge Management, 9(1):1–15. find
42. Silero Team. 2021. Silero vad: pre-trained enterprise-grade voice activity detector (vad), number detector and language classifier. https://github.com/ snakers4/silero-vad. ↗
43. Qiantong Xu, Alexei Baevski, Tatiana Likhomanenko, Paden Tomasello, Alexis Conneau, Ronan Collobert, Gabriel Synnaeve, and Michael Auli. 2021. Self-training and pre-training are complementary for speech recognition. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 3030–3034. IEEE. find
44. Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. 2023. A survey of large language models. arXiv preprint arXiv:2303.18223. ↗

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/langstream-2023-marmor-ivrit-ai-comprehensive-dataset-hebrew/