Mova0.5 pre-release⌕
☰
home / docs / papers / Modern Hebrew / Towards Safer Hebrew Communication: A Dataset for Offensive Language Detoxification

Towards Safer Hebrew Communication: A Dataset for Offensive Language Detoxification

What it is about

The paper builds HeDetox, 600 Hebrew sentence pairs in which an explicitly offensive forum comment is paired with a non-offensive rewrite. Offensive comments were chosen with a taxonomy-based classifier, with about 12% oversampling to drop borderline cases. GPT-4o rewrote them with a few-shot chain-of-thought prompt, and two annotators plus a judge corrected the rewrites under rules against heavy paraphrase. Of 100 reviewed LLM rewrites, 38% had errors, in five types: over-softening (22%), omitted content (17%), added content (14%), wrong synonyms (8%), and offensive words kept (2%). In the LLM output, sentences are shorter, proper nouns drop sharply, and MTLD falls to 0.027 (0.714 for the originals). Adding the pairs to the OLaH offensive-detection training set raises heBERT F1 from 0.700 to 0.720 and multilingual BERT from 0.586 to 0.703 on the OLaH test set.

Ideas

For Mova

I1 is a rule layer over the trees in prag (a diff of frames between source and output). I2 is a measurement for the teacher-data notes in docs/. I3 is a test list for 8. Lexicon and register. The dataset itself is a held-out test set at most, never training data for the SLM.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.26615/978-954-452-098-4-149

@inproceedings{vanetik2025safer,
  title = {{Towards Safer Hebrew Communication: A Dataset for Offensive Language Detoxification}},
  author = {Vanetik, Natalia and Liberov, Lior and Litvak, Marina and Liebeskind, Chaya},
  booktitle = {Proceedings of the 15th International Conference on Recent Advances in Natural Language Processing - Natural Language Processing in the Generative AI Era},
  year = {2025},
  doi = {10.26615/978-954-452-098-4-149},
  url = {https://doi.org/10.26615/978-954-452-098-4-149}
}

References

40 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Katherine Atwell, Sabit Hassan, and Malihe Alikhani. 2022. Appdia: A discourse-aware transformer-based style transfer model for offensive social media conversations. In Proceedings of the 29th International Conference on Computational Linguistics, pages 6063–6074. find
  2. Cristina Bosco, Felice Dell’Orletta, Fabio Poletto, Manuela Sanguinetti, Maurizio Tesconi, et al. 2018. Overview of the EVALITA 2018 Hate Speech Detection Task. In CEUR workshop proceedings, volume 2263, pages 1–9. CEUR. find
  3. Avihay Chriqui and Inbal Yahav. 2022. Hebert & hebemo: a hebrew bert model and a tool for polarity analysis and emotion recognition. INFORMS Journal on Data Science. find our card
  4. Daryna Dementieva, Nikolay Babakov, and Alexander Panchenko. 2024a. Multiparadetox: Extending text detoxification with parallel data to new languages. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers), pages 124–140. find
  5. Daryna Dementieva, Nikolay Babakov, Amit Ronen, Abinew Ali Ayele, Naquee Rizwan, Florian Schneider, Xintong Wang, Seid Muhie Yimam, Daniil Alekhseevich Moskovskiy, Elisei Stakovskii, et al. 2025. Multilingual and explainable text detoxification with parallel corpora. In Proceedings of the 31st International Conference on Computational Linguistics, pages 7998–8025. find
  6. Daryna Dementieva, Daniil Moskovskiy, Nikolay Babakov, Abinew Ali Ayele, Naquee Rizwan, Frolian Schneider, Xintog Wang, Seid Muhie Yimam, Dmitry Ustalov, Elisei Stakovskii, et al. 2024b. Overview of the multilingual text detoxification task at pan 2024. Working Notes of CLEF. find
  7. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4171–4186. find
  8. Matan Eyal, Yoav Kantor, and Reut Tsarfaty. 2022. Multilingual sequence-to-sequence models for hebrew nlp. find our card
  9. Paula Fortuna and Sérgio Nunes. 2018. A survey on automatic detection of hate speech in text. ACM Computing Surveys, 51(4):1–30. find
  10. Eliyahu Gueta, Dafna Shahaf, and Reut Tsarfaty. 2023. Explicit morphological knowledge improves pre-training of language models for hebrew. find our card
  11. Nagham Hamad, Mustafa Jarrar, Mohammad Khalilia, and Nadim Nashif. 2023. Offensive hebrew corpus and detection using bert. In 2023 20th ACS/IEEE International Conference on Computer Systems and Applications (AICCSA), pages 1–8. IEEE. find our card
  12. Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22199– 22213. find
  13. Ritesh Kumar, Atul Kr Ojha, Shervin Malmasi, and Marcos Zampieri. 2018. Benchmarking aggression identification in social media. In Proceedings of the first workshop on trolling, aggression and cyberbullying (TRAC-2018), pages 1–11. find
  14. Barbara Lewandowska-Tomaszczyk, Anna Baczkowska, Olga Dontcheva-Navrátilová, Chaya Liebeskind, Giedrė Valūnaitė Oleškevičienė, Slavko Žitnik, Marcin Trojszczak, Renata Povolná, Linas Selmistraitis, Andrius Utka, and Dangis Gudelis. 2023a. Llod schema for simplified offensive language taxonomy in multilingual detection and applications. Lodz Papers in Pragmatics, 19(2):301–324. find
  15. Barbara Lewandowska-Tomaszczyk, Anna Baczkowska, Chaya Liebeskind, Giedre Valunaite Oleskeviciene, and Slavko Žitnik. 2023b. An integrated explicit and implicit offensive language taxonomy. Lodz Papers in Pragmatics, 19(1):7–48. find
  16. Dongyuan Li, Zhen Wang, Yankai Chen, Renhe Jiang, Weiping Ding, and Manabu Okumura. 2024. A survey on deep active learning: Recent advances and new frontiers. IEEE Transactions on Neural Networks and Learning Systems, 36(4):5879–5899. find
  17. Chaya Liebeskind and Shmuel Liebeskind. 2018. Identifying abusive comments in hebrew facebook. In 2018 IEEE International conference on the science of electrical engineering in Israel (ICSEE), pages 1–5. IEEE. find
  18. Chaya Liebeskind, Marina Litvak, and Natalia Vanetik. 2024. From linguistics to practice: a case study of offensive language taxonomy in hebrew. In Proceedings of the 8th Workshop on Online Abuse and Harms (WOAH 2024), pages 110–117. find our card
  19. Chaya Liebeskind, Natalia Vanetik, and Marina Litvak. 2023. Hebrew offensive language taxonomy and dataset. Lodz Papers in Pragmatics, 19(2):325–351. find
  20. Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text Summarization Branches Out: Proceedings of the ACL-04 Workshop, pages 74–81, Barcelona, Spain. find
  21. Marina Litvak, Natalia Vanetik, Chaya Liebeskind, Omar Hmdia, and Rizek Abu Madeghem. 2022. Offensive language detection in hebrew: can other languages help? In Proceedings of the thirteenth language resources and evaluation conference, pages 3715–3723. find our card
  22. Marina Litvak, Natalia Vanetik, Yaser Nimer, Abdulrhman Skout, and Israel Beer-Sheba. 2021. Offensive language detection in semitic languages. In Multimodal Hate Speech Workshop 2021, pages 7–12. find
  23. Hamidreza Rouzegar and Masoud Makrehchi. 2024. Enhancing text classification through llm-driven active learning and human annotation. arXiv preprint arXiv:2406.12114. ↗
  24. Varvara Logacheva, Daryna Dementieva, Sergey Ustyantsev, Daniil Moskovskiy, David Dale, Irina Krotova, Nikita Semenov, and Alexander Panchenko. 2022. Paradetox: Detoxification with parallel data. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6804–6818. find
  25. Natalia Vanetik, Lior Liberov, Chaya Liebeskind, and Marina Litvak. 2025. Hedetox: Hebrew detoxification dataset and annotation guidelines. find
  26. Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-sne. Journal of machine learning research, 9(11). find
  27. Thomas Mandl, Sandip Modha, Prasenjit Majumder, Daksh Patel, Mohana Dave, Chintak Mandlia, and Aditya Patel. 2019. Overview of the HASOC track at FIRE 2019: Hate Speech and Offensive Content Identification in Indo-European Languages. In Proceedings of the 11th annual meeting of the Forum for Information Retrieval Evaluation, pages 14–17. find
  28. Philip M McCarthy. 2005. An assessment of the range and usefulness of lexical diversity measures and the potential of the measure of textual, lexical diversity (MTLD). Ph.D. thesis, The University of Memphis. find
  29. Gideon Mendels. 2015. Hebrewstopwords. https://github.com/gidim/HebrewStopWords . Accessed: 2025-07-27. ↗
  30. Daniil Moskovskiy, Nikita Sushko, Sergey Pletenev, Elena Tutubalina, and Alexander Panchenko. 2025. SynthDetoxM: Modern LLMs are Few-Shot Parallel Detoxification Data Annotators. arXiv preprint arXiv:2502.06394. ↗
  31. OpenAI. 2024. Gpt-4o technical report. https:// openai.com/research/gpt-4o . ↗
  32. Nedjma Djouhra Ousidhoum, Zizheng Lin, Hongming Zhang, Yangqiu Song, and Dit Yan Yeung. 2019. Multilingual and multi-aspect hate speech analysis. In EMNLP-IJCNLP 2019-2019 Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing, Proceedings of the Conference. https://github.com/NataliaVanetik/ OffensiveLanguageResearchLab#hedetox . ↗
  33. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837. find
  34. Michael Wiegand, Josef Ruppenhofer, and Thomas Kleinbauer. 2018. Overview of the GermEval 2018 shared task on the identification of offensive language. In Proceedings of GermEval, pages 1–10. find
  35. Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Sara Rosenthal, Noura Farra, and Ritesh Kumar. 2019a. Predicting the type and target of offensive posts in social media. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 1415–1420. find
  36. Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Sara Rosenthal, Noura Farra, and Ritesh Kumar. 2019b. Semeval-2019 task 6: Identifying and categorizing offensive language in social media (offenseval). In Proceedings of the 13th International Workshop on Semantic Evaluation, pages 75–86. find
  37. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2020. Bertscore: Evaluating text generation with bert. In International Conference on Learning Representations. find
  38. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. find our card
  39. W. Tzuf Paz-Argaman, Guy Arviv, Yoav Kantor, and Reut Tsarfaty. 2024. Hesum: A novel dataset for abstractive text summarization in hebrew. find our card
  40. Fabio Poletto, Valerio Basile, Manuela Sanguinetti, Cristina Bosco, and Viviana Patti. 2021. Resources and benchmark corpora for hate speech detection: a systematic review. Language Resources and Evaluation, 55:477–523. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2025-vanetik-safer-hebrew-communication-dataset-offensive/