Offensive Hebrew Corpus and Detection using BERT

What it is about

The paper releases a Hebrew offensive-language corpus of 15,881 tweets, collected in late 2020 and early 2021 with 55 keywords and labelled by Arabic-Hebrew bilingual annotators. Each tweet gets one or more of five classes (abusive, hate, violence, pornographic, none), plus the target group, the topic and the offensive phrase. Only about 1,200 tweets are offensive, so the authors balance the set to 2,500 for experiments. They fine-tune HeBERT and AlephBERT as binary classifiers, alone and combined with an older Hebrew Facebook dataset. The data is not reviewed fully: a second annotator revised 21% of it, and the number of changes shows where the labels are unstable (target labels changed most). AlephBERT trained on the new data and tested on the Facebook set reaches 69% accuracy, and the reverse reaches 57%. Adding the new data to the older set helps HeBERT by about 2 points but hurts AlephBERT.

Ideas

• I1. Report cross-dataset accuracy, not only in-domain accuracy. The 69% and 57% figures show that a label set does not transfer well between annotation guidelines. Any Mova offence or register detector should be tested on a second source before a claim is made.
• I2. Use the paper's annotation schema as a data format for pragmatic labels: a class, a target, a topic and the phrase that triggers the label. In prag this maps to a labelled span with a reason, which fits Mova's rule of explaining each decision.
• I3. Measure label instability by counting reviewer changes per label, as the paper does (targets changed most). A label that changes often needs a rule or a smaller label set before it is used as a gold standard.

For Mova

I1 is a measurement rule for any register or offence work in 9. Pragmatics and politeness. I2 is a data schema for prag, Hebrew side, and fits 8. Lexicon and register. I3 is a check for the labels in 12. Mova and Modern Hebrew: where it stands, what it would need before they are used. No text from the corpus is carried into Mova: the licence is non-commercial, so it is only read for the method.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. arXiv 2309.02724

@misc{hamad2023offensive,
  title = {{Offensive Hebrew Corpus and Detection using BERT}},
  author = {Hamad, Nagham and Jarrar, Mustafa and Khalilia, Mohammad and Nashif, Nadim},
  year = {2023},
  eprint = {2309.02724},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2309.02724}
}

References

38 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. S. Alsafari, S. Sadaoui, and M. Mouhoub, “Hate and offensive speech detection on arabic social media,” Online Social Networks and Media, vol. 19, p. 100096, 2020. find
2. Y. Otiefy, A. Abdelmalek, and I. E. Hosary, “Woli at semeval-2020 task 12: Arabic offensive language identification on different twitter datasets,” CoRR, 2020. find
3. H. Mubarak, S. Hassan, and S. A. Chowdhury, “Emojis as anchors to detect arabic offensive language and hate speech,” CoRR, vol. abs/2201.06723, 2022. ↗
4. N. AlHazzani, A. AlDawod, H. AlMazrou, L. AlAwlaqi, N. Al-Reshoudi, H. Al-Khalifa, and L. AlDhubayi, “Sa7r: A saudi dialect irony dataset,” pp. 60–70, 2022. find
5. Z. Waseem, T. Davidson, D. Warmsley, and I. Weber, “Understanding abuse: A typology of abusive language detection subtasks,” CoRR, vol. abs/1705.09899, 2017. ↗
6. F. Shannag, B. H. Hammo, and H. Faris, “The design, construction and evaluation of annotated arabic cyberbullying corpus,” Education and Information Technologies, pp. 1–47, 2022. find
7. D. Abuaiadah, D. Rajendran, and M. Jarrar, “Clustering arabic tweets for sentiment analysis,” in Proceedings of the 2017 IEEE/ACS 14th International Conference on Computer Systems and Applications. IEEE Computer Society, November 2017, pp. 499–506. [Online]. Available: https://doi.org/10.1109/AICCSA.2017.162 ↗
8. R. Dankar, H. Ghattas, F. Zaraket, M. Jarrar, H. Amayreh, W. Magdi, A. Rutherford, D. Masood, and M. Waehlisch, “Social media opinion mining and sentiment analysis - bahrain elections.” United Nations, Tech. Rep., 2018. Accuracy He Aleph BERT BERT 63% 68% 58% 63% 61% 63% 79% 86% 81% 79% 81% 82% 60% 57% 64% 69% Validation data Test data 250 (ours) 250 (ours) 250 (ours) 203 (D OLaH ) 203 (D OLaH ) 203 (D OLaH ) 203 (D OLaH ) 250 (ours) 500 (ours) 500 (ours) 500 (ours) 405 (D OLaH ) 405 (D OLaH ) 405 (D OLaH ) 500 (ours) 405 (D OLaH ) find
9. M. Jarrar, M. Khalilia, and S. Ghanem, “Wojood: Nested arabic named entity corpus and recognition using bert,” in Proceedings of the International Conference on Language Resources and Evaluation (LREC 2022), Marseille, France, June 2022. find
10. M. Al-Hajj and M. Jarrar, “Arabglossbert: Fine-tuning bert on context-gloss pairs for wsd.” in Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021). Online: INCOMA Ltd., sep 2021, pp. 40–48. [Online]. Available: https://doi.org/10.26615/978-954-452-072-4_005 ↗
12. H. Mubarak, K. Darwish, and W. Magdy, “Abusive language detection on arabic social media,” in Proceedings of the first workshop on abusive language online, 2017, pp. 52–56. find
13. M. Jarrar, “The arabic ontology - an arabic wordnet with ontologically clean content,” Applied Ontology Journal, vol. 16, no. 1, pp. 1–26, 2021. find
14. M. Jarrar, N. Habash, F. Alrimawi, D. Akra, and N. Zalmout, “Curras: An annotated corpus for the palestinian arabic dialect,” Journal Language Resources and Evaluation, vol. 51, no. 3, pp. 745–775, September 2017. find
15. K. E. Haff, M. Jarrar, T. Hammouda, and F. Zaraket, “Curras + baladi: Towards a levantine corpus,” in Proceedings of the International Conference on Language Resources and Evaluation (LREC 2022), Marseille, France, June 2022. find
16. M. Jarrar, F. A. Zaraket, T. Hammouda, D. M. Alavi, and M. Waahlisch, “Lisan: Yemeni, irqi, libyan, and sudanese arabic dialect copora with morphological annotations,” 2022. [Online]. Available: https://arxiv.org/abs/2212.06468 ↗
17. C. Liebeskind and S. Liebeskind, “Identifying abusive comments in hebrew facebook,” in 2018 IEEE International Conference on the Science of Electrical Engineering in Israel (ICSEE). IEEE, 2018, pp. 1–5. find
18. M. Litvak, N. Vanetik, Y. Nimer, A. Skout, and I. Beer-Sheba, “Offensive language detection in semitic languages,” in Multimodal Hate Speech Workshop 2021, 2021, pp. 7–12. find
19. N. Vanetik, M. Litvak, C. Liebeskind, O. Hmdia, and R. A. Madeghem, “Offensive language detection in hebrew: can other languages help?” in Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), 2022, pp. 3715––3723. find our card
20. A. Chriqui and I. Yahav, “Hebert & hebemo: a hebrew bert model and a tool for polarity analysis and emotion recognition,” arXiv preprint arXiv:2102.01909, 2021. ↗ our card
21. A. Seker, E. Bandel, D. Bareket, I. Brusilovsky, R. S. Greenfeld, and R. Tsarfaty, “Alephbert: A hebrew large pre-trained language model to start-off your hebrew nlp application with,” CoRR, vol. abs/2104.04052, 2021. ↗ our card
22. K. Darwish, N. Habash, M. Abbas, H. Al-Khalifa, H. T. Al-Natsheh, H. Bouamor, K. Bouzoubaa, V. Cavalli-Sforza, S. R. El-Beltagy, W. El-Hajj, M. Jarrar, and H. Mubarak, “A panoramic survey of natural language processing in the arab worlds,” Commun. ACM, vol. 64, no. 4, p. 72–81, April 2021. find
23. M. Zampieri, S. Malmasi, P. Nakov, S. Rosenthal, N. Farra, and R. Kumar, “Semeval-2019 task 6: Identifying and categorizing offensive language in social media (offenseval),” in Proceedings of the 13th International Workshop on Semantic Evaluation, SemEval@NAACL-HLT 2019, Minneapolis, MN, USA, June 6-7, 2019. Association for Computational Linguistics, 2019, pp. 652–656. find
24. ——, “Predicting the type and target of offensive posts in social media,” arXiv preprint arXiv:1902.09666, 2019. ↗
25. H. Ahn, J. Sun, C. Y. Park, and J. Seo, “Nlpdove at semeval-2020 task 12: Improving offensive language detection with cross-lingual transfer,” arXiv preprint arXiv:2008.01354, pp. 1576–1586, 2020. ↗
26. A. Al-Hassan and H. Al-Dossari, “Detection of hate speech in social networks: a survey on multilingual corpus,” in 6th International Conference on Computer Science and Information Technology, vol. 10, 2019, pp. 10–5121. find
27. E. Elsawy, M. Mokhtar, and W. Magdy, “Tweetmogaz v2: Identifying news stories in social media,” in Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management, 2014, pp. 2042–2044. find
28. N. Albadi, M. Kurdi, and S. Mishra, “Are they our brothers? analysis and detection of religious hate speech in the arabic twittersphere,” in 2018 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM). IEEE, 2018, pp. 69–76. find
29. A. Al-Hassan and H. Al-Dossari, “Detection of hate speech in arabic tweets using deep learning,” Multimedia Systems, pp. 1–12, 2021. find
30. R. Duwairi, A. Hayajneh, and M. Quwaider, “A deep learning framework for automatic detection of hate speech embedded in arabic tweets,” Arabian Journal for Science and Engineering, vol. 46, no. 4, pp. 4001–4014, 2021. find
31. I. A. Farha and W. Magdy, “Multitask learning for arabic offensive language and hate-speech detection,” in Proceedings of the 4th workshop on open-source Arabic corpora and processing tools, with a shared task on offensive language detection, 2020, pp. 86–90. find
32. H. Mubarak, A. Rashed, K. Darwish, Y. Samih, and A. Abdelali, “Arabic offensive language on twitter: Analysis and experiments,” in Proceedings of the Sixth Arabic Natural Language Processing Workshop, WANLP 2021, Kyiv, Ukraine (Virtual), April 9, 2021. Association for Computational Linguistics, 2020, pp. 126–135. find
33. M. Zampieri, P. Nakov, S. Rosenthal, P. Atanasova, G. Karadzhov, H. Mubarak, L. Derczynski, Z. Pitenis, and Ç. Çöltekin, “Semeval-2020 task 12: Multilingual offensive language identification in social media (offenseval 2020),” arXiv preprint arXiv:2006.07235, 2020. ↗
34. S. A. Chowdhury, A. Abdelali, K. Darwish, J. Soon-Gyo, J. Salminen, and B. J. Jansen, “Improving arabic text categorization using transformer training diversification,” in Proceedings of the fifth arabic natural language processing workshop, 2020, pp. 226–236. find
35. W. Antoun, F. Baly, and H. Hajj, “Arabert: Transformer-based model for arabic language understanding,” arXiv preprint arXiv:2003.00104, 2020. ↗
36. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” vol. abs/1810.04805. Association for Computational Linguistics, 2018, pp. 4171–4186. ↗
37. A. Conneau, K. Khandelwal, N. Goyal, V. Chaudhary, G. Wenzek, F. Guzmán, E. Grave, M. Ott, L. Zettlemoyer, and V. Stoyanov, “Unsupervised cross-lingual representation learning at scale.” Association for Computational Linguistics, 2020, pp. 8440–8451. find
38. A. Abdelali, S. Hassan, H. Mubarak, K. Darwish, and Y. Samih, “Pre-training bert on arabic tweets: Practical considerations,” CoRR, vol. abs/2102.10684, 2021. ↗
39. P. J. O. Suárez, B. Sagot, and L. Romary, “Asynchronous pipeline for processing huge corpora on medium to low resource infrastructures,” in 7th Workshop on the Challenges in the Management of Large Corpora (CMLC-7). Leibniz-Institut für Deutsche Sprache, 2019. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/langstream-2023-hamad-offensive-hebrew-corpus-detection-using/