Detection of Negative Campaign in Israeli Municipal Elections

What it is about

The paper introduces TONIC, a dataset of Hebrew Facebook posts by 27 mayoral candidates in 12 Israeli cities (2018 municipal elections). Of 4,426 posts, 2,632 were campaign-related; 2,064 of these were labelled negative campaign (majority rate 0.784; annotator kappa 0.862). The authors compare traditional classifiers (random forest, logistic regression, XGBoost) on tf-idf, character n-grams, unigrams to trigrams and BERT vectors, with and without sentiment labels from HeBERT, against fine-tuned multilingual BERT and AlephBERT. The best model is logistic regression on unigrams with sentiment labels (F1 0.819, accuracy 0.884, test split 20%). Transfer from an offensive-language dataset (OLaH) lowers accuracy for every model, so the two tasks differ. Of 30 misclassified posts that were inspected, 23 errors came from the annotators' labels, and 22 of the 30 were false negatives.

Ideas

• I1. Count-based baseline first: on about 2,600 posts, unigram counts plus a sentiment feature beat fine-tuned transformers. Before any Hebrew model is claimed to help, Mova needs this kind of baseline, and a sentiment score from a lexicon can be tested as one more feature (9. Pragmatics and politeness).
• I2. Audit the gold labels: the authors found most of their errors were annotation errors, not model errors. Take a random sample of disagreements on any Hebrew test set, re-label them by hand, and report the change beside the score.
• I3. Negative about a person is not the same as offensive: the paper separates content that attacks an opponent from offensive language. A pragmatic rule can flag a post as attacking a named candidate (target present, not vulgar) without using the offence labels.

For Mova

I1 and I3 go into 9. Pragmatics and politeness as tests. I2 is a rule for the test plan in 12. Mova and Modern Hebrew: where it stands, what it would need. The dataset is a candidate test set; its licence is CC BY 4.0 as given, and it is not used as training data for the release until its use is checked.

Cite

The paper itself, not this card. BibTeX from the library's record of the work.

@inproceedings{litvak2022detection,
  title = {{Detection of Negative Campaign in Israeli Municipal Elections}},
  author = {Litvak, Marina and Vanetik, Natalia and Talker, Sagiv and Machlouf, Or},
  booktitle = {Proceedings of the Third Workshop on Threat, Aggression and Cyberbullying (TRAC 2022)},
  year = {2022},
  url = {https://aclanthology.org/2022.trac-1.9/}
}

References

22 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dandelion Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. 2015. TensorFlow: Large-scale machine learning on heterogeneous systems. Software available from tensorflow.org. find
2. Hadi Abdine, Yanzhu Guo, Virgile Rennard, and Michalis Vazirgiannis. 2022. Political communities on twitter: Case study of the 2022 french presidential election. In Proceedings of The LREC 2022 workshop on Natural Language Processing for Political Sciences, pages 62–71, Marseille, France. European Language Resources Association. find
3. Haithem Afli, Mehwish Alam, Houda Bouamor, Cristina Blasi Casagran, Colleen Boland, and Sahar Ghannay, editors. 2022. Proceedings of The LREC 2022 workshop on Natural Language Processing for Political Sciences. European Language Resources Association, Marseille, France. find
4. Joseph Asunka, Sarah Brierley, Miriam Golden, Eric Kramon, and George Ofosu. 2019. Electoral fraud or violence: The effect of observers on party manipulation strategies. British Journal of Political Science, 49(1):129–151. find
5. Mateusz Baran, Mateusz Wójcik, Piotr Kolebski, MichaÅ‚ Bernaczyk, Krzysztof Rajda, Lukasz Augustyniak, and Tomasz Kajdanowicz. 2022. Electoral agitation dataset: The use case of the polish election. In Proceedings of The LREC 2022 workshop on Natural Language Processing for Political Sciences, pages 32–36, Marseille, France. European Language Resources Association. find
6. Dan Bernhardt and Meenakshi Ghosh. 2020. Positive and negative campaigning in primary and general elections. Games and Economic Behavior, 119:98– 104. find
7. Ekaba Bisong. 2019. Building machine learning and deep learning models on Google cloud platform: A comprehensive guide for beginners. Apress. find
8. Ashish Chaturvedi. 2005. Rigging elections with violence. Public Choice, 125(1):189–202. find
9. Tianqi Chen, Tong He, Michael Benesty, Vadim Khotilovich, Yuan Tang, Hyunsu Cho, Kailong Chen, et al. 2015. Xgboost: extreme gradient boosting. R package version 0.4-2, 1(4):1–4. find
10. François Chollet et al. 2015. Keras. https:// github.com/fchollet/keras. ↗
11. Avihay Chriqui and Inbal Yahav. 2021. Hebert & hebemo: a hebrew bert model and a tool for polarity analysis and emotion recognition. arXiv preprint arXiv:2102.01909. ↗ our card
12. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: pre-training of deep bidirectional transformers for language understanding. CoRR, abs/1810.04805. ↗
13. Giovanna Maria Invernizzi. 2019. Electoral competition and factional sabotage. Available at SSRN 3329622. find
14. Marina Litvak, Natalia Vanetik, Yaser Nimer, Abdulrhman Skout, and Israel Beer-Sheba. 2021. Offensive language detection in semitic languages. In Multimodal Hate Speech Workshop 2021, pages 7–12. find
15. Paul S Martin. 2004. Inside the black box of negative campaign effects: Three reasons why negative campaigns mobilize. Political psychology, 25(4):545– 562. find
16. Mahesh Pal. 2005. Random forest classifier for remote sensing classification. International journal of remote sensing, 26(1):217–222. find
17. F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830. find
18. Eric Sanders and Antal van den Bosch. 2022. Correlating political party names in tweets, newspapers and election results. In Proceedings of The LREC 2022 workshop on Natural Language Processing for Political Sciences, pages 8–15, Marseille, France. European Language Resources Association. find
19. Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108. ↗
20. Amit Seker, Elron Bandel, Dan Bareket, Idan Brusilovsky, Refael Shaked Greenfeld, and Reut Tsarfaty. 2021. Alephbert: A hebrew large pre-trained language model to start-off your hebrew nlp application with. arXiv preprint arXiv:2104.04052. ↗ our card
21. Stergios Skaperdas and Bernard Grofman. 1995. Modeling negative campaigning. American Political Science Review, 89(1):49–61. find
22. Raymond E Wright. 1995. Logistic regression. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2022-litvak-detection-negative-campaign-israeli-municipal/