How Challenging is Sarcasm versus Irony Classification?: A Study With a Dataset from English Literature

What it is about

The paper asks whether sarcasm (irony with a target of ridicule, so it changes sentiment towards that target) can be told from plain irony, using 4,306 quotes from literature with user tags (753 sarcasm, 677 irony, 2,876 philosophy). Three experienced annotators labelled 501 of them: their agreement with the tags is lower for sarcasm vs irony than for sarcasm vs philosophy (for one annotator kappa 0.62 vs 0.65, the same trend for all three); confusions come from self-deprecation and from contempt expressed without positive words. An SVM with the incongruity features of Joshi et al. (2015) reaches average F 0.846 for sarcasm vs philosophy (0.80 class-balanced) but only 0.654 for sarcasm vs irony. The authors note that a sarcastic quote can become a sincere one by changing a single word, and vice versa.

Ideas

• I1. Literary quotes, not tweets, are the closest outside material to Mova's tales, and the result says the sarcasm/irony border is hard even for people. Keep Mova's irony output at one level (ironic or not) plus the target, and decide sarcasm only by the explicit target rule (a person, especially the addressee, as the object of the reversed evaluation; see zeng-2022-survey-automatic-irony-processing-linguistic I2) rather than by a learned classifier.
• I2. The one-word switch observation is a ready minimal-pair test for prag/src/irony.rs: pairs that differ in one evaluative or situational word («a woman needs a man like a fish needs a bicycle» vs «… like a fish needs water») must flip the candidate decision. Write a dozen such pairs by hand from public-domain sayings and keep them as a gate with negative controls.

For Mova

34. Irony and figurative language §2 (verbal irony and sarcasm), §10 (Mova's detector), §12 (figurative language in the tales). Code: prag/src/irony.rs. Measure: the minimal-pair gate (all pairs must flip).

Cite

The paper itself, not this card. BibTeX from the library's record of the work.

@inproceedings{joshi2016how,
  title = {{How Challenging is Sarcasm versus Irony Classification?: A Study With a Dataset from English Literature}},
  author = {Joshi, Aditya and Tripathi, Vaibhav and Bhattacharyya, Pushpak and Carman, Mark and Singh, Meghna and Saraswati, Jaya and Shukla, Rajita},
  booktitle = {Proceedings of the Australasian Language Technology Association Workshop 2016},
  year = {2016},
  url = {https://aclanthology.org/U16-1013/}
}

References

22 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Rehan Akbani, Stephen Kwek, and Nathalie Japkow-icz, 2004. Machine Learning: ECML 2004: 15th European Conference on Machine Learning, Pisa, Italy, September 20-24, 2004. Proceedings, chapter Applying Support Vector Machines to Imbalanced Datasets, pages 39–50. Springer Berlin Heidelberg, Berlin, Heidelberg. find
2. David Bamman and Noah A Smith. 2015. Contextualized sarcasm detection on twitter. find
3. Konstantin Buschmeier, Philipp Cimiano, and Roman Klinger. 2014. An impact analysis of features in a classification approach to irony detection in product reviews. find
4. Dmitry Davidov, Oren Tsur, and Ari Rappoport. 2010. Semi-supervised recognition of sarcastic sentences in twitter and amazon. In Proceedings of the Fourteenth Conference on Computational Natural Language Learning, pages 107–116. Association for Computational Linguistics. find
5. David K Elson and Kathleen McKeown. 2010. Automatic attribution of quoted speech in literary narrative. In AAAI. Citeseer. find
6. Roberto González-Ibáñez, Smaranda Muresan, and Nina Wacholder. 2011. Identifying sarcasm in twitter: A closer look. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: Short Papers - Volume 2, HLT ’11, pages 581–586, Stroudsburg, PA, USA. Association for Computational Linguistics. find
7. Aditya Joshi, Vinita Sharma, and Pushpak Bhattacharyya. 2015. Harnessing context incongruity for sarcasm detection. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, volume 2, pages 757–762. find
8. Aditya Joshi, Kevin Patel, Vaibhav Tripathi, Pushpak Bhattacharyya, and Mark J Carman. 2016. Are word embedding-based features useful for sarcasm detection? In EMNLP. find
9. Roger J. Kreuz and Gina M. Caucci. 2007. Lexical influences on the perception of sarcasm. In Proceedings of the Workshop on Computational Approaches to Figurative Language, FigLanguages ’07, pages 1– 4, Stroudsburg, PA, USA. Association for Computational Linguistics. find
10. Christopher J Lee and Albert N Katz. 1998. The differential role of ridicule in sarcasm and irony. Metaphor and Symbol, 13(1):1–15. find
11. Jennifer Ling and Roman Klinger. 2016. An empirical, quantitative analysis of the differences between sarcasm and irony. In Semantic Web: ESWC 2016 Satellite Events. find
12. Stephanie Lukin and Marilyn Walker. 2013. Really? well. apparently bootstrapping improves the performance of sarcasm and nastiness classifiers for online dialogue. find
13. Diana Maynard and Mark A Greenwood. 2014. Who cares about sarcastic tweets? investigating the impact of sarcasm on sentiment analysis. find
14. Soujanya Poria, Erik Cambria, Devamanyu Hazarika, and Prateek Vij. 2016. A deeper look into sarcastic tweets using deep convolutional neural networks. arXiv preprint arXiv:1610.08815. ↗
15. Antonio Reyes, Paolo Rosso, and Davide Buscaldi. 2012. From humor recognition to irony detection: The figurative language of social media. Data Knowl. Eng., 74:1–12, April. find
16. Ellen Riloff, Ashequl Qadir, Prafulla Surve, Lalin-dra De Silva, Nathan Gilbert, and Ruihong Huang. 2013a. Sarcasm as contrast between a positive sentiment and negative situation. find our card
17. Ellen Riloff, Ashequl Qadir, Prafulla Surve, Lalin-dra De Silva, Nathan Gilbert, and Ruihong Huang. 2013b. Sarcasm as contrast between a positive sentiment and negative situation. In EMNLP, volume 13, pages 704–714. find our card
18. Andrew Skabar and Khaled Abdalgader. 2010. Improving sentence similarity measurement by incorporating sentential word importance. In AI 2010: Advances in Artificial Intelligence, pages 466–475. Springer. find
19. Anders Søgaard. 2012. Mining wisdom. NAACL-HLT 2012, page 54. find
20. Yuchun Tang, Yan-Qing Zhang, Nitesh V Chawla, and Sven Krasser. 2009. Svms modeling for highly imbalanced classification. Systems, Man, and Cybernetics, Part B: Cybernetics, IEEE Transactions on, 39(1):281–288. find
21. Joseph Tepperman, David R Traum, and Shrikanth Narayanan. 2006. ” yeah right”: sarcasm recognition for spoken dialogue systems. find
22. Byron C Wallace and Laura Kertz Do Kook Choe. 2014. Humans require context to infer ironic intent (so computers probably do, too). find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/joshi-2016-how-challenging-is-sarcasm-versus/