Mova0.5 pre-release⌕
☰
home / docs / papers / English / A fine-grained comparison of pragmatic language understanding in humans and language models

A fine-grained comparison of pragmatic language understanding in humans and language models

What it is about

Seven pragmatic phenomena (white lies, indirect requests, irony, flouted maxims, metaphor, humour, coherence) are tested with expert-written multiple-choice items: a 1–3 sentence story, a character's utterance, and «what did they mean?». The options are typed: the pragmatic answer, the literal answer, and distractors built from lexical overlap with the story, loose association, or a non-sequitur. Models up to 11B parameters and GPT-3.5 are compared with 374 human participants, zero-shot. Small models are at chance; the largest reach near-human accuracy on several tasks and, like humans, err mostly towards the literal option rather than the distractors; humour, irony and maxims (expectation violations) stay hardest. Removing the context story hurts models and humans on the same tasks, scrambling words barely hurts the models (a sign of lexical shortcuts), and item-level accuracy correlates with humans for indirect speech, irony, maxims and metaphor.

Ideas

For Mova

34. Irony and figurative language (irony, metaphor, humour), 33-implicature-and-presupposition.md §2 (maxims), 31-speech-acts.md §5. Code: world/src/knowledge.rs (diagnosis by distractor type, perturbations), prag/src/irony.rs, prag/src/infer.rs. Measures: share of errors per distractor type; accuracy drop under context removal and scrambling.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/2023.acl-long.230 · arXiv 2212.06801

@inproceedings{hu2023fine,
  title = {{A fine-grained comparison of pragmatic language understanding in humans and language models}},
  author = {Hu, Jennifer and Floyd, Sammy and Jouravlev, Olessia and Fedorenko, Evelina and Gibson, Edward},
  booktitle = {Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2023},
  doi = {10.18653/v1/2023.acl-long.230},
  eprint = {2212.06801},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2023.acl-long.230}
}

References

99 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Felix K. Ameka and Marina Terkourafi. 2019. What if. . . ? Imagining non-Western perspectives on pragmatic theory and practice. Journal of Pragmatics, 145:72–82. find
  2. Jacob Andreas. 2022. Language Models as Agent Models. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 5769–5779, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics. find
  3. Clara Andrés-Roqueta and Napoleon Katsos. 2017. The Contribution of Grammar, Vocabulary and Theory of Mind in Pragmatic Language Competence in Children with Autistic Spectrum Disorders. Frontiers in Psychology, 8. find
  4. Ian Apperly. 2011. Mindreaders: The cognitive basis of "Theory of Mind". Psychology Press, New York. find
  5. Salvatore Attardo. 2000. Irony as relevant inappropriateness. Journal of Pragmatics, 32(6):793–826. find
  6. John L. Austin. 1975. How to do things with words. find
  7. Simge Aykan and Erhan Nalçacı. 2018. Assessing Theory of Mind by Humor: The Humor Comprehension and Appreciation Test (ToM-HCAT). Frontiers in Psychology, 9. find
  8. Simon Baron-Cohen, Alan M. Leslie, and Uta Frith. 1985. Does the autistic child have a “theory of mind”? Cognition, 21(1):37–46. find
  9. Emily M. Bender and Alexander Koller. 2020. Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5185–5198, Online. Association for Computational Linguistics. find
  10. Leon Bergen, Roger Levy, and Noah D. Goodman. 2016. Pragmatic reasoning through semantic inference. Semantics and Pragmatics, 9. find
  11. Anne Beyer, Sharid Loáiciga, and David Schlangen. 2021. Is Incoherence Surprising? Targeted Evaluation of Coherence Prediction from Language Models. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4164–4173, Online. Association for Computational Linguistics. find
  12. Luca Bischetti, Irene Ceccato, Serena Lecce, Elena Cav-allini, and Valentina Bambini. 2019. Pragmatics and theory of mind in older adults’ humor comprehension. Current Psychology. find
  13. Yonatan Bisk, Ari Holtzman, Jesse Thomason, Jacob Andreas, Yoshua Bengio, Joyce Chai, Mirella Lapata, Angeliki Lazaridou, Jonathan May, Aleksandr Nisnevich, Nicolas Pinto, and Joseph Turian. 2020. Experience Grounds Language. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8718–8735, Online. Association for Computational Linguistics. find
  14. Damián E. Blasi, Joseph Henrich, Evangelia Adamou, David Kemmerer, and Asifa Majid. 2022. Over-reliance on English hinders cognitive science. Trends in Cognitive Sciences, 26(12):1153–1170. Publisher: Elsevier. find
  15. W.C. Booth. 1974. A Rhetoric of Irony. Literature/Criticism - The University of Chicago Press. University of Chicago Press. find
  16. Susan E. Brennan, Alexia Galati, and Anna K. Kuhlen. 2010. Two Minds, One Dialog: Coordinating Speaking and Understanding. In Brian H. Ross, editor, Psychology of Learning and Motivation, volume 53, pages 301–344. Academic Press. find
  17. Penelope Brown and Stephen C. Levinson. 1987. Politeness: Some Universals in Language Usage. Cambridge University Press. find our card
  18. Penelope Brown, Mark A. Sicoli, and Olivier Le Guen. 2021. Cross-speaker repetition and epistemic stance in Tzeltal, Yucatec, and Zapotec conversations. Journal of Pragmatics, 183:256–272. find
  19. Stephen A. Butterfill and Ian A. Apperly. 2013. How to Construct a Minimal Theory of Mind. Mind & Language, 28(5):606–637. Publisher: John Wiley & Sons, Ltd. find
  20. Carl Camden, Michael T. Motley, and Ann Wilson. 1984. White lies in interpersonal communication: A taxonomy and preliminary investigation of social motivations. Western Journal of Speech Communication, 48(4):309–325. Publisher: Routledge. find
  21. Dallas Card, Peter Henderson, Urvashi Khandelwal, Robin Jia, Kyle Mahowald, and Dan Jurafsky. 2020. With Little Power Comes Great Responsibility. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9263–9274, Online. Association for Computational Linguistics. find
  22. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vin-odkumar Prabhakaran, Emily Reif, Nan Du, Ben Hutchinson, Reiner Pope, James Bradbury, Jacob Austin, Michael Isard, Guy Gur-Ari, Pengcheng Yin, Toju Duke, Anselm Levskaya, Sanjay Ghemawat, Sunipa Dev, Henryk Michalewski, Xavier Garcia, Vedant Misra, Kevin Robinson, Liam Fedus, Denny Zhou, Daphne Ippolito, David Luan, Hyeontaek Lim, Barret Zoph, Alexander Spiridonov, Ryan Sepassi, David Dohan, Shivani Agrawal, Mark Omernick, Andrew M. Dai, Thanumalayan Sankaranarayana Pillai, Marie Pellat, Aitor Lewkowycz, Erica Moreira, Rewon Child, Oleksandr Polozov, Katherine Lee, Zongwei Zhou, Xuezhi Wang, Brennan Saeta, Mark Diaz, Orhan Firat, Michele Catasta, Jason Wei, Kathy Meier-Hellstern, Douglas Eck, Jeff Dean, Slav Petrov, and Noah Fiedel. 2022. PaLM: Scaling Language Modeling with Pathways. arXiv preprint. find
  23. Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Sharan Narang, Gaurav Mishra, Adams Yu, Vincent Zhao, Yanping Huang, Andrew Dai, Hongkun Yu, Slav Petrov, Ed H. Chi, Jeff Dean, Jacob Devlin, Adam Roberts, Denny Zhou, Quoc V. Le, and Jason Wei. 2022. Scaling Instruction-Finetuned Language Models. arXiv preprint. find
  24. Ishita Dasgupta, Demi Guo, Andreas Stuhlmüller, Samuel J. Gershman, and Noah D. Goodman. 2018. Evaluating Compositionality in Sentence Embeddings. In Proceedings of the Cognitive Science Society. find
  25. Lambert Deckers and Philip Kizer. 1975. Humor and the Incongruity Hypothesis. The Journal of Psychology, 90(2):215–218. find
  26. Judith Degen. 2023. The Rational Speech Act Framework. Annual Review of Linguistics, 9(1):519–540. find
  27. Marie-Julie Demedardi, Claire Brechet, Edouard Gen-taz, and Catherine Monnier. 2021. Prosocial lying in children between 4 and 11 years of age: The role of emotional understanding and empathy. Journal of Experimental Child Psychology, 203:105045. find
  28. Christina Dideriksen, Morten H Christiansen, Mark Dingemanse, Malte Højmark-Bertelsen, Christer Johansson, Kristian Tylén, and Riccardo Fusaroli. 2022. Language specific constraints on conversation: Evidence from Danish and Norwegian. PsyArXiv preprint. find
  29. Judit Dombi, Tetyana Sydorenko, and Veronika Timpe-Laughlin. 2022. Common ground, cooperation, and recipient design in human-computer interactions. Journal of Pragmatics, 193:4–20. find
  30. Ivan Enrici, Bruno G. Bara, and Mauro Adenzato. 2019. Theory of Mind, pragmatics and the brain: Converging evidence for the role of intention processing as a core feature of human communication. Pragmatics & Cognition, 26(1):5–38. Sammy Floyd, Olessia Jouravlev, Zachary Mineroff, Leon Bergen, Evelina Fedorenko, and Edward Gibson. In prep. Deciphering the structure of pragmatics: A large-scale individual differences investigation. find
  31. Simeon Floyd. 2021. Conversation and Culture. Annual Review of Anthropology, 50(1):219–240. Publisher: Annual Reviews. find
  32. Michael C Frank, Andrés Goméz Emilsson, Benjamin Peloquin, Noah D. Goodman, and Christopher Potts. 2018. Rational speech act models of pragmatic reasoning in reference games. PsyArXiv preprint. find
  33. Michael C. Frank and Noah D. Goodman. 2012. Predicting Pragmatic Reasoning in Language Games. Science, 336(6084):998–998. find
  34. Michael Franke and Gerhard Jäger. 2016. Probabilistic pragmatics, or why Bayes’ rule is probably important for pragmatics. Zeitschrift für Sprachwissenschaft, 35(1):3–44. find
  35. Richard Futrell, Ethan Wilcox, Takashi Morita, Peng Qian, Miguel Ballesteros, and Roger Levy. 2019. Neural language models as psycholinguistic subjects: Representations of syntactic state. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 32–42, Minneapolis, Minnesota. Association for Computational Linguistics. find
  36. Raymond W. Gibbs. 1979. Contextual effects in understanding indirect requests. Discourse Processes, 2(1):1–10. Publisher: Routledge. find
  37. Herbert P. Grice. 1975. Logic and Conversation. In Peter Cole and Jerry L. Morgan, editors, Syntax and Semantics: Speech Acts, volume 3, pages 41–58. Academic Press. find
  38. Yuling Gu, Yao Fu, Valentina Pyatkin, Ian Magnus-son, Bhavana Dalvi Mishra, and Peter Clark. 2022. Just-DREAM-about-it: Figurative Language Understanding with DREAM-FLUTE. In Proceedings of the 3rd Workshop on Figurative Language Processing (FLP), pages 84–93, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics. find
  39. Francesca G.E. Happé. 1993. Communicative competence and theory of mind in autism: A test of relevance theory. Cognition, 48(2):101–119. find
  40. Cecilia Heyes. 2014. Submentalizing: I Am Not Really Reading Your Mind. Perspectives on Psychological Science, 9(2):131–143. find
  41. Laurence R. Horn. 1972. On the semantic properties of logical operators in English. PhD Thesis, University of California Los Angeles. find
  42. Yik Kwan Hsu and Him Cheung. 2013. Two mentalizing capacities and the understanding of two types of lie telling in children. Developmental Psychology, 49:1650–1659. find
  43. Jennifer Hu, Jon Gauthier, Peng Qian, Ethan Wilcox, and Roger Levy. 2020. A Systematic Assessment of Syntactic Generalization in Neural Language Models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1725–1744, Online. Association for Computational Linguistics. ↗
  44. Nir Jacoby and Evelina Fedorenko. 2020. Discourse-level comprehension engages medial frontal Theory of Mind brain regions even for expository texts. Language, Cognition and Neuroscience, 35(6):780–796. find
  45. Paloma Jeretic, Alex Warstadt, Suvrat Bhooshan, and Adina Williams. 2020. Are Natural Language Inference Models IMPPRESsive? Learning IMPlicature and PRESupposition. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8690–8705, Online. Association for Computational Linguistics. find our card
  46. Justine T. Kao, Leon Bergen, and Noah D. Goodman. 2014. Formalizing the Pragmatics of Metaphor Understanding. In Proceedings of the 36th Annual Meeting of the Cognitive Science Society. find
  47. Justine T. Kao and Noah D. Goodman. 2014. Let’s talk (ironically) about the weather: Modeling verbal irony. In Proceedings of the 36th Annual Meeting of the Cognitive Science Society. find
  48. Melissa Kline Struhl, Jeanne Gallée, Zuzanna Balewski, and Evelina Fedorenko. 2018. Understanding jokes draws most heavily on the Theory of Mind brain network. PsyArXiv preprint. find
  49. Michal Kosinski. 2023. Theory of Mind May Have Spontaneously Emerged in Large Language Models. arXiv preprint. find
  50. Elisa Kreiss, Fei Fang, Noah Goodman, and Christopher Potts. 2022. Concadia: Towards image-based text generation with a purpose. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 4667–4684, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics. find
  51. Tom McCoy, Ellie Pavlick, and Tal Linzen. 2019. Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3428–3448, Florence, Italy. Association for Computational Linguistics. find our card
  52. G. Lakoff and M. Johnson. 1980. Metaphors We Live By. University of Chicago Press. find
  53. Julian Michael. 2020. To Dissect an Octopus: Making Sense of the Form/Meaning Debate. find
  54. Matthew Le, Y-Lan Boureau, and Maximilian Nickel. 2019. Revisiting the Evaluation of Theory of Mind through Question Answering. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 5872–5877, Hong Kong, China. Association for Computational Linguistics. find
  55. Jarrod Moss and Christian D. Schunn. 2015. Comprehension through explanation as the interaction of the brain’s coherence and cognitive control networks. Frontiers in Human Neuroscience, 9. find
  56. Alan M. Leslie, Ori Friedman, and Tim P. German. 2004. Core mechanisms in ‘theory of mind’. Trends in Cognitive Sciences, 8(12):528–533. find
  57. Stephen Levinson. 2000. Presumptive meaning: The theory of generalized conversational implicature. MIT Press. find
  58. Elissa Li, Sebastian Schuster, and Judith Degen. 2021. Predicting Scalar Inferences From "Or" to "Not Both" Using Neural Sentence Encoders. In Proceedings of the Society for Computation in Linguistics, volume 4. find our card
  59. Jin Li. 2012. Cultural Foundations of Learning: East and West. Cambridge University Press, Cambridge. find
  60. Tal Linzen, Emmanuel Dupoux, and Yoav Goldberg. 2016. Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies. Transactions of the Association for Computational Linguistics, 4:521– 535. Place: Cambridge, MA Publisher: MIT Press. find
  61. Emmy Liu, Chenxuan Cui, Kenneth Zheng, and Graham Neubig. 2022. Testing the Ability of Language Models to Interpret Figurative Language. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4437–4452, Seattle, United States. Association for Computational Linguistics. find
  62. Eleonore Lumer and Hendrik Buschmeier. 2022. Modeling Social Influences on Indirectness in a Rational Speech Act Approach to Politeness. In Proceedings of the 44th Annual Conference of the Cognitive Science Society. find
  63. Kyle Mahowald, Anna A. Ivanova, Idan A. Blank, Nancy Kanwisher, Joshua B. Tenenbaum, and Evelina Fedorenko. 2023. Dissociating language and thought in large language models: A cognitive perspective. arXiv preprint. find
  64. R.A. Martin and T. Ford. 2018. The Psychology of Humor: An Integrative Approach. Academic Press. find
  65. Aida Nematzadeh, Kaylee Burns, Erin Grant, Alison Gopnik, and Tom Griffiths. 2018. Evaluating Theory of Mind in Question Answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2392–2400, Brussels, Belgium. Association for Computational Linguistics. find
  66. Yixin Nie, Yicheng Wang, and Mohit Bansal. 2019. Analyzing Compositionality-Sensitivity of NLI Models. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01):6867–6874. find
  67. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. 2022. Training language models to follow instructions with human feedback. find
  68. Christopher Potts. 2020. Is it possible for language models to achieve language understanding? find
  69. Christopher Potts, Daniel Lassiter, Roger Levy, and Michael C. Frank. 2016. Embedded Implicatures as Pragmatic Inferences under Compositional Lexical Uncertainty. Journal of Semantics, 33(4):755–802. find
  70. Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language Models are Unsupervised Multitask Learners. find
  71. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. Journal of Machine Learning Research, 21(140):1–67. find
  72. Paula Rubio-Fernandez. 2021. Pragmatic markers: the missing link between language and Theory of Mind. Synthese, 199(1):1125–1158. find
  73. Paula Rubio-Fernandez and Julian Jara-Ettinger. 2020. Incrementality and efficiency shape pragmatics across languages. Proceedings of the National Academy of Sciences, 117(24):13399–13404. find
  74. Laura Ruis, Akbir Khan, Stella Biderman, Sara Hooker, Tim Rocktäschel, and Edward Grefenstette. 2022. Large language models are not zero-shot communicators. arXiv preprint. find
  75. Michael Henry Tessler and Michael Franke. 2018. Not unreasonable: Carving vague dimensions with contraries and contradictions. In Proceedings of the 40th Annual Conference of the Cognitive Science Society. find
  76. Maarten Sap, Ronan Le Bras, Daniel Fried, and Yejin Choi. 2022. Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMs. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3762–3780, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics. find
  77. Xiaoyu Tong, Ekaterina Shutova, and Martha Lewis. 2021. Recent advances in neural metaphor processing: A linguistic, cognitive and social perspective. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4673–4686, Online. Association for Computational Linguistics. find
  78. Maarten Sap, Hannah Rashkin, Derek Chen, Ronan Le Bras, and Yejin Choi. 2019. Social IQa: Commonsense Reasoning about Social Interactions. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4463– 4473, Hong Kong, China. Association for Computational Linguistics. find
  79. Anna Trosborg, editor. 2010. Pragmatics across Languages and Cultures. De Gruyter Mouton. find
  80. Ayse Pinar Saygin and Ilyas Cicekli. 2002. Pragmatics in human-computer conversations. Journal of Pragmatics, 34(3):227–258. find
  81. Corrie Vendetti, Deepthi Kamawar, and Katherine E. Andrews. 2019. Theory of mind and preschoolers’ understanding of misdeed and politeness lies. Developmental Psychology, 55(4):823–834. find
  82. Sebastian Schuster, Yuxing Chen, and Judith Degen. 2020. Harnessing the linguistic signal to predict scalar inferences. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5387–5403, Online. Association for Computational Linguistics. find our card
  83. Yizhong Wang, Swaroop Mishra, Pegah Alipoormo-labashi, Yeganeh Kordi, Amirreza Mirzaei, Atharva Naik, Arjun Ashok, Arut Selvan Dhanasekaran, Anjana Arunkumar, David Stap, Eshaan Pathak, Giannis Karamanolakis, Haizhi Lai, Ishan Purohit, Ishani Mondal, Jacob Anderson, Kirby Kuznia, Krima Doshi, Kuntal Kumar Pal, Maitreya Patel, Mehrad Moradshahi, Mihir Parmar, Mirali Purohit, Neeraj Varshney, Phani Rohitha Kaza, Pulkit Verma, Ravsehaj Singh Puri, Rushang Karia, Savan Doshi, Shailaja Keyur Sampat, Siddhartha Mishra, Sujan Reddy A, Sumanta Patro, Tanay Dixit, and Xudong Shen. 2022. Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 5085–5109, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics. find
  84. John R. Searle. 1975. Indirect Speech Acts. In Speech Acts, pages 59–82. Brill, Leiden, The Netherlands. find
  85. Chirag Shah and Emily M. Bender. 2022. Situating Search. In ACM SIGIR Conference on Human Information Interaction and Retrieval, CHIIR ’22, pages 221–232, New York, NY, USA. Association for Computing Machinery. Event-place: Regensburg, Germany. find
  86. Nicola Spotorno, Eric Koun, Jérôme Prado, Jean-Baptiste Van Der Henst, and Ira A. Noveck. 2012. Neural evidence that utterance-processing entails mentalizing: The case of irony. NeuroImage, 63(1):25–39. find
  87. Kevin Stowe, Prasetya Utama, and Iryna Gurevych. 2022. IMPLI: Investigating NLI Models’ Performance on Figurative Language. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5375–5388, Dublin, Ireland. Association for Computational Linguistics. find our card
  88. Ross Taylor, Marcin Kardas, Guillem Cucurull, Thomas Scialom, Anthony Hartshorn, Elvis Saravia, Andrew Poulton, Viktor Kerkez, and Robert Stojnic. 2022. Galactica: A Large Language Model for Science. arXiv preprint. find
  89. Tomer Ullman. 2023. Large Language Models Fail on Trivial Alterations to Theory-of-Mind Tasks. arXiv preprint. find
  90. Thomas C. Veatch. 1998. A theory of humor. Humor, 11(2):161–216. find
  91. Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V. Le. 2022. Finetuned Language Models are Zero-Shot Learners. In International Conference on Learning Representations. find
  92. D. Wilson and D. Sperber. 2012. Meaning and Relevance. Cambridge University Press. find
  93. Deirdre Wilson and Dan Sperber. 1992. On verbal irony. Lingua, 87(1):53–76. find
  94. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020. Transformers: State-of-the-Art Natural Language Processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45, Online. Association for Computational Linguistics. find
  95. Erica J. Yoon, Michael Henry Tessler, Noah D. Goodman, and Michael C. Frank. 2016. Talking with tact: Polite language as a balance between informativity and kindness. In Proceedings of the Annual Meeting of the Cognitive Science Society. find
  96. Erica J. Yoon, Michael Henry Tessler, Noah D. Goodman, and Michael C. Frank. 2020. Polite Speech Emerges From Competing Social Goals. Open Mind, 4:71–87. find
  97. George Yule. 1996. Pragmatics, 1 edition. Oxford Introduction to Language Study. Oxford University Press. find
  98. Amir Zadeh, Michael Chan, Paul Pu Liang, Edmund Tong, and Louis-Philippe Morency. 2019. Social-IQ: A Question Answering Benchmark for Artificial Social Intelligence. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8799–8809. find
  99. Zilong Zheng, Shuwen Qiu, Lifeng Fan, Yixin Zhu, and Song-Chun Zhu. 2021. GRICE: A Grammar-based Dataset for Recovering Implicature and Conversational rEasoning. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 2074–2085, Online. Association for Computational Linguistics. find our card

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/hu-2023-fine-grained-comparison-pragmatic-language/