Comparing human and LLM politeness strategies in free production

What it is about

The study replicates Yoon et al. (2020) with LLMs. A speaker gives feedback on a performance rated 0–3 hearts, with the goal of being informative, kind, or both, and chooses among «was / wasn't» + terrible, bad, good, amazing. Models of 70B parameters and more reproduce the human pattern (Spearman up to 0.75): with both goals, a poor performance gets «wasn't terrible» rather than «was bad». Models choose more categorically than humans and stay soft even when the goal is purely informative. In open-ended production (1,248 responses), evaluators preferred LLM answers (66% of trials). Annotation with Brown & Levinson's four super-strategies (positive, negative, off-record, bald on-record) plus the Danescu-Niculescu-Mizil markers shows that models overuse negative politeness (hedges, questions, distance) where humans use positive politeness. Negation as softening appears in only 6.6% of free evaluative answers.

Ideas

• I1. Fix how polarity treats negated evaluatives. «It wasn't terrible» is not neutral and not positive: it is a softened low evaluation (litotes). In prag/src/feats.rs and the polarity rule planned in the prag README («negation scope over the evaluative word»), map a negated strong negative to neg with hedge=1, and a negated strong positive («wasn't amazing») to a mild neg. Measure polarity macro-F1 in prag eval on silver sentences with neg × eval.
• I2. Add the social utility of Yoon et al. to the exact RSA module (prag/src/rsa.rs). The speaker's utility becomes φ·informational + (1−φ)·social value of the listener's inferred state, with φ a small rational number. Then «wasn't terrible» given 0 hearts can be derived and explained by the L0/S1/L1 tables. Negative control: with φ = 1 the litotes is never chosen.
• I3. Group the 20 politeness detectors of world/src/talk.rs into the four Brown & Levinson super-strategies and report the dominant one per utterance. This gives the dialogue layer one label to explain («negative politeness: hedge + question») and gives chapter 32 a count table over the tales' dialogue.

For Mova

32. Politeness and hedging §1.2 and §6, 33. Implicature and presupposition §9 and §12 (RSA; litotes); prag/src/rsa.rs, prag/src/feats.rs, world/src/talk.rs; measures: polarity macro-F1 in prag eval, RSA unit tests with hand-computed fractions.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.emnlp-main.820 · arXiv 2506.09391

@inproceedings{zhao2025comparing,
  title = {{Comparing human and LLM politeness strategies in free production}},
  author = {Zhao, Haoran and Hawkins, Robert D.},
  booktitle = {Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
  doi = {10.18653/v1/2025.emnlp-main.820},
  eprint = {2506.09391},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2025.emnlp-main.820}
}

References

41 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Theo Araujo and Nadine Bol. 2024. From speaking like a person to being personal: The effects of personalized, regular interactions with conversational agents. Computers in Human Behavior: Artificial Humans, 2(1):100030. find
2. Malika Aubakirova and Mohit Bansal. 2016. Interpreting neural networks to improve politeness comprehension. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2035–2041. find
3. Holly P Branigan, Martin J Pickering, Jamie Pearson, and Janet F McLean. 2010. Linguistic alignment between people and computers. Journal of pragmatics, 42(9):2355–2368. find
4. Penelope Brown and Stephen C Levinson. 1987. Politeness: Some universals in language usage. Cambridge university press. find our card
5. Fausto Carcassi and Michael Franke. 2023. How to handle the truth: A model of politeness as strategic truth-stretching. In Proceedings of the Annual Meeting of the Cognitive Science Society. find
6. Cristian Danescu-Niculescu-Mizil, Moritz Sudhof, Dan Jurafsky, Jure Leskovec, and Christopher Potts. 2013. A computational approach to politeness with application to social factors. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 250–259. find
7. Liye Fu, Susan Fussell, and Cristian Danescu-Niculescu-Mizil. 2020. Facilitating the communication of politeness through fine-grained paraphrasing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 5127–5140. find our card
8. Erving Goffman. 1967. Interaction ritual: Essays in face-to-face behavior. Routledge. find
9. Nicole Gotzner and Gregory Scontras. 2024. On the role of loopholes in polite communication: Linking subjectivity and pragmatic inference. Open Mind, 8:500–510. find
10. Herbert Paul Grice. 1975. Logic and conversation. In & J. L. Morgan. P. Cole, editor, Syntax and Semantics, Vol. 3, Speech Acts, pages 41–58. find
11. Joy He-Yueya, Wanjing Anya Ma, Kanishk Gandhi, Benjamin W Domingue, Emma Brunskill, and Noah D Goodman. 2024. Psychometric alignment: Capturing human knowledge distributions via language models. arXiv preprint arXiv:2407.15645. ↗
12. Beverly Hill, Sachiko Ide, Shoko Ikuta, Akiko Kawasaki, and Tsunao Ogino. 1986. Universals of linguistic politeness: Quantitative evidence from japanese and american english. Journal of pragmatics, 10(3):347–371. find
13. Jennifer Hu, Sammy Floyd, Olessia Jouravlev, Evelina Fedorenko, and Edward Gibson. 2022. A fine-grained comparison of pragmatic language understanding in humans and language models. arXiv preprint arXiv:2212.06801. ↗ our card
14. Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu. 2025. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Transactions on Information Systems, 43(2):1–55. find
15. Sachiko Ide. 1989. Formal forms and discernment: Two neglected aspects of universals of linguistic politeness. find
16. Mingyue Jian and Siddharth Narayanaswamy. 2024. Are LLMs good pragmatic speakers? In NeurIPS Workshop on Behavioral Machine Learning. find
17. Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guillaume Lample, Lélio Renard Lavaud, Lucile Saulnier, Marie-Anne Lachaux, Pierre Stock, Sandeep Subramanian, Sophia Yang, and 7 others. 2024. Mixtral of experts. arXiv preprint arXiv:2401.04088. ↗
18. Geoffrey Leech. 2014. The pragmatics of politeness. find
19. Benjamin Lipkin, Lionel Wong, Gabriel Grand, and Josh Tenenbaum. 2023. Evaluating statistical language models as pragmatic reasoners. In Proceedings of the Annual Meeting of the Cognitive Science Society. find
20. Ryan Liu, Theodore Sumers, Ishita Dasgupta, and Thomas L Griffiths. 2024. How do large language models navigate conflicts between honesty and helpfulness? In Forty-first International Conference on Machine Learning. find
21. Miriam A Locher. 2013. Relational work and interpersonal pragmatics. Journal of Pragmatics, 58:145– 149. find
22. Eleonore Lumer and Hendrik Buschmeier. 2022. Modeling social influences on indirectness in a rational speech act approach to politeness. In Proceedings of the Annual Meeting of the Cognitive Science Society, volume 44. find
23. Aman Madaan, Amrith Setlur, Tanmay Parekh, Barnabas Poczos, Graham Neubig, Yiming Yang, Ruslan Salakhutdinov, Alan W Black, and Shrimai Prabhumoye. 2020. Politeness transfer: A tag and generate approach. arXiv preprint arXiv:2004.14257. ↗
24. Sourabrata Mukherjee, Vojtěch Hudeček, and Ondřej Dušek. 2023. Polite chatbot: A text style transfer application. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics: Student Research Workshop, pages 87–93. find
25. Sonia Krishna Murthy, Tomer Ullman, and Jennifer Hu. 2025. One fish, two fish, but not the whole sea: Alignment reduces language models’ conceptual diversity. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 11241– 11258, Albuquerque, New Mexico. Association for Computational Linguistics. find
26. Vera Neplenbroek, Arianna Bisazza, and Raquel Fernández. 2025. Reading between the prompts: How stereotypes shape llm’s implicit personalization. Preprint, arXiv:2505.16467. ↗
27. Tong Niu and Mohit Bansal. 2018. Polite dialogue generation without parallel data. Transactions of the Association for Computational Linguistics, 6:373– 389. find
28. Steven Pinker, Martin A Nowak, and James J Lee. 2008. The logic of indirect speech. Proceedings of the National Academy of sciences, 105(3):833–838. find
29. Priyanshu Priya, Mauajama Firdaus, and Asif Ekbal. 2024. Computational politeness in natural language processing: A survey. ACM Computing Surveys, 56(9):1–42. find
30. Jakub Prochazka, Martin Ovcari, and Michal Durinik. 2020. Sandwich feedback: The empirical evidence of its effectiveness. Learning and Motivation, 71:101649. find
31. Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992, Hong Kong, China. Association for Computational Linguistics. find
32. Laura Ruis, Akbir Khan, Stella Biderman, Sara Hooker, Tim Rocktäschel, and Edward Grefenstette. 2024. The goldilocks of pragmatic understanding: Fine-tuning strategy matters for implicature resolution by llms. Advances in Neural Information Processing Systems, 36. find
33. Helen Spencer-Oatey. 2011. Conceptualising ‘the relational’in pragmatics: Insights from metapragmatic emotion and (im) politeness comments. Journal of Pragmatics, 43(14):3565–3578. find
34. Zhen Tan, Alimohammad Beigi, Song Wang, Ruocheng Guo, Amrita Bhattacharjee, Bohan Jiang, Mansooreh Karami, Jundong Li, Lu Cheng, and Huan Liu. 2024. Large language models for data annotation: A survey. arXiv e-prints, pages arXiv–2402. find
35. Richard J Watts. 2003. Politeness. Cambridge University Press. find
36. Adam Waytz, Joy Heafner, and Nicholas Epley. 2014. The mind in the machine: Anthropomorphism increases trust in an autonomous vehicle. Journal of experimental social psychology, 52:113–117. find
37. Anna Wierzbicka. 2003. Cross-cultural pragmatics. Walter de Gruyter Inc. find
38. Ziwei Xu, Sanjay Jain, and Mohan Kankanhalli. 2025. Hallucination is inevitable: An innate limitation of large language models. Preprint, arXiv:2401.11817. ↗
39. Akhila Yerukola, Saujas Vaduguru, Daniel Fried, and Maarten Sap. 2024. Is the pope catholic? yes, the pope is catholic. generative evaluation of non-literal intent resolution in LLMs. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 265–275, Bangkok, Thailand. Association for Computational Linguistics. find
40. Erica J. Yoon, Michael Henry Tessler, Noah D. Goodman, and Michael C. Frank. 2020. Polite speech emerges from competing social goals. Open Mind, 4:71–87. find
41. Qi Zhou, Bin Li, Lei Han, and Min Jou. 2023. Talking to a bot or a wall? how chatbots vs. human agents affect anticipated communication quality. Computers in Human Behavior, 143:107674. Experiment 1 Methods find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/zhao-2025-comparing-human-llm-politeness-strategies/