Speech Act Patterns for Improving Generalizability of Explainable Politeness Detection Models

What it is about

Explainable politeness classifiers built on word features do well in the domain they were trained on and badly elsewhere. Here GPT-4 marks, inside each request of the Stanford Politeness Corpus, the words, phrases or sentences that perform one of ten acts derived from the Danescu-Niculescu-Mizil politeness strategies (gratitude, greeting, apology, direct question, direct start, hedging, positive and negative sentiment, request, deference); a manual check of 100 marked spans found 83% right, with hedging and direct questions confused most. The spans of each act are clustered by contextual embeddings with a label-aware decision tree, so «I am sorry» and «apologies» land together, and the cluster ids become features of a logistic regression or SVM. Trained on Wikipedia requests and tested on Stack Exchange, act clusters plus words reach 71.3% against 67.2% for the best earlier explainable model (fine-tuned BERT 75.5%); in-domain they do slightly worse than word clusters.

Ideas

I1. Ask the teacher for the span that carries the act, not only the sentence label: one extra column in the silver prompt (prag/src/silver.rs) with the cue words for act, hedge and polarity. Harvested spans that recur become candidate entries for prag/data/cues.tsv after a count threshold and a precision check against the silver, so the teacher's knowledge ends up as a deterministic cue list. Measure how many new cues pass and the act/hedge macro-F1 with them.
I2. Test Mova's 20 politeness strategy detectors in world/src/talk.rs across genres as the paper does (tales vs lessons vs Tatoeba), and add act-level features (the sentence's act, hedge, indirect) next to the word-level detectors. The paper's lesson is that act-level features carry across domains and word features do not.
I3. When auditing teacher labels, look first at hedge vs question, the confusion the paper found in GPT-4's spans (a question introduced by how could we know… marked as hedging).

For Mova

32. Politeness and hedging §6 (computational politeness; Mova's detectors) and 31-speech-acts.md §9. Code: prag/src/silver.rs (span column), prag/data/cues.tsv, world/src/talk.rs. Measures: precision of harvested cues on the silver, act/hedge macro-F1, cross-genre politeness accuracy.

Topics: Speech act theory

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.findings-acl.970

@inproceedings{aljanaideh2025speech,
  title = {{Speech Act Patterns for Improving Generalizability of Explainable Politeness Detection Models}},
  author = {Aljanaideh, Ahmad},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2025},
  year = {2025},
  doi = {10.18653/v1/2025.findings-acl.970},
  url = {https://doi.org/10.18653/v1/2025.findings-acl.970}
}

References

22 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. GPT-4 Technical Report. OpenAI. find
Sajid Ali, Tamer Abuhmed, Shaker El-Sappagh, Khan Muhammad, Jose M Alonso-Moral, Roberto Con-falonieri, Riccardo Guidotti, Javier Del Ser, Natalia Díaz-Rodríguez, and Francisco Herrera. 2023. Explainable Artificial Intelligence (XAI): What we Know and What is Left to Attain Trustworthy Artificial Intelligence. Information fusion, 99:101805. ↗
Ahmad Aljanaideh, Eric Fosler-Lussier, and Marie-Catherine de Marneffe. 2020. Contextualized Embeddings for Enriching Linguistic Analyses on Politeness. In Proceedings of the 28th International Conference on Computational Linguistics, pages 2181– 2190. ↗ our card
John Langshaw Austin. 1975. How to do Things With Words. Harvard University Press. find
Tom B Brown. 2020. Language Models are Few-shot Learners. Advances in Neural Information Processing Systems. find
Feng Chen, Liqin Wang, Julie Hong, Jiaqi Jiang, and Li Zhou. 2024. Unmasking Bias in Artificial Intelligence: A Systematic Review of Bias Detection and Mitigation Strategies in Electronic Health Record-based Models. Journal of the American Medical Informatics Association, 31(5):1172–1183. ↗
Tobias Clement, Nils Kemmerzell, Mohamed Abdelaal, and Michael Amberg. 2023. XAIR: A Systematic Metareview of Explainable AI (XAI) Aligned to the Software Development Process. Machine Learning and Knowledge Extraction, 5(1):78–108. ↗
Cristian Danescu-Niculescu-Mizil, Moritz Sudhof, Dan Jurafsky, Jure Leskovec, and Christopher Potts. 2013. A Computational Approach to Politeness with Application to Social Factors. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics. find
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of The 2019 Conference of the North American Chapter of the Association for Computational Linguistics. ↗
Vikas Hassija, Vinay Chamola, Atmesh Mahapatra, Ab-hinandan Singal, Divyansh Goel, Kaizhu Huang, Simone Scardapane, Indro Spinelli, Mufti Mahmud, and Amir Hussain. 2024. Interpreting Black-Box Models: A Review on Explainable Artificial Intelligence. Cognitive Computation, 16(1):45–74. ↗
Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2020. Deberta: Decoding-enhanced BERT with Disentangled Attention. arXiv preprint arXiv:2006.03654. ↗
Yeo Jin Kim, Halim Acosta, Wookhee Min, Jonathan Rowe, Bradford Mott, Snigdha Chaturvedi, and James Lester. 2024. Dual process masking for dialogue act recognition. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 15270–15283. ↗
Ruizhe Li, Chenghua Lin, Matthew Collinson, Xiao Li, and Guanyi Chen. 2018. A dual-attention hierarchical recurrent neural network for dialogue act classification. Conference on Computational Natural Language Learning (CoNLL. ↗
Jionghao Lin, Wei Tan, Lan Du, Wray Buntine, David Lang, Dragan Gašević, and Guanliang Chen. 2023. Enhancing educational dialogue act classification with discourse context and sample informativeness. IEEE Transactions on Learning Technologies. ↗
Pantelis Linardatos, Vasilis Papastefanopoulos, and Sotiris Kotsiantis. 2020. Explainable AI: A Review of Machine Learning Interpretability Methods. Entropy, 23(1):18. ↗
Yinhan Liu. 2019. Roberta: A Robustly Optimized BERT Pretraining Approach. arXiv preprint arXiv:1907.11692. ↗
Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. 2011. Scikit-learn: Machine Learning in Python. the Journal of machine Learning research, 12:2825–2830. find
Alec Radford. 2018. Improving Language Understanding by Generative Pre-training. find
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9. find
Vipul Raheja and Joel Tetreault. 2019. Dialogue Act Classification with Context-aware Self-attention. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). ↗
Eugénio Ribeiro, Ricardo Ribeiro, and David Martins de Matos. 2019. Deep dialog act recognition using multiple token, segment, and context information representations. Journal of Artificial Intelligence Research, 66:861–899. ↗
Andreas Stolcke, Klaus Ries, Noah Coccaro, Elizabeth Shriberg, Rebecca Bates, Daniel Jurafsky, Paul Taylor, Rachel Martin, Carol Van Ess-Dykema, and Marie Meteer. 2000. Dialogue act modeling for automatic tagging and recognition of conversational speech. Computational linguistics, 26(3):339–373. ↗ our card

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/aljanaideh-2025-speech-act-patterns-improving-generalizability/