"You might think about slightly revising the title": Identifying Hedges in Peer-tutoring Interactions

What it is about

The paper classifies clauses of teenage peer-tutoring dialogue (about 24,000 clauses, more than 90% not hedged; data under NDA) into three hedge types — subjectivizers («I think», «I would»), propositional hedges or approximators («just», «kind of», «or whatever») and apologizers used as softeners («oh sorry, that's nine») — plus «not hedged». The authors first fix inconsistent annotation of just and would, then compare a regular-expression rule classifier built from Brown & Levinson, Fraser and Rowland, gradient-boosted trees over interpretable features (the rule label weighted by the rule's precision, lemma uni/bigrams, POS n-grams, LIWC, tutoring moves, nonverbal cues), MLP/LSTM with sentence embeddings, an attention CNN and fine-tuned BERT. Trees over the rule label plus n-grams win (weighted F1 79.0 on the three hedge classes vs 70.6 for BERT and 67.6 for the rules alone); adding POS and LIWC barely helps, nonverbal and tutoring-move features not at all. SHAP shows the top features per class: the rule label, «I», «would», «just», «sorry», and «oh» before «sorry».

Ideas

• I1. Split Mova's binary hedge into the three functional classes (subjectivizer / approximator / softening apology) in prag/src/schema.rs. Chapter 32 already notes that the label has no shield–approximator distinction. The cue classes in prag/data/cues.tsv map almost directly: I think, I guess → shield; kind of, sort of, a bit, almost → approximator; sorry inside a clause that carries a proposition → softener.
• I2. Use the paper's hybrid recipe in the SLM: feed the rule-based hedge decision, weighted by that rule's measured precision on the silver, as one slot into the perceptron and IGTree (prag/src/feats.rs), next to the raw word slots. Measure hedge macro-F1 in prag eval with and without the slot.
• I3. A standalone «Oh sorry» with no proposition is not a hedge (it is an apology or repair); only a «sorry» attached to content softens it. Encode this as a rule for act=apologize versus hedge=1, and check the silver for both cases.
• I4. would and just are the hardest cues; the rule classifier lost precision when it added would. Before adding them to cues.tsv, measure their per-cue precision against the silver hedge label.

For Mova

32. Politeness and hedging §7 (approximators vs shields) and §9–10; prag/src/schema.rs (hedge type), prag/src/feats.rs, prag/data/cues.tsv; measure: hedge accuracy and macro-F1 in prag eval, with the shuffled-label control.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/2022.acl-long.153 · arXiv 2306.14911

@inproceedings{raphalen2022you,
  title = {{"You might think about slightly revising the title": Identifying Hedges in Peer-tutoring Interactions}},
  author = {Raphalen, Yann and Clavel, Chloé and Cassell, Justine},
  booktitle = {Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2022},
  doi = {10.18653/v1/2022.acl-long.153},
  eprint = {2306.14911},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2022.acl-long.153}
}

References

39 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ken Hyland. 1998. Hedging in scientific research articles, volume 54. John Benjamins Publishing. find our card
2. Heike Adel and Hinrich Schütze. 2017. Exploring different dimensions of attention for uncertainty detection. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 22–34, Valencia, Spain. Association for Computational Linguistics. find
3. Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu. 2017. Lightgbm: A highly efficient gradient boosting decision tree. Advances in neural information processing systems, 30:3146–3154. find
4. Shoshana Blum-Kulka. 1987. Indirectness and politeness in requests: Same or different? Journal of pragmatics, 11(2):131–146. find
5. Penelope Brown and Stephen C Levinson. 1987. Politeness: Some universals in language usage, volume 4. Cambridge university press. find
6. Klaus Krippendorff. 2004. Reliability in content analysis: Some common misconceptions and recommendations. Human communication research, 30(3):411– 433. find
7. George Lakoff. 1975. Hedges: A study in meaning criteria and the logic of fuzzy concepts. In Contemporary research in philosophical logic and linguistic semantics, pages 221–271. Springer. find our card
8. Matthew Leach. 2005. Rapport: A key to treatment success. Complementary therapies in clinical practice, 11:262–5. find
9. Ilya Loshchilov and Frank Hutter. 2018. Decoupled weight decay regularization. In International Conference on Learning Representations. find
10. Scott M Lundberg, Gabriel G Erion, and Su-In Lee. 2018. Consistent individualized feature attribution for tree ensembles. arXiv preprint arXiv:1802.03888. ↗
11. Scott M Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. In Proceedings of the 31st international conference on neural information processing systems, pages 4768–4777. find
12. Howard Lune and Bruce L Berg. 2017. Qualitative research methods for the social sciences. Pearson. find
13. Michael Madaio, Justine Cassell, and Amy Ogan. 2017. The impact of peer tutors’ use of indirect feedback and instructions. Philadelphia, PA: International Society of the Learning Sciences. find
14. Ben Medlock and Ted Briscoe. 2007. Weakly supervised learning for hedge classification in scientific literature. In Proceedings of the 45th annual meeting of the association of computational linguistics, pages 992–999. find
15. Christoph Molnar. 2020. Interpretable machine learning. Lulu. com. find
16. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32:8026– 8037. find
17. James W Pennebaker, Ryan L Boyd, Kayla Jordan, and Kate Blackburn. 2015. The development and psychometric properties of liwc2015. Technical report. find
18. Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532–1543. find
19. Ellen F Prince, Joel Frader, Charles Bosk, et al. 1982. On hedging in physician-physician discourse. Linguistics and the Professions, 8(1):83–97. find
20. Anna Prokofieva and Julia Hirschberg. 2014. Hedging and speaker commitment. In 5th Intl. Workshop on Emotion, Social Signals, Sentiment & Linked Open Data, Reykjavik, Iceland. find
21. Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992. find
22. Oscar J Romero, Ran Zhao, and Justine Cassell. 2017. Cognitive-inspired conversational-strategy reasoner for socially-aware agents. In IJCAI, pages 3807– 3813. find
23. Tim Rowland. 2007. ‘well maybe not exactly, but it’s around fifty basically?’: Vague language in mathematics classrooms. In Vague language explored, pages 79–96. Springer. find
24. Sascha Schneider, Steve Nebel, Simon Pradel, and Günter Daniel Rey. 2015. Mind your ps and qs! how polite instructions affect learning with multimedia. Computers in Human Behavior, 51:546–555. find
25. Tanmay Sinha and Justine Cassell. 2015. We click, we align, we learn: Impact of influence and convergence processes on student learning and rapport building. In Proceedings of the 1st Workshop on Modeling INTERPERsonal SynchrONy And InfLuence, INTER-PERSONAL ’15, page 13–20, New York, NY, USA. Association for Computing Machinery. find
26. Helen Spencer-Oatey. 2005. (im)politeness, face and perceptions of rapport: Unpackaging their bases and interrelationships. 1(1):95–119. find
27. György Szarvas, Veronika Vincze, Richárd Farkas, György Móra, and Iryna Gurevych. 2012. Cross-genre and cross-domain detection of semantic uncertainty. Computational Linguistics, 38(2):335–367. find
28. Buzhou Tang, Xiaolong Wang, Xuan Wang, Bo Yuan, and Shixi Fan. 2010. A cascade method for detecting hedges and their scope in natural language text. In Proceedings of the Fourteenth Conference on Computational Natural Language Learning–Shared Task, pages 13–17. find
29. Morgan Ulinski, Seth Benjamin, and Julia Hirschberg. 2018. Using hedge detection to improve committed belief tagging. In Proceedings of the Workshop on Computational Semantics beyond Events and Roles, pages 1–5. find our card
30. Teppo Varttala. 1999. Remarks on the communicative functions of hedging in popular scientific and specialist research articles on medicine. English for specific purposes, 18(2):177–200. find
31. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems, pages 5998–6008. find
32. Erik Velldal. 2011. Predicting speculation: a simple disambiguation approach to hedge detection in biomedical literature. Journal of Biomedical Semantics, 2(5):1–14. find
33. Veronika Vincze. 2014. Uncertainty detection in natural language texts. PhD, University of Szeged, page 141. find
34. Ning Wang, W Lewis Johnson, Richard E Mayer, Paola Rizzo, Erin Shaw, and Heather Collins. 2008. The politeness effect: Pedagogical agents and learning outcomes. International journal of human-computer studies, 66(2):98–112. find
35. Shawn Warner-Garcia. 2014. Laughing when nothing’s funny: The pragmatic use of coping laughter in the negotiation of conversational disagreement. Pragmatics, 24(1):157–180. find
36. Timothy Williamson. 2002. Vagueness. Routledge. find
37. Ran Zhao, Alexandros Papangelis, and Justine Cassell. 2014. Towards a dyadic computational model of rapport management for human-virtual agent interaction. In International Conference on Intelligent Virtual Agents, pages 514–527. Springer. find
38. Ran Zhao, Tanmay Sinha, Alan W Black, and Justine Cassell. 2016a. Automatic recognition of conversational strategies in the service of a socially-aware dialog system. In Proceedings of the 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 381–392. find
39. Ran Zhao, Tanmay Sinha, Alan W Black, and Justine Cassell. 2016b. Socially-aware virtual agents: Automatically assessing dyadic rapport from temporal patterns of behavior. In International conference on intelligent virtual agents, pages 218–233. Springer. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/raphalen-2022-you-might-think-about-slightly/