Unravelling Indirect Answers to Wh-Questions: Corpus Construction, Analysis, and Generation

What it is about

Indirect answers to wh-questions are much rarer and harder to spot than those to yes/no questions: from 2,040 annotated BNC wh-questions only 35 had one, and the final corpus has 458 pairs, mostly from COCA (how 214, what 75, why 63, where 32, when 31, who 29, which 14). The paper uses Ginzburg's response-space taxonomy (direct answer, indirect answer, dependent question, clarification, acknowledgement, evasions such as ignoring, changing the topic or «difficult to answer»), with a formal definition: an indirect answer is not itself a direct answer but, together with a shared bridging proposition, entails one. Agreement on whether a reply is indirect is low (pairwise kappa 0.22–0.44 for four annotators, 0.60 between two experts). A pilot on 141 pairs labels the knowledge needed to recover the direct answer: basic linguistic knowledge 30.5% («It's inspected daily» answers «When was it last inspected?»), common sense 24%, idioms and figurative language 14%, shared discourse knowledge 10%, speaker goals 9%, beliefs 8.5%. A DialoGPT fine-tuned on the 458 pairs produced an indirect answer in only 1 of 20 test cases.

Ideas

I1. Give Mova's reply reading a direct/indirect test that needs no learning: a reply to a wh-question is direct if its frame fills the asked role (Ask::Role(slot)) with an entity or value of the right type, indirect if it shares a frame or entity with the question but leaves that role empty, and an evasion otherwise («I don't know», a counter-question). Print which role was or was not filled. Use it in world/src/talk.rs and on quoted replies in tales.
I2. The «basic linguistic knowledge» cases are the cheap ones: a frequency or habit adverb answers when (daily, every morning, always), a manner adverb or by-phrase answers how, a purpose or because clause answers why. Check that the frame reader fills when:/manner/cause from these forms; count with world knowledge diagnose how many course questions of these types fail at «nothing remembered matched».
I3. Given the low human agreement, do not spend teacher calls on labelling indirectness of wh-answers; spend them on yes/no answers (see louis-2020-i-d-rather-just-go), where agreement is high.

For Mova

38. Dialogue, discourse markers and reported speech §9 (questions and answers in dialogue), 26-questions.md (wh-questions). Code: prag/src/frames.rs (Ask::Role, filled roles), world/src/talk.rs, world/src/knowledge.rs (diagnosis). Measures: share of course wh-questions failing for an unfilled when/manner/cause role before and after; precision of the direct/indirect/evasion test on a hand-checked set of tale replies.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/2023.sigdial-1.30

@inproceedings{yusupujiang2023unravelling,
  title = {{Unravelling Indirect Answers to Wh-Questions: Corpus Construction, Analysis, and Generation}},
  author = {Yusupujiang, Zulipiye and Ginzburg, Jonathan},
  booktitle = {Proceedings of the 24th Annual Meeting of the Special Interest Group on Discourse and Dialogue},
  year = {2023},
  doi = {10.18653/v1/2023.sigdial-1.30},
  url = {https://doi.org/10.18653/v1/2023.sigdial-1.30}
}

References

14 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

Joseph L Fleiss. 1971. Measuring nominal scale agreement among many raters. Psychological bulletin, 76(5):378. find
Penelope Brown, Stephen C Levinson, and Stephen C Levinson. 1987. Politeness: Some universals in language usage, volume 4. Cambridge university press. find
Joseph L Fleiss, Bruce Levin, and Myunghee Cho Paik. 2003. Statistical Methods for Rates and Proportions. John Wiley & Sons. find
Lou Burnard, editor. 2007. Reference guide for the British National Corpus (XML Edition). Oxford University Computing Services on behalf of the BNC Consortium. Acess 20.03.2017. find
Barbara A Fox and Sandra A Thompson. 2010. Responses to Wh-Questions in English Conversation. Research on Language and Social Interaction, 43(2):133–156. find
Jonathan Ginzburg. 2012. The Interactive Stance: Meaning for Conversation. Oxford University Press, Oxford. find
Jonathan Ginzburg and Ivan A. Sag. 2000. Interrogative Investigations: the form, meaning and use of English Interrogatives. Number 123 in CSLI Lecture Notes. CSLI Publications, Stanford: California. find
Jonathan Ginzburg, Zulipiye Yusupujiang, Chuyuan Li, Kexin Ren, Aleksandra Kucharska, and Pawel Lup-kowski. 2022. Characterizing the response space of questions: data and theory. Dialogue & Discourse, 13(2):79–132. find
Jonathan Ginzburg, Zulipiye Yusupujiang, Chuyuan Li, Kexin Ren, and Paweł Łupkowski. 2019. Characterizing the Response Space of Questions: a Corpus Study for English and Polish. In Proceedings of the 20th Annual SIGdial Meeting on Discourse and Dialogue, pages 320–330, Stockholm, Sweden. Association for Computational Linguistics. find
Nancy Green and Sandra Carberry. 1994a. Generating indirect answers to yes-no questions. In Proceedings of the Seventh International Workshop on Natural Language Generation. find
Svartvik, Jan, editor. 1990. The London–Lund corpus of spoken English : Description and research, volume 82 of Lund Studies in English. Lund University Press. Book Editor. find
Junya Takayama, Tomoyuki Kajiwara, and Yuki Arase. 2021. DIRECT: Direct and Indirect Responses in Conversational Text Corpus. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 1980–1989, Punta Cana, Dominican Republic. Association for Computational Linguistics. find
Zulipiye Yusupujiang, Alafate Abulimiti, and Jonathan Ginzburg. 2022. Classifying the response space of questions: A machine learning approach. In SemDial 2022-26th Workshop on the Semantics and Pragmatics of Dialogue, pages 59–69. find
Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, and Bill Dolan. 2020. DIALOGPT : Large-Scale Generative Pre-training for Conversational Response Generation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 270–278, Online. Association for Computational Linguistics. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/yusupujiang-2023-unravelling-indirect-answers-wh-questions/