ISO-Standard Domain-Independent Dialogue Act Tagging for Conversational Agents

What it is about

No large corpus is annotated with ISO 24617-2, so the authors map five existing corpora (Switchboard, AMI, MapTask, VerbMobil, BT Oasis) onto a subset of it: the Task, Social Obligations and Feedback dimensions, with Inform, Commissive and Directive left unsplit because most source schemes lack the context to split them. Corpora are very unbalanced: Switchboard is 60% information-giving and under 1% action discussion, MapTask has over 30% action discussion. A linear SVM is first checked on Switchboard (42 tags): n-grams 71.7%, plus the previous act 74.6%, plus part-of-speech tags indexed by their position in the utterance 76.2% (plain POS gives nothing), plus word embeddings 76.7%, near the 77.0% neural state of the art. On the mapped data a binary model first decides the dimension, then a second model the function; the best feature set (n-grams, previous act, positional POS, positional dependency relations) reaches 67.1% on DialogBank, 74.3% on Alexa Prize user turns and 82.3% on socialbot logs. Training on all five corpora beats any single one; dropping Switchboard or AMI hurts most.

Ideas

• I1. Add positional part-of-speech slots to prag/src/feats.rs: UPOS of words 1–4 and of the last word, each as its own slot (today only u1). The paper's gain came from position-indexed tags, not from tags as a bag. Measure act and form macro-F1 in prag eval (cross-validated), with the shuffled-label control.
• I2. Same for dependency relations: the relation of the first two words and of the root's first dependent as positional slots (please/kindly as discourse/advmod, a vocative, an aux at the start). On ISO-mapped data these helped where plain trees did not.
• I3. Two steps for acts: first a small model or rule decides social obligation (thank, apologize, greet) vs the rest, using the cue list; then the 13-way act model runs on the rest. Rare SOM acts (thank 0.1%, apologize 0.1%) would no longer compete with assert in one softmax-like vote. Measure per-act recall for thank/apologize/greet and overall act accuracy.
• I4. When outside data is added for training (any licence-clean dialogue corpus), run the paper's leave-one-corpus-out ablation in prag eval, so a corpus that hurts tales is seen and dropped.

For Mova

31. Speech acts §7.2–7.3 (ISO mapping) and §9 (features of the act labeller). Code: prag/src/feats.rs (positional UPOS and deprel slots), prag/src/model.rs and prag/src/agree.rs (two-step act decision), prag/data/cues.tsv. Measures: act and form macro-F1 and per-act recall in prag eval.

Topics: Speech act theory

Cite

The paper itself, not this card. BibTeX from the library's record of the work. arXiv 1806.04327

@inproceedings{mezza2018iso,
  title = {{ISO-Standard Domain-Independent Dialogue Act Tagging for Conversational Agents}},
  author = {Mezza, Stefano and Cervone, Alessandra and Stepanov, Evgeny and Tortoreto, Giuliano and Riccardi, Giuseppe},
  booktitle = {Proceedings of the 27th International Conference on Computational Linguistics},
  year = {2018},
  eprint = {1806.04327},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1806.04327}
}

References

38 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Jan Alexandersson, Bianka Buschbeck-Wolf, Tsutomu Fujinami, Michael Kipp, Stephan Koch, Elisabeth Maier, Norbert Reithinger, Birte Schmitz, and Melanie Siegel. 1998. Dialogue acts in Verbmobil 2. DFKI Saarbrücken. find
2. Anne H Anderson, Miles Bader, Ellen Gurman Bard, Elizabeth Boyle, Gwyneth Doherty, Simon Garrod, Stephen Isard, Jacqueline Kowtko, Jan McAllister, Jim Miller, et al. 1991. The hcrc map task corpus. Language and speech, 34(4):351–366. find
3. John Langshaw Austin and JO Urmson. 1962. How to Do Things with Words. The William James Lectures Delivered at Harvard University in 1955.[Edited by James O. Urmson.]. Clarendon Press. find
4. Kevin K Bowden, Jiaqi Wu, Shereen Oraby, Amita Misra, and Marilyn Walker. 2017. Slugbot: An application of a novel and scalable open domain socialbot framework. Alexa Prize Proceedings. find
5. Harry Bunt, Jan Alexandersson, Jean Carletta, Jae-Woong Choe, Alex Chengyu Fang, Koiti Hasida, Kiyong Lee, Volha Petukhova, Andrei Popescu-Belis, Laurent Romary, Claudia Soria, and David Traum. 2010. Towards an iso standard for dialogue act annotation. Seventh conference on International Language Resources and Evaluation (LREC’10). find
6. Harry Bunt, Jan Alexandersson, Jae-Woong Choe, Alex Chengyu Fang, Koiti Hasida, Volha Petukhova, Andrei Popescu-Belis, and David R Traum. 2012. Iso 24617-2: A semantically-based standard for dialogue annotation. In LREC, pages 430–437. find
7. Harry Bunt, Volha Petukhova, Andrei Malchanau, Kars Wijnhoven, and Alex Chengyu Fang. 2016. The dialogbank. In LREC. find
8. Harry Bunt. 1999. Dynamic interpretation and dialogue theory. The structure of multimodal dialogue, 2:1–8. find
9. Harry Bunt. 2009. The dit++ taxonomy for functional dialogue markup. In AAMAS 2009 Workshop, Towards a Standard Markup Language for Embodied Dialogue Acts, pages 13–24. find
10. Susanne Burger, Karl Weilhammer, Florian Schiel, and Hans G Tillmann. 2000. Verbmobil data collection and annotation. In Verbmobil: Foundations of speech-to-speech translation, pages 537–549. Springer. find
11. Jean Carletta. 2006. Announcing the ami meeting corpus. The ELRA Newsletter 11(1), January-March, p. 3-5. find
12. Alessandra Cervone, Giuliano Tortoreto, Stefano Mezza, Enrico Gambi, and Giuseppe Riccardi. 2017. Roving mind: a balancing act between open–domain and engaging dialogue systems. In Alexa Prize Proceedings. find
13. Shammur Absar Chowdhury, Evgeny A Stepanov, and Giuseppe Riccardi. 2016. Transfer of corpus-specific dialogue act annotation to iso standard: Is it worth it? In LREC. find
14. Mark G. Core and James F. Allen. 1997. Coding dialogs with the damsl annotation scheme. In Proceedings of AAAI Fall Symposium on Communicative Action in Humans and Machines. find
15. Rong-En Fan, Kai-Wei Chang, Cho-Jui Hsieh, Xiang-Rui Wang, and Chih-Jen Lin. 2008. Liblinear: A library for large linear classification. J. Mach. Learn. Res., 9:1871–1874, June. find
16. Alex C. Fang, Jing Cao, Harry Bunt, and Xiaoyue Liu. 2012. The annotation of the Switchboard Corpus with the new ISO standard for dialogue act analysis. In Workshop on Interoperable Semantic Annotation. find
17. Hao Fang, Hao Cheng, Elizabeth Clark, Ariel Holtzman, Maarten Sap, Mary Ostendorf, Yejin Choi, and Noah A. Smith. 2017. Sounding board – university of washington’s alexa prize submission. In Alexa Prize Proceedings. find
18. John J Godfrey, Edward C Holliman, and Jane McDaniel. 1992. Switchboard: Telephone speech corpus for research and development. In Acoustics, Speech, and Signal Processing, 1992. ICASSP-92., 1992 IEEE International Conference on, volume 1, pages 517–520. IEEE. find
19. Narendra Gupta, Gokhan Tur, Dilek Hakkani-Tur, Srinivas Bangalore, Giuseppe Riccardi, and Mazin Gilbert. 2006. The at&t spoken language understanding system. IEEE Transactions on Audio, Speech, and Language Processing, 14(1):213–222. find
20. Matthew Henderson, Blaise Thomson, and Jason D Williams. 2014. The second dialog state tracking challenge. In SIGDIAL Conference, pages 263–272. find
21. Yangfeng Ji, Gholamreza Haffari, and Jacob Eisenstein. 2016. A latent variable recurrent neural network for discourse relation language models. In Proceedings of NAACL-HLT, pages 332–342. find
22. Dan Jurafsky. 1997. Switchboard swbd-damsl shallow-discourse-function. Annotation, Technical Report, 97-02, University of Colorado, CO, USA. find
23. Harshit Kumar, Arvind Agarwal, Riddhiman Dasgupta, Sachindra Joshi, and Arun Kumar. 2017. Dialogue act sequence labeling using hierarchical encoder with crf. arXiv preprint arXiv:1709.04250. ↗
24. Geoffrey Leech and Martin Weisser. 2003. Generic speech act annotation for task-oriented dialogues. In Procs. of the 2003 Corpus Linguistics Conference, pp. 441Y446. Centre for Computer Corpus Research on Language Technical Papers, Lancaster University. find
25. Quinn McNemar. 1947. Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika, 12(2):153–157, Jun. find
26. F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research. find
27. Volha Petukhova, Martin Gropp, Dietrich Klakow, Anna Schmidt, Gregor Eigner, Mario Topf, Stefan Srb, Petr Motlicek, Blaise Potard, John Dines, et al. 2014a. The dbox corpus collection of spoken human-human and human-machine dialogues. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC” 14), number EPFL-CONF-201766. European Language Resources Association (ELRA). find
28. Volha Petukhova, Andrei Malchanau, and Harry Bunt. 2014b. Interoperability of dialogue corpora through ISO 24617-2-based querying. In LREC. find
29. Patti J Price. 1990. Evaluation of spoken language systems: The atis domain. In Speech and Natural Language: Proceedings of a Workshop Held at Hidden Valley, Pennsylvania, June 24-27, 1990. find
30. Silvia Quarteroni and Giuseppe Riccardi. 2010. Classifying dialog acts in human-human and human-machine spoken conversations. In INTERSPEECH 2010, 11th Annual Conference of the International Speech Communication Association, pages 2514–2517. find
31. Silvia Quarteroni, Alexei V Ivanov, and Giuseppe Riccardi. 2011. Simultaneous dialog act segmentation and classification from human-human spoken conversations. In Acoustics, Speech and Signal Processing (ICASSP), 2011 IEEE International Conference on, pages 5596–5599. IEEE. find
32. Ashwin Ram, Rohit Prasad, Chandra Khatri, Anu Venkatesh, Raefer Gabriel, Qing Liu, Jeff Nunn, Behnam Hedayatnia, Ming Cheng, Ashish Nagar, Eric King, Kate Bland, Amanda Wartick, Yi Pan, Han Song, Sk Jayadevan, Gene Hwang, and Art Pettigrue. 2017. Conversational ai: The science behind the alexa prize. In Alexa Prize Proceedings. find
33. Elizabeth Shriberg, Raj Dhillon, Sonali Bhagat, Jeremy Ang, and Hannah Carvey. 2004. The icsi meeting recorder dialog act (mrda) corpus. Technical report, INTERNATIONAL COMPUTER SCIENCE INST BERKELEY CA. find
34. Andreas Stolcke, Klaus Ries, Noah Coccaro, Elizabeth Shriberg, Rebecca Bates, Daniel Jurafsky, Paul Taylor, Rachel Martin, Carol Van Ess-Dykema, and Marie Meteer. 2000. Dialogue act modeling for automatic tagging and recognition of conversational speech. Computational Linguistics, 26(3). find our card
35. David Traum. 1996. Conversational agency: The trains-93 dialogue manager. In In Susann LuperFoy, Anton Nijhholt, and Gert Veldhuijzen van Zanten, editors, Proceedings of Twente Workshop on Language Technology, TWLT-II. Citeseer. find
36. V.N. Vapnik. 1995. The Nature of Statistical Learning Theory. Springer. find
37. Puyang Xu and Ruhi Sarikaya. 2013. Convolutional neural network based triangular crf for joint intent detection and slot filling. In Automatic Speech Recognition and Understanding (ASRU), 2013 IEEE Workshop on, pages 78–83. IEEE. find
38. Xuesong Yang, Yun-Nung Chen, Dilek Hakkani-Tür, Paul Crook, Xiujun Li, Jianfeng Gao, and Li Deng. 2017. End-to-end joint learning of natural language understanding and dialogue manager. In Acoustics, Speech and Signal Processing (ICASSP), 2017 IEEE International Conference on, pages 5690–5694. IEEE. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/mezza-2018-iso-standard-domain-independent-dialogue/