Dependency Parsing for Spoken Dialog Systems

What it is about

The authors annotate 1,500 automatic-speech-recognition transcripts of users talking to an open-domain chatbot, using SCUD, an extension of UD 2.0 for spoken human-machine dialogue. SCUD adds the discourse relation for non-syntactic markers (such as a pragmatic "like"), reparandum and flat for disfluencies and repetitions, goeswith for words split by ASR, node insertion for omitted words, and a new label, preterm, for user turns that the system cut short (about 5% of turns). Two annotators agreed on 86.4% of unlabeled and 80.5% of labeled dependencies. A biaffine parser trained on UD-EWT and then fine-tuned on ConvBank reaches 85.05 UAS and 77.82 LAS on 500 held-out sentences; UD-EWT alone reaches 77.41 and 65.85, and ConvBank alone with 1,000 utterances reaches 75.65 and 58.29.

Ideas

• I1. Add the SCUD labels that Mova lacks (reparandum and flat for repeats, discourse for filler "like", goeswith for ASR splits) to the mova dialect's MISC for ATIS and ESLSpok, and report LAS on ATIS dev and cross-validation, where ATIS currently sits at 63.0 LAS.
• I2. gparse: start from the EWT model and continue on the ATIS or ESLSpok training folds, and compare with a model trained from scratch on the same folds; the paper's pretrain-then-fine-tune gain was about 7 UAS points. Decide on cross-validation.
• I3. Measure en parse-eval on EWT dev with punctuation removed and text lowercased, since the paper's transcripts have no punctuation (punct is 11.5% of EWT tokens), and report the LAS loss.
• I4. Check the goeswith rule in the annot gate with treeagree on ASR-style teacher output, so that split words are linked rather than parsed as separate heads.

For Mova

ATIS (LAS 63.0, lowercase queries without punctuation) and ESLSpok are Mova's spoken and dialogue data, so this paper frames the problem Mova already has. The ideas touch 37. Register and variation, 38-dialogue-and-discourse-markers.md and the annot gates. Use cross-validation for decisions and dev for the rest.

Topics: Discourse analysis · Dependency parsing

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/d19-1162 · arXiv 1909.03317

@inproceedings{davidson2019dependency,
  title = {{Dependency Parsing for Spoken Dialog Systems}},
  author = {Davidson, Sam and Yu, Dian and Yu, Zhou},
  booktitle = {Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)},
  year = {2019},
  doi = {10.18653/v1/d19-1162},
  eprint = {1909.03317},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/d19-1162}
}

References

29 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Allison Adams. 2017. Dependency parsing and dialogue systems : an investigation of dependency parsing for commercial application. find
2. Frederic Bechet, Alexis Nasr, and Benoit Favre. 2014. Adapting dependency parsing to spontaneous speech for open domain spoken language understanding. In Fifteenth Annual Conference of the International Speech Communication Association. find
3. Douglas Biber and Susan Conrad. 1999. Lexical bundles in conversation and academic prose. Language and Computers, 26:181–190. find
4. Andrew Caines, Michael McCarthy, and Paula Buttery. 2017. Parsing transcripts of speech. In Proceedings of the Workshop on Speech-Centric Natural Language Processing, pages 27–36. find
5. Ronald Carter and Michael McCarthy. 2017. Spoken grammar: Where are we and where are we going? Applied Linguistics, 38(1):1–20. find
6. Chun-Yen Chen, Dian Yu, Weiming Wen, Yi Mang Yang, Jiaping Zhang, Mingyang Zhou, Kevin Jesse, Austin Chau, Antara Bhowmick, Shreenath Iyer, et al. 2018. Gunrock: Building a human-like social bot by leveraging large scale real user data. find
7. Junhwi Choi, Donghyeon Lee, Seounghan Ryu, Kyu-song Lee, Kyungduk Kim, Hyungjong Noh, and Gary Geunbae Lee. 2016. Engine-Independent ASR Error Management for Dialog Systems, pages 193– 203. Springer International Publishing, Cham. find
8. Kevin Clark, Minh-Thang Luong, Christopher D. Manning, and Quoc V. Le. 2018. Semi-supervised sequence modeling with cross-view training. CoRR, abs/1809.08370. ↗
9. Christine Doran, John Aberdeen, Laurie Damianos, and Lynette Hirschman. 2003. Comparing several aspects of human-computer and human-human dialogues. In Current and new directions in discourse and dialogue, pages 133–159. Springer. find
10. Timothy Dozat and Christopher D Manning. 2016. Deep biaffine attention for neural dependency parsing. arXiv preprint arXiv:1611.01734. ↗ our card
11. Rahhal Errattahi, Salil Deena, Asmaa El Hannani, Hassan Ouahmane, and Thomas Hain. 2018. Improving asr error detection with rnnlm adaptation. In 2018 IEEE Spoken Language Technology Workshop (SLT), pages 190–196. IEEE. find
12. John J. Godfrey, Edward C. Holliman, and Jane Mc-Daniel. 1992. Switchboard: Telephone speech corpus for research and development. In Proceedings of the 1992 IEEE International Conference on Acoustics, Speech and Signal Processing - Volume 1, ICASSP’92, pages 517–520, Washington, DC, USA. IEEE Computer Society. find
13. Sonal Gupta, Rushin Shah, Mrinal Mohit, Anuj Kumar, and Mike Lewis. 2018. Semantic parsing for task oriented dialog using hierarchical representations. CoRR, abs/1810.07942. ↗
14. Richard Johansson and Pierre Nugues. 2008. Dependency-based semantic role labeling of propbank. pages 69–78. find our card
15. Veton Kepuska and Gamal Bohouta. 2018. Next-generation of virtual personal assistants (microsoft cortana, apple siri, amazon alexa and google home). In 2018 IEEE 8th Annual Computing and Communication Workshop and Conference (CCWC), pages 99–103. IEEE. find
16. Thomas Kollar, Danielle Berry, Lauren Stuart, Karolina Owczarzak, Tagyoung Chung, Lambert Mathias, Michael Kayser, Bradford Snow, and Spyros Matsoukas. 2018. The alexa meaning representation language. pages 177–184. find
17. Geoffrey Leech. 2000. Grammars of spoken english: New outcomes of corpus-oriented research. Language learning, 50(4):675–724. find
18. Yijia Liu, Yi Zhu, Wanxiang Che, Bing Qin, Nathan Schneider, and Noah A Smith. 2018. Parsing tweets into universal dependencies. arXiv preprint arXiv:1804.08228. ↗ our card
19. Christopher D. Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven J. Bethard, and David Mc-Closky. 2014. The Stanford CoreNLP natural language processing toolkit. In Association for Computational Linguistics (ACL) System Demonstrations, pages 55–60. find
20. Marie W. Meteer and Ann A. Taylor. 1995. Dysfluency annotation stylebook for the Switchboard corpus. find
21. Alexis Nasr, Frederic Bechet, Benoit Favre, Thierry Bazillon, Jose Deulofeu, and Andre Valli. 2014. Automatically enriching spoken corpora with syntactic information for linguistic studies. In LREC, pages 854–858. Joakim Nivre, Željko Agić, Lars Ahrenberg, Maria Jesus Aranzabe, Masayuki Asahara, Aitziber Atutxa, Miguel Ballesteros, John Bauer, Kepa Bengoetxea, Riyaz Ahmad Bhat, Eckhard Bick, Cristina Bosco, Gosse Bouma, Sam Bowman, Marie Candito, Gülşen Cebiroğlu Eryiğit, Giuseppe G. A. Celano, Fabricio Chalub, Jinho Choi, Çağrı Çöltekin, Miriam Connor, Elizabeth Davidson, Marie-Catherine de Marneffe, Valeria de Paiva, Arantza Diaz de Ilarraza, Kaja Dobrovoljc, Timothy Dozat, Kira Droganova, Puneet Dwivedi, Marhaba Eli, Tomaž Erjavec, Richárd Farkas, Jennifer Foster, Cláudia Freitas, Katarı́na Gajdošová, Daniel Galbraith, Marcos Garcia, Filip Ginter, Iakes Goenaga, Koldo Gojenola, Memduh Gökırmak, Yoav Goldberg, Xavier Gómez Guinovart, Berta Gonzáles Saavedra, Matias Grioni, Normunds Grūzı̄tis, Bruno Guillaume, Nizar Habash, Jan Hajič, Linh Hà Mỹ, Dag Haug, Barbora Hladká, Petter Hohle, Radu Ion, Elena Irimia, Anders Johannsen, Fredrik Jørgensen, Hüner Kaşıkara, Hiroshi Kanayama, Jenna Kanerva, Natalia Kotsyba, Simon Krek, Veronika Laippala, Phng Lê H`ông, Alessandro Lenci, Nikola Ljubešić, Olga Lyashevskaya, Teresa Lynn, Aibek Makazhanov, Christopher Manning, Cătălina Mărănduc, David Mareček, Héctor Martı́nez Alonso, André Martins, Jan Mašek, Yuji Matsumoto, Ryan McDonald, Anna Missilä, Verginica Mititelu, Yusuke Miyao, Simonetta Montemagni, Amir More, Shunsuke Mori, Bohdan Moskalevskyi, Kadri Muischnek, Nina Mustafina, Kaili Müürisep, Lng Nguy˜ên Thi., Huy`ên Nguy˜ên Thi. find
22. Minh, Vitaly Nikolaev, Hanna Nurmi, Stina Ojala, Petya Osenova, Lilja Øvrelid, Elena Pascual, Marco Passarotti, Cenel-Augusto Perez, Guy Perrier, Slav Petrov, Jussi Piitulainen, Barbara Plank, Martin Popel, Lauma Pretkalniņa, Prokopis Prokopidis, Tiina Puolakainen, Sampo Pyysalo, Alexandre Rademaker, Loganathan Ramasamy, Livy Real, Laura Rituma, Rudolf Rosa, Shadi Saleh, Manuela Sanguinetti, Baiba Saulı̄te, Sebastian Schuster, Djamé Seddah, Wolfgang Seeker, Mojgan Seraji, Lena Shakurova, Mo Shen, Dmitry Sichinava, Natalia Silveira, Maria Simi, Radu Simionescu, Katalin Simkó, Mária Šimková, Kiril Simov, Aaron Smith, Alane Suhr, Umut Sulubacak, Zsolt Szántó, Dima Taji, Takaaki Tanaka, Reut Tsarfaty, Francis Tyers, Sumire Uematsu, Larraitz Uria, Gertjan van Noord, Viktor Varga, Veronika Vincze, Jonathan North Washington, Zdeněk Žabokrtský, Amir Zeldes, Daniel Zeman, and Hanzhi Zhu. 2017. Universal dependencies 2.0. LINDAT/CLARIN digital library at the Institute of Formal and Applied Linguistics (ÚFAL), Faculty of Mathematics and Physics, Charles University. find
23. Natalia Silveira, Timothy Dozat, Marie-Catherine de Marneffe, Samuel Bowman, Miriam Connor, John Bauer, and Christopher D. Manning. 2014. A gold standard dependency corpus for English. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC-2014). find
24. Sara Stymne, Miryam de Lhoneux, Aaron Smith, and Joakim Nivre. 2018. Parser training with heterogeneous treebanks. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 619–625. find
25. Joakim Nivre, Marie-Catherine De Marneffe, Filip Ginter, Yoav Goldberg, Jan Hajic, Christopher D Manning, Ryan T McDonald, Slav Petrov, Sampo Pyysalo, Natalia Silveira, et al. 2016. Universal dependencies v1: A multilingual treebank collection. In LREC. find our card
26. Joakim Nivre, Johan Hall, and Jens Nilsson. 2006. Maltparser: A data-driven parser-generator for dependency parsing. In In Proc. of LREC-2006, pages 2216–2219. find
27. Siyao Peng and Amir Zeldes. 2018. All roads lead to ud: Converting stanford and penn parses to english universal dependencies with multilayer annotations. In Proceedings of the Joint Workshop on Linguistic Annotation, Multiword Expressions and Constructions (LAW-MWE-CxG-2018), pages 167–177. find our card
28. Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. Glove: Global vectors for word representation. In In EMNLP. find
29. Peng Qi, Timothy Dozat, Yuhao Zhang, and Christopher D Manning. 2019. Universal dependency parsing from scratch. arXiv preprint arXiv:1901.10457. ↗

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/ud-2019-davidson-dependency-parsing-spoken-dialog-systems/