Joint Transition-based Dependency Parsing and Disfluency Detection for Automatic Speech Recognition Texts

What it is about

Earlier joint parsers for disfluent speech were trained and tested on human transcripts, but real input is ASR output, which adds recognition errors on top of disfluencies. The authors make a tree-annotated ASR corpus from Switchboard: they run a Kaldi recipe on the speech, align its output to the transcript with NULL tokens, and project the gold tree onto the ASR words, labelling the arcs that touch errors with an "error" label (15.6% of the 857,493 tokens). Their arc-eager parser adds an Edit action that removes one disfluent token at the top of the stack, LeftArcError and RightArcError actions for ASR-error tokens, a backoff feature shared by an arc and its error variant, and confusion-network posterior features from the recognizer. On the ASR test text, UAS rises from 72.7 (base parser) to 76.2, and disfluency F1 from 60.3 to 62.5, whereas the same kind of parser trained and tested on clean transcripts reaches 89.7 UAS, so the gap is large.

Ideas

• I1. en/src/parse.rs: an Edit transition that removes a disfluent token from the top of the stack, one token at a time, so that the action sequence stays 2n-1; measure LAS on a spoken subset of EWT or ESLspok (repairs and filled pauses marked), against the same parser without the action.
• I2. A projection tool in treebanks/en/: align an ASR hypothesis or any noisy text to its gold tokens with NULL tokens, and project the gold tree with error arcs; measure the share of tokens with error arcs on our own noisy text, to compare with the paper's 15.6%.
• I3. en/src/parse.rs backoff feature: for rare arc labels, score the arc and its rare variant through one shared feature set; measure LAS on rare relations in EWT test, which are the sparse ones in the perceptron.
• I4. en/src/ptag.rs margin features: the tagger's beam margin for a token as a feature for the parser, in place of the recognizer's confusion-network posteriors; measure UAS on noisy text that we build from open transcripts.

For Mova

Touches languages/en/37-register-and-variation, 38-dialogue-and-discourse-markers and 29-ellipsis-word-order-information-structure, and the en crate (parse.rs, ptag.rs) and treebanks/en/. Switchboard is LDC material, so any use is test only. Measure first LAS on a spoken EWT or ESLspok slice with repairs, run twice for determinism, before any new action is added.

Topics: Dependency parsing

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/d16-1109

@inproceedings{yoshikawa2016joint,
  title = {{Joint Transition-based Dependency Parsing and Disfluency Detection for Automatic Speech Recognition Texts}},
  author = {Yoshikawa, Masashi and Shindo, Hiroyuki and Matsumoto, Yuji},
  booktitle = {Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing},
  year = {2016},
  doi = {10.18653/v1/d16-1109},
  url = {https://doi.org/10.18653/v1/d16-1109}
}

References

19 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Hao Cheng, Hao Fang, and Mari Ostendorf. 2015. Open-domain name error detection using a multi-task rnn. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 737– 746. Association for Computational Linguistics. find
2. Eunah Cho, Jan Niehues, and Alex Waibel. 2014. Tight integration of speech disfluency removal into smt. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, volume 2: Short Papers (EACL), pages 43–47. Association for Computational Linguistics. find
3. Marie-Catherine de Marneffe, Bill MacCartney, and Christopher D. Manning. 2006. Generating typed dependency parses from phrase structure parses. In In Proceedings of the 5th International Conference on Language Resources and Evaluation (LREC). find our card
4. James Ferguson, Greg Durrett, and Dan Klein. 2015. Disfluency detection with a semi-markov model and prosodic features. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL), pages 257–262. Association for Computational Linguistics. find
5. J. J. Godfrey, E. C. Holliman, and J. McDaniel. 1992. switchboard: Telephone speech corpus for research and development . In Acoustics, Speech, and Signal Processing, 1992. ICASSP-92., 1992 IEEE International Conference on (Volume:1 ). Proc. IEEE Int. Conf. Acoust. Speech Sig. Proc. find
6. Matthew Honnibal and Mark Johnson. 2014. Joint incremental disfluency detection and dependency parsing. In Transactions of the Association of Computational Linguistics Volume 2, Issue 1 (TACL), pages 131–142. Association for Computational Linguistics. find
7. Julian Hough and Matthew Purver. 2014. Strongly incremental repair detection. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 78–89. Association for Computational Linguistics. find
8. Julian Hough and David Schlangen. 2015. Recurrent neural networks for incremental disfluency detection. Interspeech 2015. find
9. Liang Huang, Suphan Fayong, and Yang Guo. 2012. Structured perceptron with inexact search. In Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics. find our card
10. Yang Liu, Elizabeth Shriberg, and Andreas Stolcke. 2003. Automatic disfluency identification in coversa-tional speech using multiple knowledge sources. In In Proceedings of the 8th Eurospeech Conference. find
11. Marius Alexandru Marin. 2015. In Effective Use of Cross-Domain Parsing in Automatic Speech Recognition and Error Detection. Ph.D. thesis. University of Washington. find
12. Xian Qian and Yang Liu. 2013. Disfluency detection using multi-step stacked learning. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics. find
13. Mohammad Sadegh Rasooli and Joel Tetreault. 2014. Non-monotonic parsing of fluent umm i mean disfluent sentences. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, volume 2: Short Papers (EACL), pages 48–53. Association for Computational Linguistics. find
14. Elizabeth Shriberg. 1994. In Preliminaries to a Theory of Speech Disfluencies. Ph.D. thesis. University of California, Berkeley. find
15. Kristina Toutanova, Dan Klein, Christopher Manning, and Singer Yoram. 2003. Feature-rich part-of-speech tagging with a cyclic dependency network. In In Proceedings of the 2003 Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics, pages 173– 180. Association for Computational Linguistics. find
16. Shuangzhi Wu, Dongdong Zhang, Ming Zhou, and Tiejun Zhao. 2015. Efficient disfluency detection with transition-based parsing. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) (ACL), pages 495–503. Association for Computational Linguistics. find
17. Yue Zhang and Joakim Nivre. 2011. Transition-based dependency parsing with rich non-local features. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (ACL), pages 188–193. Association for Computational Linguistics. find
18. Xiaohui Zhang, Jan Trmal, Daniel Povey, and Sanjeev Khudanpur. 2014. Improving deep neural network acoustic models using generalized maxout networks. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). find
19. Muhua Zhu, Yue Zhang, Wenliang Chen, Miu Zhang, and Jingbo Zhu. 2013. Fast and accurate shift-reduce constituent parsing. In In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (ACL), pages 434–443. Association for Computational Linguistics. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2016-yoshikawa-joint-transition-based-dependency-parsing/