Mova0.5 pre-release⌕
☰
home / docs / papers / English / Dependency Parsing Evaluation for Low-resource Spontaneous Speech

Dependency Parsing Evaluation for Low-resource Spontaneous Speech

What it is about

The authors take the child-parent conversations of one child (Eve, aged 18 to 27 months) from the CHILDES Brown corpus and convert the CHILDES dependency scheme to UD, mostly by rules: direct relabelings, communicators moved to discourse, copulas and prepositions re-headed to their content words, and coordinated items attached as UD does. A Stanza biaffine parser trained on UD-EWT reaches LAS 86.30 on EWT test. On parent speech it performs about as well, but on child speech it is much weaker, especially at 18 and 21 months. A small amount of in-domain child speech used for fine-tuning improves those young-age scores. The error analysis shows that many discrepancies come from the parser not following the conversion choices (noun-noun modifiers, verb particles, vocatives), and from telegraphic two-word utterances that have several possible readings.

Ideas

For Mova

CHILDES is part of the shipped training data, so the converter decisions shape what the model learns from child speech. The ideas touch 37. Register and variation, 38-dialogue-and-discourse-markers.md (communicators, tagmarkers) and 22-phrases.md (preposition heads). Start with I1 and I2 on dev. Check Data and model licences before any CHILDES material is used beyond measurement.

Topics: Dependency parsing

Cite

The paper itself, not this card. BibTeX from the library's record of the work.

@inproceedings{liu2021dependency,
  title = {{Dependency Parsing Evaluation for Low-resource Spontaneous Speech}},
  author = {Liu, Zoey and Prud'hommeaux, Emily},
  booktitle = {Proceedings of the Second Workshop on Domain Adaptation for NLP},
  year = {2021},
  url = {https://aclanthology.org/2021.adaptnlp-1.16/}
}

References

34 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Allison Adams. 2017. Dependency parsing and dialogue systems: an investigation of dependency parsing for commercial application. find
  2. Željko Agić, Anders Johannsen, Barbara Plank, Héctor Martı́nez Alonso, Natalie Schluter, and Anders Søgaard. 2016. Multilingual projection for parsing truly low-resource languages. Transactions of the Association for Computational Linguistics, 4:301– 312. find
  3. Taylor Berg-Kirkpatrick, David Burkett, and Dan Klein. 2012. An empirical investigation of statistical significance in NLP. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 995–1005, Jeju Island, Korea. Association for Computational Linguistics. find
  4. Douglas Biber. 1991. Variation across speech and writing. Cambridge University Press. find
  5. Gosse Bouma, Yuji Matsumoto, Stephan Oepen, Kenji Sagae, Djamé Seddah, Weiwei Sun, Anders Søgaard, Reut Tsarfaty, and Dan Zeman, editors. 2020. Proceedings of the 16th International Conference on Parsing Technologies and the IWPT 2020 Shared Task on Parsing into Enhanced Universal Dependencies. Association for Computational Linguistics, Online. find
  6. Roger Brown. 1973. A first language: The early stages. Harvard University Press. find
  7. Sam Davidson, Dian Yu, and Zhou Yu. 2019. Dependency parsing for spoken dialog systems. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 1513– 1519, Hong Kong, China. Association for Computational Linguistics. ↗ our card
  8. Kaja Dobrovoljc and Joakim Nivre. 2016. The Universal Dependencies treebank of spoken Slovenian. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 1566–1573, Portorož, Slovenia. European Language Resources Association (ELRA). ↗
  9. Timothy Dozat, Peng Qi, and Christopher D. Manning. 2017. Stanford’s graph-based neural dependency parser at the CoNLL 2017 shared task. In Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pages 20–30, Vancouver, Canada. Association for Computational Linguistics. find our card
  10. Rotem Dror, Gili Baumer, Segev Shlomov, and Roi Reichart. 2018. The hitchhiker’s guide to testing statistical significance in natural language processing. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1383–1392, Melbourne, Australia. Association for Computational Linguistics. find
  11. Bradley Efron and Robert J Tibshirani. 1994. An introduction to the bootstrap. CRC press. find
  12. Jason M. Eisner. 1996. Three new probabilistic models for dependency parsing: An exploration. In COL-ING 1996 Volume 1: The 16th International Conference on Computational Linguistics. find our card
  13. Fernanda Ferreira and Michael Anes. 1994. Why study spoken language? In Morton Ann Gernsbacher, editor, Handbook of psycholinguistics, page 33–56. Academic Press. find
  14. John J Godfrey, Edward C Holliman, and Jane Mc-Daniel. 1992. Switchboard: Telephone speech corpus for research and development. In Acoustics, Speech, and Signal Processing, IEEE International Conference on, volume 1, pages 517–520. IEEE Computer Society. find
  15. Shai Gretz, Alon Itai, Brian MacWhinney, Bracha Nir, and Shuly Wintner. 2015. Parsing hebrew childes transcripts. Language Resources and Evaluation, 49(1):107–145. find
  16. Dan Kondratyuk and Milan Straka. 2019. 75 languages, 1 model: Parsing Universal Dependencies universally. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2779–2795, Hong Kong, China. Association for Computational Linguistics. find our card
  17. Lingpeng Kong, Nathan Schneider, Swabha Swayamdipta, Archna Bhatia, Chris Dyer, and Noah A. Smith. 2014. A dependency parser for tweets. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1001–1012, Doha, Qatar. Association for Computational Linguistics. find our card
  18. Sandra Kübler, Ryan McDonald, and Joakim Nivre. 2009. Dependency parsing. Synthesis lectures on human language technologies, 1(1):1–127. Linguistics: System Demonstrations, pages 101– 108, Online. Association for Computational Linguistics. find
  19. Yijia Liu, Yi Zhu, Wanxiang Che, Bing Qin, Nathan Schneider, and Noah A. Smith. 2018. Parsing tweets into Universal Dependencies. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 965–975, New Orleans, Louisiana. Association for Computational Linguistics. find our card
  20. Kenji Sagae, Eric Davis, Alon Lavie, Brian MacWhinney, and Shuly Wintner. 2007. High-accuracy annotation and parsing of CHILDES transcripts. In Proceedings of the Workshop on Cognitive Aspects of Computational Language Acquisition, pages 25–32, Prague, Czech Republic. Association for Computational Linguistics. ↗
  21. Brian MacWhinney. 2000. The CHILDES Project: Tools for analyzing talk. transcription format and programs, volume 1. Psychology Press. find
  22. Ryan McDonald, Joakim Nivre, Yvonne Quirmbach-Brundage, Yoav Goldberg, Dipanjan Das, Kuzman Ganchev, Keith Hall, Slav Petrov, Hao Zhang, Oscar Täckström, Claudia Bedini, Núria Bertomeu Castelló, and Jungmee Lee. 2013. Universal Dependency annotation for multilingual parsing. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 92–97, Sofia, Bulgaria. Association for Computational Linguistics. find
  23. Susanne Miyata, Kenji Sagae, and Brian MacWhinney. 2013. The syntax parser grasp for childes (in japanese). Journal of Health and Medical Science, (3):45–62. find
  24. Jan Odijk, Alexis Dimitriadis, Martijn Van der Klis, Marjo Van Koppen, Meie Otten, and Remco van der Veen. 2018. The anncor childes treebank. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan, pages 2275–2283. European Language Resources Association (ELRA). find
  25. Roy C O’Donnell. 1974. Syntactic differences between speech and writing. American Speech, 49(1/2):102– 110. find
  26. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. GloVe: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543, Doha, Qatar. Association for Computational Linguistics. ↗
  27. Peng Qi, Timothy Dozat, Yuhao Zhang, and Christopher D. Manning. 2018. Universal Dependency parsing from scratch. In Proceedings of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pages 160–170, Brussels, Belgium. Association for Computational Linguistics. find our card
  28. Peng Qi, Yuhao Zhang, Yuhui Zhang, Jason Bolton, and Christopher D. Manning. 2020. Stanza: A python natural language processing toolkit for many human languages. In Proceedings of the 58th Annual Meeting of the Association for Computational Kenji Sagae, Eric Davis, Alon Lavie, Brian MacWhinney, and Shuly Wintner. 2010. Morphosyntactic annotation of childes transcripts. Journal of child language, 37(3):705–729. find our card
  29. Kenji Sagae, Alon Lavie, and Brian MacWhinney. 2005. Automatic measurement of syntactic development in child language. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL’05), pages 197–204, Ann Arbor, Michigan. Association for Computational Linguistics. ↗
  30. Kenji Sagae, Brian MacWhinney, and Alon Lavie. 2004. Adding syntactic annotations to transcripts of parent-child dialogs. In Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC’04), Lisbon, Portugal. European Language Resources Association (ELRA). ↗
  31. Justin Spence, Zoey Liu, Palakurthy Kayla, and Lee-Wynant Tyler. 2018. Syntactic annotation of a hupa text corpus. In Working Papers in Athabaskan Languages: Alaska Native Language Center Working Papers, pages 37–53. Fairbanks, AK: ANLC Publications. find
  32. Weiwei Sun, Yufei Chen, Xiaojun Wan, and Meichun Liu. 2019. Parsing Chinese sentences with grammatical relations. Computational Linguistics, 45(1):95– 136. find
  33. Clara Vania, Yova Kementchedjhieva, Anders Søgaard, and Adam Lopez. 2019. A systematic comparison of methods for low-resource dependency parsing on genuinely low-resource languages. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 1105–1116, Hong Kong, China. Association for Computational Linguistics. find
  34. Daniel Zeman, Joakim Nivre, Mitchell Abrams, Elia Ackermann, et al. 2020. Universal dependencies 2.7. LINDAT/CLARIAH-CZ digital library at the Institute of Formal and Applied Linguistics (ÚFAL), Faculty of Mathematics and Physics, Charles University. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/ud-2021-liu-dependency-parsing-evaluation-low-resource/