Classifying Syntactic Errors in Learner Language

Summary

The authors present SERCL, an automatic taxonomy and classifier for syntactic errors in learner language, where a syntactic error is one whose correction changes the part of speech, dependency relation or morphological features involved. For each erroneous span and its correction, the method picks the node closest to the root of the dependency tree, and the error type is the ordered pair of the two nodes' labels. Because labels come from Universal Dependencies, no hand-built category set is needed and the approach carries over to other languages. Reliability is checked by comparing results on automatic versus manual parses of a learner treebank and by relating the types to NUCLE's manual categories and to ERRANT. The paper describes syntactic error distributions in learner English and learner Russian and shows that leading GEC systems correct syntactic errors less well than other errors. It offers a multilingual, fine-grained error analysis compatible with standard NLP tooling.

How Mova uses it

• en/seeds/errors/errant-taxonomy.md: cited as the complement to ERRANT — SERCL classifies syntactically a large share of what ERRANT leaves as OTHER, and supports Mova's view that error types with a trace in the UD tree can be detected by tree rules.
• It is background reading for the error-rule section; no SERCL classifier is implemented.

Effectiveness in Mova

Background reading only; not measured separately.

👨‍🔬💥

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.conll-1.7 · arXiv 2010.11032

@inproceedings{choshen2020classifying,
  title = {{Classifying Syntactic Errors in Learner Language}},
  author = {Choshen, Leshem and Nikolaev, Dmitry and Berzak, Yevgeni and Abend, Omri},
  booktitle = {Proceedings of the 24th Conference on Computational Natural Language Learning},
  year = {2020},
  doi = {10.18653/v1/2020.conll-1.7},
  eprint = {2010.11032},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2020.conll-1.7}
}

References

26 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Leshem Choshen and Omri Abend. 2018c. Reference-less measure of faithfulness for grammatical error correction. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. find
2. Omri Abend and A. Rappoport. 2013. Universal conceptual cognitive annotation (UCCA). In ACL. find our card
3. Hiroki Asano, Masato Mita, Tomoya Mizumoto, and Jun Suzuki. 2019. The AIP-tohoku system at the Daniel Dahlmeier, Hwee Tou Ng, and Siew Mei Wu. 2013. Building a large annotated corpus of learner english: The nus corpus of learner english. In Proceedings of the eighth workshop on innovative use of NLP… find
4. Gaoqi Rao, Qi Gong, Baolin Zhang, and Endong Xun. 2018. Overview of nlptea-2018 share task chinese grammatical error diagnosis. In Proceedings of the 5th Workshop on Natural Language Processing Techniques for Educational Applications, pages 42– 51. find
5. Bonnie J. Dorr. 1994. Machine translation divergences: a formal description and proposed solution. Computational linguistics, 20(4):597–635. find our card
6. Alla Rozovskaya and Dan Roth. 2019. Grammar error correction in morphologically rich languages: The case of russian. Transactions of the Association for Computational Linguistics, 7:1–17. find
7. Noura Farra, Nadi Tomeh, Alla Rozovskaya, and Nizar Habash. 2014. Generalized character-level spelling error correction. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), volume 2, pages 161–167. find
8. Roman Grundkiewicz, Marcin Junczys-Dowmunt, and Kenneth Heafield. 2019. Neural grammatical error correction systems with unsupervised pre-training on synthetic data. In Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 252–263, Florence, Ita… find
9. Alla Rozovskaya, Dan Roth, and Vivek Srikumar. 2014. Correcting grammatical verb errors. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, pages 358–367. find
10. Keisuke Sakaguchi, Matt Post, and Benjamin Van Durme. 2017. Error-repair dependency parsing for ungrammatical texts. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 189–195. find
11. Marcin Junczys-Dowmunt, Roman Grundkiewicz, Shubha Guha, and Kenneth Heafield. 2018. Approaching neural grammatical error correction as a low-resource machine translation task. In NAACL-HLT. find
12. Felix Stahlberg and Bill Byrne. 2019. The CUED’s grammatical error correction systems for BEA-2019. In Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 168–175, Florence, Italy. Association for Computational Linguistics. find
13. Yoav Kantor, Yoav Katz, Leshem Choshen, Naftali Naftali, and Noam Slonim. 2019. Learning to combine grammatical error corrections. In BEA 2019 Shared Task: Grammatical Error Correction. find
14. Milan Straka, Jan Hajic, and Jana Straková. 2016. Udpipe: Trainable pipeline for processing conll-u files performing tokenization, morphological analysis, pos tagging and parsing. In LREC. find our card
15. Jared Lichtarge, Christopher Alberti, Shankar Kumar, Noam Shazeer, Niki Parmar, and Simon Tong. 2019. Corpora generation for grammatical error correction. In NAACL-HLT. find
16. Ben Swanson and Elif Yamangil. 2012. Correction detection and error type selection as an ESL educational aid. In Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 357–361, Montréal, Canada. Associati… find
17. Tomoya Mizumoto, Yuta Hayashibe, Mamoru Komachi, Masaaki Nagata, and Yuji Matsumoto. 2012. The effect of learner corpus size in grammatical error correction of esl writings. In COLING. find
18. Diane Nicholls. 2003. The cambridge learner corpus: Error coding and analysis for lexicography and elt. In Proceedings of the Corpus Linguistics 2003 conference, volume 16, pages 572–581. find
19. Dmitry Nikolaev, Ofir Arviv, Taelin Karidi, Neta Kenneth, Veronika Mitnik, Lilja Maria Saeboe, and Omri Abend. 2020. Fine-grained analysis of cross-linguistic syntactic divergences. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1159–1176, Online. A… find
20. Joakim Nivre, Marie-Catherine de Marneffe, Filip Ginter, Yoav Goldberg, Jan Hajic, Christopher D. Manning, Ryan McDonald, Slav Petrov, Sampo Pyysalo, Natalia Silveira, Reut Tsarfaty, and Daniel Zeman. 2016. Universal Dependencies v1: A multilingual treebank collection. In Proc. of LREC. find our card
21. Shuyao Xu, Jiehao Zhang, Jin Chen, and Long Qin. 2019. Erroneous data generation for grammatical error correction. In Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 149–158, Florence, Italy. Association for Computational Linguistics. find
22. Wajdi Zaghouani, Behrang Mohit, Nizar Habash, Ossama Obeid, Nadi Tomeh, Alla Rozovskaya, Noura Farra, Sarah Alkuhlani, and Kemal Oflazer. 2014. Large scale Arabic error annotation: Guidelines and framework. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LR… find
23. Tatiana Zdorenko. 2010. Subject omission in russian: a study of the russian national corpus. In Stefan Th. Gries, Stefanie Wulff, and Mark Davies, editors, Corpus-linguistic applications. Current studies, new directions, pages 119–133. Brill Rodopi. find
24. Daniel Zeman, Jan Hajič, Martin Popel, Martin Potthast, Milan Straka, Filip Ginter, Joakim Nivre, and Slav Petrov. 2018. find
25. Conll 2018 shared task: multilingual parsing from raw text to universal dependencies. In Proceedings of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pages 1–21. find our card
26. Junchao Zheng, Courtney Napoles, Joel Tetreault, and Kostiantyn Omelianchuk. 2018. How do you correct run-on sentences it’s not as easy as it seems. In Proceedings of the 2018 EMNLP Workshop W-NUT: The 4th Workshop on Noisy User-generated Text, pages 33–38. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/choshen-2020-classifying-syntactic-errors-learner-language/