Analyzing the Impact of Spelling Errors on POS-Tagging and Chunking in Learner English
What it is about
Spelling errors in learner English create unknown words, and unknown words are a main cause of POS-tagging and chunking failures. The authors measure how much spelling errors cost, using the Konan-JIEM corpus of 3,260 sentences and 30,517 tokens, of which 654 words (2.1%) are spelling errors. Both taggers and chunkers are linear-chain CRFs, trained on an in-house tagged corpus of 213,017 tokens. Adding prefix and suffix features to the basic features raises tagging accuracy from 93.97% to 95.31%. Correcting the spelling errors by hand (the ideal spell checker) adds only 0.23 points, and a real spell checker adds 0.06 points on top of the affix features, so the authors conclude that a spell checker is not needed. For chunking the ideal correction gives 0.2 points, which is significant, and the checker gives 0.03, which is not. Of the 105 tags that the affix system missed and the gold system got right, 54 were unknown words caused by the spelling error.
Ideas
- I1.
en/src/ptag.rs: add prefix and suffix features to the perceptron tagger, as a closed table of affixes (-ing, -ed, -ly, -tion), and measure how many spelling-broken forms recover their tag ("studing" to VBG). The paper's 1.3-point gain is the baseline to beat. - I2. Report the ceiling before any spelling module: the gap between the tagger on raw input and on hand-corrected input is the most a spell checker can gain. If it is below a set threshold, do not build the module.
- I3. Stratify errors by type (typographical, homophone, confusion, split, merge, inflection) and count tag failures per type, as the paper does for its 105 failures. Use it to decide which rule to write next in 06. Spelling rules: inflection, derivation, compounds and national standards.
- I4. Negative control: replace a fraction of tokens with random real words and check that the tagger's accuracy drops by about the same amount. A tagger that is unaffected by the noise is not reading the tokens.
For Mova
The chapter is 06. Spelling rules: inflection, derivation, compounds and national standards, with 05. The writing system for the character side. The code points are en/src/tok.rs (tokenisation of misspelled tokens), en/src/ptag.rs (the perceptron tagger, where affix features go) and en/src/tag.rs. The measure: tagging accuracy with and without affix features, and the gold-correction ceiling, on a learner set that Mova owns (not an NC corpus), run twice for determinism, with the random-replacement control.
Cite
@inproceedings{mizumoto2017analyzing,
title = {{Analyzing the Impact of Spelling Errors on POS-Tagging and Chunking in Learner English}},
author = {Mizumoto, Tomoya and Nagata, Ryo},
booktitle = {Proceedings of the 4th Workshop on Natural Language Processing Techniques for Educational Applications (NLPTEA 2017)},
year = {2017},
url = {https://aclanthology.org/W17-5909/}
}References
- Jana Z. Sukkarieh and John Blackmore. 2009. c-rater: Automatic Content Scoring for Short Constructed Responses. In Proceedings of FLAIRS, pages 290– 295. find
- Adwait Ratnaparkhi. 1996. A Maximum Entropy Model for Part-Of-Speech Tagging. In Proceedings of EMNLP, pages 133–142. find
- Keisuke Sakaguchi, Tomoya Mizumoto, Mamoru Komachi, and Yuji Matsumoto. 2012. Joint English Spelling Error Correction and POS Tagging for Language Learners Writing. In Proceedings of COL-ING, pages 2357–2374. find our card
- Fei Sha and Fernando Pereira. 2003. Shallow Parsing with Conditional Random Fields. In Proceedings of HLT-NAACL, pages 134–141. find
- Jan Aarts and Sylviane Granger. 1998. Tag sequences in learner corpora: a key to interlanguage grammar and discourse. In Sylviane Granger, editor, Learner English on Computer, pages 132–141. Ad-dison Wesley Longman: London and New York. The POS-tagger was trained with all features. find
This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/mizumoto-2017-analyzing-impact-spelling-errors-pos/