Mova0.5 pre-release⌕
☰
home / docs / papers / English / Synthetic Data for English Lexical Normalization: How Close Can We Get to Manually Annotated Data?

Synthetic Data for English Lexical Normalization: How Close Can We Get to Manually Annotated Data?

What it is about

Lexical normalisation maps non-standard tweet words to standard ones, and supervised normalisers need annotated data that exists for few languages and domains. The authors generate training pairs from raw tweets in two directions: adding noise to clean tweets, and normalising noisy tweets without labels. Noise is added by Brown clusters, by two word-embedding neighbourhoods, and by a custom error generator with ten categories (typographical, missing apostrophe, spelling list, repetition, vowel shortening, slang, phonetic, and others). The MoNoise normaliser trained on the best generated data reaches 94.29% accuracy on the LexNorm1.2 test set, against 95.22% with human annotation and 88.45% for copying the input; the best fully unsupervised variant reaches 92.04%. In a human test, five annotators labelled 100 tweets as human or machine-written, with an average accuracy of 64.8%. In an ablation, removing the missing-apostrophe or phonetic category lowers accuracy most, while removing typographical errors has almost no effect.

Ideas

For Mova

The chapters are 06. Spelling rules: inflection, derivation, compounds and national standards for the noise rules, 07. Punctuation, capital letters, abbreviations and numbers for the apostrophe, and 19. The lexicon: vocabulary, multiword expressions and register for slang and register. The code points are en/src/morph.rs (rule tables), en/src/tok.rs (tokens and contractions) and en/src/dict.rs. The measure: normalisation accuracy, precision and recall on a held-out set, the per-rule ablation, all run twice for determinism, with a shuffled-pair negative control that must score lower.

Cite

The paper itself, not this card. BibTeX from the library's record of the work.

@inproceedings{dekker2020synthetic,
  title = {{Synthetic Data for English Lexical Normalization: How Close Can We Get to Manually Annotated Data?}},
  author = {Dekker, Kelly and Goot, Rob van der},
  booktitle = {Proceedings of the Twelfth Language Resources and Evaluation Conference},
  year = {2020},
  url = {https://aclanthology.org/2020.lrec-1.773/}
}

References

25 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Brown, P. F., Desouza, P. V., Mercer, R. L., Pietra, V. J. D., and Lai, J. C. (1992). Class-based n-gram models of natural language. Computational linguistics, 18(4):467– 479. find
  2. Çolakoğlu, T., Sulubacak, U., and Tantuğ, A. C. (2019). Normalizing non-canonical Turkish texts using machine translation approaches. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop, pages 267–272, Florence, Italy, July. Association for Computational Linguistics. find
  3. Contractor, D., Faruquie, T. A., and Subramaniam, L. V. (2010). Unsupervised cleansing of noisy text. In Proceedings of the 23rd International Conference on Computational Linguistics: Posters, COLING ’10, pages 189–196, Stroudsburg, PA, USA. Association for Computational Linguistics. find
  4. Doval, Y., Vilares, J., and Gómez-Rodrı́guez, C. (2020). Towards robust word embeddings for noisy texts. arXiv preprint arXiv:1911.10876.v3. ↗
  5. Eisenstein, J. (2013). What to do about bad language on the internet. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 359–369, Atlanta, Georgia, June. Association for Computational Linguistics. find
  6. Foster, J. and Andersen, Ø. E. (2009). Generrate: generating errors for use in grammatical error detection. In Proceedings of the fourth workshop on innovative use of nlp for building educational applications, pages 82–90. Association for Computational Linguistics. find
  7. Gouws, S., Hovy, D., and Metzler, D. (2011). Unsupervised mining of lexical variants from noisy text. In Proceedings of the First workshop on Unsupervised Learning in NLP, pages 82–90. Association for Computational Linguistics. find
  8. Han, B. and Baldwin, T. (2011). Lexical normalisation of short text messages: Makn sens a# twitter. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1, pages 368–378. Association for Computational Linguistics. find
  9. Han, B., Cook, P., and Baldwin, T. (2012). Automatically constructing a normalisation dictionary for microblogs. In Proceedings of the 2012 joint conference on empirical methods in natural language processing and computational natural language learning, pages 421–432. Association for Computational Linguistics. find
  10. Kasewa, S., Stenetorp, P., and Riedel, S. (2018). Wronging a right: Generating better errors to improve grammatical error detection. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4977–4983, Brussels, Belgium, October-November. Association for Computational Linguistics. find
  11. Li, C. and Liu, Y. (2014). Improving text normalization via unsupervised model and discriminative reranking. In Proceedings of the ACL 2014 Student Research Workshop, pages 86–93. find
  12. Liu, X., Zhang, S., Wei, F., and Zhou, M. (2011). Recognizing named entities in tweets. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies - Volume 1, HLT ’11, pages 359–367, Stroudsburg, PA, USA. Association for Computational Linguistics. find
  13. Mikolov, T., Chen, K., Corrado, G., and Dean, J. (2013). Efficient estimation of word representations in vector space. Proceedings of Workshop at ICLR. find
  14. Owoputi, O., O’Connor, B., Dyer, C., Gimpel, K., and Schneider, N. (2012). Part-of-speech tagging for twitter: Word clusters and other advances. School of Computer Science. find
  15. Pennington, J., Socher, R., and Manning, C. D. (2014). Glove: Global vectors for word representation. find
  16. Proudfoot, C. (2011). An analysis of the relationship between writing skills and Short Messaging Service language: a self–regulatory perspective. Ph.D. thesis, North-West University. find
  17. Schulz, S., Pauw, G. D., Clercq, O. D., Desmet, B., Hoste, V., Daelemans, W., and Macken, L. (2016). Multimodular text normalization of dutch user-generated content. ACM Transactions on Intelligent Systems and Technology (TIST), 7(4):61. find
  18. Tiwari, H. (2015). Internet slang dataset. https://floatcode.wordpress.com/2015/11/28/ internet-slang-dataset/. [Online; accessed 4-June-2019]. ↗
  19. van der Goot, R. and van Noord, G. (2017). MoNoise: Modeling noise using a modular normalization system. Computational Linguistics in the Netherlands Journal, 7:129–144, 12/2017. find
  20. van der Goot, R., van Noord, R., and van Noord, G. (2018). A taxonomy for in-depth evaluation of normalization for user generated content. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC-2018). find
  21. van der Goot, R. (2019a). MoNoise: A multi-lingual and easy-to-use lexical normalization tool. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 201– 206, Florence, Italy, July. Association for Computational Linguistics. find
  22. van der Goot, R. (2019b). Normalization and parsing algorithms for uncertain input. Ph.D. thesis, University of Groningen. find
  23. Wikipedia:Lists of common misspellings. (2017). Wikipedia:lists of common misspellings — Wikipedia, the free encyclopedia. https://en.wikipedia.org/wiki/Wikipedia: Lists_of_common_misspellings. [Online; accessed 8-May-2019]. ↗
  24. Xu, K., Xia, Y., and Lee, C.-H. (2015). Tweet normalization with syllables. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 920–928, Beijing, China, July. Association for Computational Linguistics. find
  25. Yang, Y. and Eisenstein, J. (2013). A log-linear model for unsupervised text normalization. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 61–72. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/dekker-2020-synthetic-data-english-lexical-normalization/