Comparing Rule-based and SMT-based Spelling Normalisation for English Historical Texts

What it is about

Historical spellings must be mapped to present-day spelling before taggers and parsers can work on them. The paper compares two systems on the ARCHER corpus (English texts from 1650 to 1850): VARD2, a rule-based tool that combines a replacement dictionary, SoundEx-like matching and letter-replacement rules with hand-set weights, and a character-level statistical machine translation (SMT) model. Training used 109 annotated documents (about 200,000 words, 6,975 normalisations); evaluation used 30 documents with 1,467 normalisations. Inter-annotator agreement was 83%. The rule-based system reaches up to 94% precision at 74% recall once trained on the ARCHER data; the SMT system is less precise (80.5% precision, 64.4% recall on the full evaluation set) but improves accuracy in every tested period. Its main errors are missed apostrophes (mans, o'er) and endings such as -ie and -y, while VARD2 wrongly maps second-person -st verb forms (hast, darest) to plurals. Keeping function words such as thy and thee out of normalisation gave a clear gain.

Ideas

I1. en/src/emode.rs: a layered normaliser in the same spirit as VARD2. First a closed dictionary of Early Modern forms, then letter-replacement rules with fixed weights chosen by counts on a training split, then a dictionary check. Report precision and recall per century, run twice.
I2. Protect function words by construction: a frozen list of forms that are never rewritten (thy, thee, thou, ye), taken from the training data. Measure how many false rewrites the list prevents on a held-out split.
I3. Add an apostrophe and clitic rule set (mans to man's, y'are to you're) to the tokeniser, and count recall errors by class, as the paper's error analysis does, so each rule is traceable.

For Mova

The chapters are 2. Spelling, print conventions and tokens and 06. Spelling rules: inflection, derivation, compounds and national standards. The code is en/src/emode.rs (closed table), en/src/tok.rs (tokeniser) and en/src/morph.rs. The measure: precision and recall per century on held-out Early Modern lessons, with the SMT-style lesson that a smaller, period-specific rule set can beat a larger one. Negative control: run the normaliser on modern text; it must change almost nothing.

Topics: Machine translation

Cite

The paper itself, not this card. BibTeX from the library's record of the work.

@inproceedings{schneider2017comparing,
  title = {{Comparing Rule-based and SMT-based Spelling Normalisation for English Historical Texts}},
  author = {Schneider, Gerold and Pettersson, Eva and Percillier, Michael},
  booktitle = {Proceedings of the NoDaLiDa 2017 Workshop on Processing Historical Language},
  year = {2017},
  url = {https://aclanthology.org/W17-0508/}
}

References

16 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

Alistair Baron and Paul Rayson. 2008. VARD 2: A tool for dealing with spelling variation in historical corpora. In Proceedings of the Postgraduate Conference in Corpus Linguistics, Birmingham. Aston University. find our card
Douglas Biber, Edward Finegan, and Dwight Atkinson. 1994. Archer and its challenges: Compiling and exploring a representative corpus of historical english registers. In Udo Fries, Peter Schneider, and Gunnel Tottie, editors, Creating and using English language corpora, Papers from the 14th International Conference on English Language Research on Computerized Corpora, Zurich 1993, pages 1–13. Rodopi, Amsterdam. find
BNC Consortium. 2007. The British National Corpus, Version 3. Distributed by Oxford University Computing Services on behalf of the BNC Consortium. http://www.natcorp.ox.ac.uk/. ↗
Peter Brown, Vincent Della Pietra, Stephen Della Pietra, and Robert Mercer. 1993. The Mathematics of Statistical Machine Translation: Parameter Estimation. Computational Linguistics, 19(2), pages 263–311. find
Marcello Federico, Nicola Bertoldi, and Mauro Cettolo. 2008. IRSTLM: an open source toolkit for handling large scale language models. in Proceedings of Interspeech 2008, pages 1618–1621. find
Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondřej Bojar, Alexandra Constantin, and Evan Herbst, 2007. Moses: Open Source Toolkit for Statistical Machine Translation. in Proceedings of the 45th Annual Meeting of the ACL on Interactive Poster and Demonstration Sessions, pages 177–180. find
Hrafn Loftsson. 2008. Tagging icelandic text: A linguistic rule-based approach. Nordic Journal of Linguistics, 31(1). find
Franz Josef Och and Hermann Ney. 2003. A Systematic Comparison of Various Statistical Alignment Models. Computational Linguistics, 1(29), pages 19–51. find
Eva Pettersson, Beáta Megyesi, and Jörg Tiedemann. 2013. An SMT approach to automatic annotation of historical text. In Proceedings of the NoDaLiDa 2013 workshop on Computational Historical Linguistics. find
Eva Pettersson, Beáta Megyesi, and Joakim Nivre. 2014. A multilingual evaluation of three spelling normalisation methods for historical text. In Proceedings of the 8th Workshop on Language Technology for Cultural Heritage, Social Sciences, and Humanities (LaTeCH) @ EACL 2014, pages 32–41, Gothenburg, Sweden. find
Paul Rayson, Dawn Archer, Alistair Baron, Jonathan Culpeper, and Nicholas Smith. 2007. Tagging the bard: Evaluating the accuracy of a modern POS tagger on Early Modern English corpora. In Proceedings of Corpus Linguistics 2007. University of Birmingham, UK. find
Silke Scheible, Richard J. Whitt, Martin Durrell, and Paul Bennett. 2011. Evaluating an ’off-the-shelf’ POS-tagger on Early Modern German text. In Proceedings of the ACL-HLT 2011 Workshop on Language Technology for Cultural Heritage, Social Sciences, and Humanities (LaTeCH 2011), Portland, Oregon. find
Christer Samuelsson and Atro Voutilainen. 1997. Comparing a linguistic and a stochastic tagger. In Proceedings of of ACL/EACL Joint Conference, Madrid. find
Yves Scherrer and Tomaz̆ Erjavec. 2013. Modernizing historical Slovene words with character-based SMT. In Proceedings of the 4th Biennial Workshop on Balto-Slavic Natural Language Processing, pages 58–62. find
Gerold Schneider, Hans Martin Lehmann, and Peter Schneider. 2014. Parsing Early Modern English corpora. Literary and Linguistic Computing, first published online February 6, 2014 doi:10.1093/ llc/fqu001. find
Jörg Tiedemann. 2009. Character-based PSMT for closely related languages. Proceedings of 13th Annual Conference of the European Association for Machine Translation (EAMT’09), pages 12–19. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/schneider-2017-comparing-rule-based-smt-based/