Learning attention for historical text normalization by learning to pronounce

What it is about

The paper treats historical normalisation as a character-level transduction with an encoder-decoder RNN, so no explicit character alignment is needed. A second task, grapheme-to-phoneme mapping of modern words, is used as auxiliary data for multi-task learning. The models are tested on 43 Anselm texts, using the first 1,000 tokens of each as the test set and 2,000 to 11,000 tokens for training. All encoder-decoder models beat Norma and the averaged perceptron, and the multi-task models beat the earlier bi-LSTM. Attention alone does not help, and it hurts the multi-task model. Of the 210.9 errors per test set made by the base model, attention fixes 47.7 and multi-task learning 45.4, and 27.7 of these fixes overlap.

Ideas

• I1. en/src/morph.rs: use a letter-to-sound key as a second check on candidate modern forms, built from the dictionary's pronunciations where available. A candidate must match the historical form on the sound-class key. Measure the candidate ranking accuracy on the test split of tools/gutenberg lessons.
• I2. Overlap report in en morph-eval: count the items fixed by each of two methods and their overlap, as the paper does. This shows whether two rule sets fix the same errors before combining them.
• I3. Position-wise counts: learn a character-to-character alignment table by counts, which is the transparent analogue of the attention weights, and measure word accuracy against the base rules on the same 1,000-token test sets.

For Mova

The chapter is 09. Early Modern and archaic forms. The code is en/src/morph.rs and en morph-eval. The paper's neural models are not for Mova; the sound-key and overlap ideas are the parts that can be built with counts and rules. English figures must come from our own lessons.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/p17-1031

@inproceedings{bollmann2017learning,
  title = {{Learning attention for historical text normalization by learning to pronounce}},
  author = {Bollmann, Marcel and Bingel, Joachim and Søgaard, Anders},
  booktitle = {Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2017},
  doi = {10.18653/v1/p17-1031},
  url = {https://doi.org/10.18653/v1/p17-1031}
}

References

1 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Mayce Al Azawi, Muhammad Zeshan Afzal, and Thomas M. Breuel. 2013. Normalizing historical orthography for OCR historical documents using LSTM. In Proceedings of the 2nd International Workshop on Historical Document Imaging and Processing. ACM, pages 80–85. https://doi.org/10.1145/2501115.2501131. ↗

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2017-bollmann-learning-attention-historical-text-normalization/