Optimizing the Neural Network Training for OCR Error Correction of Historical Hebrew Texts

What it is about

OCR of historical Hebrew newspapers (the JPress collection) is noisy, and neural post-correctors need large hand-corrected sets. The authors learn the typical OCR errors of the corpus from about a hundred hand-corrected articles, inject those errors into clean text to make large artificial training pairs, and train a light neural corrector. Training on corpus-specific injected errors beats random noise, and the result depends strongly on the period and genre of the clean text: trained on the Bible, the corrector introduces more errors than it fixes on modern-era newspapers. It also beats general spellcheckers on this task. This is the short version of the period-specific study (hbo-2023-suissa-toward-period-specific-optimized-neural).

Ideas

• I1. Period matters for any Hebrew lexicon-based correction: a word list from the Torah must not be used to correct or normalise later Hebrew, nor the reverse. A rule for Mova's own lookups: each lexicon carries its period, and a lookup outside the period is flagged.
• I2. Errors learned from a small hand-corrected sample, then injected at scale, is a cheap way to build test sets; Mova can do the same for its damaged-text robustness test (see the companion card's confusion table).

For Mova

Mova does no OCR; the neural corrector is out of scope and the NC-ND licence keeps the paper to tests and citation. I1 is a note in 12. Biblical, Mishnaic and Modern Hebrew compared (why tools for one stage misread another) and a field in macula dict. I2 is the method for a test harness only.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. arXiv 2307.16220

@misc{suissa2023optimizing,
  title = {{Optimizing the Neural Network Training for OCR Error Correction of Historical Hebrew Texts}},
  author = {Suissa, Omri and Elmalech, Avshalom and Zhitomirsky-Geffet, Maayan},
  year = {2023},
  eprint = {2307.16220},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2307.16220}
}

References

20 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Chiron, G., Doucet, A., & Moreux, J.: Competition on Post-OCR Text Correction. In: ICDAR2017 Competition on Post-OCR Text Correction, pp. 1423–1428. (2017). find
2. Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y.: Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) (2014). find
3. Ghosh, S., & Kristensson, P. O.: Neural Networks for Text Correction and Completion in Keyboard Decoding. (2017). find
4. D’hondt, E., Grouin, C., & Grau, B.: Generating a Training Corpus for OCR Post-Correction Using Encoder-Decoder Model. In: Proceedings of the Eighth International Joint Conference on Natural Language Processing. vol. 1, pp. 1006-1014. (2017). find
5. Hochreiter, S., & Schmidhuber, J. J.: Long short-term memory. Neural Computation, 9(8), 1–32 (1997). find
6. Lansdall-Welfare, T., Sudhahar, S., Thompson, J., Lewis, J., & Cristianini, N.: Content analysis of 150 years of British periodicals. In: Proceedings of the National Academy of Sciences. vol. 114 (4), pp. 457–465. (2017) find
7. Levenshtein, Vladimir I.: Binary codes capable of correcting deletions, insertions, and reversals. Soviet physics doklady 10(8), 707-710 (1966). find
8. Needle, S. B., Christus, A. S. D., & Needleman, Saul B., and C. D. W.: A General Method Applicable to the Search for Similarities in the Amino Acid Sequence of Two Proteins. Journal of Molecular Biology, 48(3), 443–453 (1970). find
9. Pascanu, R., Mikolov, T., & Bengio, Y.: On the difficulty of training Recurrent Neural Networks. In: International conference on machine learning, pp. 1310-1318. (2013). find
10. Raaijmakers, S.: A Deep Graphical Model for Spelling Correction. In: BNAIC 2013, Proceedings of the 25th Benelux Conference on Artificial Intelligence. Delft, The Netherlands (2013). find
11. Reynaert, M.: Non-interactive OCR post-correction for giga-scale digitization projects. In: Lecture Notes in Computer Science, LNCS, vol. 4919, pp. 617–630. Springer, Heidelberg (2008). find
12. Schuster, M., & Paliwal, K. K.: Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing, 45(11), 2673–2681 (1997). find
13. Sutskever, I., Vinyals, O., & Le, Q. V.: Sequence to Sequence Learning with Neural Networks. In Advances in neural information processing systems, 3104-3112 (2014) find
14. Werbos, P.: Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences. Ph. D. dissertation, Harvard University. (1974). find
15. The British Newspaper Archive, https://www.britishnewspaperarchive.co.uk/, last accessed 2019/06/20. ↗
16. NYTimes.com, "Google Books: A Complex and Controversial Experiment", last accessed 2015/10/28. find
17. JPress Collection Homepage, http://web.nli.org.il/sites/JPress/Hebrew/Pages/default.aspx, last accessed 2019/06/20. ↗
18. Project Ben Yehuda Homepage, https://bybe.benyehuda.org/, last accessed 2019/06/20. ↗
19. Project Gutenberg Homepage, https://www.gutenberg.org/, last accessed 2019/06/20. ↗
22. Ali, A., & Renals, S. (2018, July). Word error rate estimation for speech recognition: e-WER. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) (pp. 20-24 find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/hbo-2023-suissa-optimizing-neural-network-training-ocr/