Mova0.5 pre-release⌕
☰
home / docs / papers / Ancient Hebrew / Toward a Period-Specific Optimized Neural Network for OCR Error Correction of Historical Hebrew Texts

Toward a Period-Specific Optimized Neural Network for OCR Error Correction of Historical Hebrew Texts

What it is about

The full version of the authors' work on Hebrew OCR post-correction. They align 70% of hand-corrected JPress newspaper articles with their OCR output (Needleman–Wunsch) to learn character-level, period-specific confusions — the commonest being ח→ה (about 8%), ד→ר (5%), ג→נ, ב→כ, ם→ס, ח→ת, ו↔י — and inject them into clean corpora of several periods (the Bible, the Ben-Yehuda literary corpus, modern texts) to make artificial training data. A greedy search tunes the network's structure and hyperparameters. The best network, an LSTM trained on Ben-Yehuda texts with period-specific errors, reaches 94% accuracy against 85% for a GRU baseline with generic errors; networks trained on the Bible or on modern Hebrew do badly, sometimes adding errors. Genres of the same period transfer well, and the method also helps on historical Polish.

Ideas

For Mova

I1 goes to 1. Script and pointing (letter shapes) and a test harness for macula dict lookup. I2 is a paragraph in 12-biblical-mishnaic-modern.md. I3 applies to any spelling normaliser Mova builds for Mishnaic or unvocalised text. Being NC-ND, the paper is cited and measured against, never copied.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.1145/3479159 · arXiv 2307.16213

@misc{suissa2023toward,
  title = {{Toward a Period-Specific Optimized Neural Network for OCR Error Correction of Historical Hebrew Texts}},
  author = {Suissa, Omri and Zhitomirsky-Geffet, Maayan and Elmalech, Avshalom},
  year = {2023},
  doi = {10.1145/3479159},
  eprint = {2307.16213},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.1145/3479159}
}

References

54 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ali, A., & Renals, S. (2018). Word error rate estimation for speech recognition: e-WER. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) (pp. 20-24). find
  2. Amrhein, C., & Clematide, S. (2018). Supervised OCR error detection and correction using statistical and neural machine translation methods. Journal for Language Technology and Computational Linguistics (JLCL), 33(1), 49-76. find
  3. Bernstein, M.S., Little, G., Miller, R.C., Hartmann, B., Ackerman, M.S., Karger, D.R., Crowell, D. and Panovich, K. (2010), “October. Soylent: a word processor with a crowd inside” In Proceedings of the 23nd annual ACM symposium on User interface software and technology. ACM. pp. 313-322. find
  4. Barzilai, J., & Borwein, J. M. (1988). Two-point step size gradient methods. IMA journal of numerical analysis, 8(1), 141-148. find
  5. Chiron, G., Doucet, A., Coustaty, M., & Moreux, J. P. (2017, November). Icdar2017 competition on post-ocr text correction. In 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR) (Vol. 1, pp. 1423-1428). IEEE. find
  6. Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014). Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). https://doi.org/10.3115/v1/d14-1179 ↗
  7. Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014). Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555. ↗
  8. Chrupała, G. (2014). Normalizing tweets with edit scripts and recurrent neural embeddings. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) (pp. 680-686). find
  9. Deng, L. & Liu, Y. (Eds.). (2018). Deep learning in natural language processing. Springer. find
  10. Dietterich, T. G. (1998). Approximate statistical tests for comparing supervised classification learning algorithms. Neural computation, 10(7), 1895-1923. find
  11. Drobac, S., Kauppinen, P. S., & Linden, B. K. J. (2017). OCR and post-correction of historical Finnish texts. In Proceedings of the 21st Nordic Conference on Computational Linguistics, NoDaLiDa, 22-24 May 2017, Gothenburg, Sweden. Linköping University Electronic Press. find
  12. Evershed, J., & Fitch, K. (2014, May). Correcting noisy OCR: Context beats confusion. In Proceedings of the First International Conference on Digital Access to Textual Cultural Heritage (pp. 45-51). find
  13. Ghosh, S., & Kristensson, P. O. (2017). Neural networks for text correction and completion in keyboard decoding. arXiv preprint arXiv:1709.06429. ↗
  14. Grouin, C., D’hondt, E., & Grau, B. (2017). Generating a training corpus for OCR post-correction using encoder-decoder model. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers) (pp. 1006-1014). find
  15. Hakala, K., Vesanto, A., Miekka, N., Salakoski, T., & Ginter, F. (2019). Leveraging Text Repetitions and Denoising Autoencoders in OCR Post-correction. arXiv preprint arXiv:1906.10907. ↗
  16. Hämäläinen, M., & Hengchen, S. (2019). From the paft to the fiiture: a fully automatic nmt and word embeddings method for OCR post-correction. arXiv preprint arXiv:1910.05535. ↗
  17. Hasnat, M. A., Habib, S. M., & Khan, M. (2008). A high performance domain specific OCR for Bangla script. In Novel Algorithms and Techniques In Telecommunications, Automation and Industrial Electronics (pp. 174-178). Springer, Dordrecht. find
  18. Hochreiter, S., & Schmidhuber, J. (1997). https://doi.org/10.1162/neco.1997.9.8.1735 ↗
  19. Long Hochreiter, S. (1998). The vanishing gradient problem during learning recurrent neural nets and problem solutions. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, 6(02), 107-116. find
  20. Hoffman, J. (2004). In the beginning: A short history of the Hebrew language. NYU Press. find
  21. Kissos, I., & Dershowitz, N. (2016, April). OCR error correction using character correction and feature-based word classification. In 2016 12th IAPR Workshop on Document Analysis Systems (DAS) (pp. 198-203). IEEE. find
  22. Klein, G., Kim, Y., Deng, Y., Senellart, J., & Rush, A. M. (2017). Opennmt: Open-source toolkit for neural machine translation. arXiv preprint arXiv:1701.02810. ↗
  23. Kolak, O., & Resnik, P. (2002, March). OCR error correction using a noisy channel model. In Human Language Technology Conference (HLT Short-Term Memory. Neural Computation, 9(8), 1735–1780. 2002). find
  24. Krishna, A., Majumder, B. P., Bhat, R. S., & Goyal, P. (2018). Upcycle your ocr: Reusing ocrs for post-ocr text correction in romanised sanskrit. arXiv preprint arXiv:1809.02147. ↗
  25. Kulsum-Binder, R., & Bahloul, M. (2016). Slavic Complexity: A Comparison of English and Polish Noun Properties. find
  26. Jatowt, A., Coustaty, M., Nguyen, N. V., & Doucet, A. (2019, June). Deep statistical analysis of OCR errors for effective post-OCR processing. In 2019 ACM/IEEE Joint Conference on Digital Libraries (JCDL) (pp. 29-38). IEEE. find
  27. Jin, J., Yan, Z., Fu, K., Jiang, N., & Zhang, C. (2016). Neural network architecture optimization through submodularity and supermodularity. stat, 1050, 1. find
  28. Jurafsky, D., & Martin, J. H. (2009). Spelling Correction and the Noisy Channel. The Spelling Correction Task. Speech and Language Processing, 2nd Ed. Prentice-Hall. find
  29. Kingma, D. P., & Ba, J. (2014). Adam: A Method for Stochastic Optimization, 1–15. Learning. https://doi.org/10.1063/1.4902458. ↗
  30. Kumar, A., & Lehal, G. S. (2016). Automatic text correction for Devanagari OCR. Indian Journal of Science and Technology, 9(45). find
  31. Lansdall-Welfare, T., Sudhahar, S., Thompson, J., Lewis, J., Team, F.N. and Cristianini, N. (2017) “Content analysis of 150 years of British periodicals”, Proceedings of the National Academy of Sciences, Vol. 114 No. 4, pp. E457-E465. find
  32. Levenshtein, V.I., (1966), “Binary codes capable of correcting deletions, insertions, and reversals”, Soviet physics doklady, Vol. 10, No. 8, pp. 707-710. find
  33. Manning, C. D., Surdeanu, M., Bauer, J., Finkel, J. R., Bethard, S., & McClosky, D. (2014, June). The Stanford CoreNLP natural language processing toolkit. In Proceedings of 52nd annual meeting of the association for computational linguistics: system demonstrations (pp. 55-60). find
  34. Mikolov, T., Deoras, A., Povey, D., Burget, L., & Černocký, J. (2011). Strategies for training large scale neural network language models. In 2011 IEEE Workshop on Automatic Speech Recognition & Understanding (pp. 196-201). IEEE. find
  35. Mokhtar, K., Bukhari, S. S., & Dengel, A. (2018, April). OCR Error Correction: State-of-the-Art vs an NMT-based Approach. In 2018 13th IAPR International Workshop on Document Analysis Systems (DAS) (pp. 429-434). IEEE. find
  36. Needleman, S.B. and Wunsch, C.D. (1970), “A general method applicable to the search for similarities in the amino acid sequence of two proteins”, Journal of molecular biology, Vol. 48 N. 3, pp. 443-453. find
  37. Nguyen, T. T. H., Jatowt, A., Nguyen, N. V., Coustaty, M., & Doucet, A. (2020, August). Neural Machine Translation with BERT for Post-OCR Error Detection and Correction. In Proceedings of the ACM/IEEE Joint Conference on Digital Libraries in 2020 (pp. 333-336). find
  38. Nguyen, Q. D., Le, D. A., & Zelinka, I. (2019). OCR Error Correction for Unconstrained Vietnamese Handwritten Text. In Proceedings of the Tenth International Symposium on Information and Communication Technology (pp. 132-138). find
  39. Pascanu, R., Mikolov, T., & Bengio, Y. (2013, February). On the difficulty of training recurrent neural networks. In International conference on machine learning (pp. 1310-1318). find
  40. Raaijmakers, S. (2013), “A deep graphical model for spelling correction”, paper presented at the 25th Benelux Conference on Artificial Intelligence (BNAIC), 7 Nov-8 Nov, Delft, The Netherlands, available at: https://pdfs.semanticscholar.org/75aa/9ccef270af5ac31023876257fa32c49024b5.pdf (accessed 29 July 2019). ↗
  41. Reynaert, M. (2008). Non-interactive OCR post-correction for giga-scale digitization projects. In International Conference on Intelligent Text Processing and Computational Linguistics (pp. 617-630). Springer, Berlin, Heidelberg. find
  42. Rigaud, C., Doucet, A., Coustaty, M., & Moreux, J. P. (2019, September). ICDAR 2019 competition on post-OCR text correction. In 2019 International Conference on Document Analysis and Recognition (ICDAR) (pp. 1588-1593). IEEE. find
  43. Schuster, M., & Paliwal, K. K. (1997). Bidirectional recurrent neural networks. IEEE transactions on Signal Processing, 45(11), 2673-2681. https://doi.org/10.1109/78.650093. ↗
  44. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. (2014). Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research, 15(1), 1929-1958. find
  45. Suissa, O., Elmalech, A., & Zhitomirsky-Geffet, M. (2019). Toward the optimized crowdsourcing strategy for OCR post-correction. Aslib Journal of Information Management. Vol. 72 No. 2, pp. 179-197. https://doi.org/10.1108/AJIM-07-2019-0189. ↗
  46. Suissa, O., Elmalech, A. Zhitomirsky-Geffet, M. (2020). Optimizing the Neural Network Training for OCR Error Correction of Historical Hebrew Texts. Poster in Proceedings of I-Conference 2020, Boras, Sweden. find our card
  47. Sutskever, I., Vinyals, O., & Le, Q. V. (2014). Sequence to sequence learning with neural networks. In Advances in neural information processing systems (pp. 3104-3112). find
  48. Tsarfaty, R., Seddah, D., Goldberg, Y., Kübler, S., Versley, Y., Candito, M., ... & Tounsi, L. (2010, June). Statistical parsing of morphologically-rich languages (spmrl) what, how and whither. In Proceedings of the NAACL HLT 2010 First Workshop on Statistical Parsing of Morphologically-Rich Languages (pp. 1-12). find
  49. Tsarfaty, R., Seker, A., Sadde, S., & Klein, S. (2019). What's Wrong with Hebrew NLP? And How to Make it Right. arXiv preprint arXiv:1908.05453. ↗ our card
  50. Volk, M., Clematide, S., & Furrer, L. (2016). Crowdsourcing an OCR Gold Standard for a German and French Heritage Corpus. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016), 975–982. https://doi.org/10.5167/UZH-124786 ↗
  51. Wehrmann, J., Becker, W., Cagnini, H. E., & Barros, R. C. (2017). A character-based convolutional neural network for language-agnostic Twitter sentiment analysis. In 2017 International Joint Conference on Neural Networks (IJCNN) (pp. 2384-2391). IEEE.. find
  52. Werbos, P. J. (1974). New tools for prediction and analysis in the behavioral sciences. Ph. D. dissertation, Harvard University. find
  53. Wu, Y., Schuster, M., Chen, Z., Le, Q. V., Norouzi, M., Macherey, W., ... & Dean, J. (2016). Google's neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144. ↗
  54. Yin, W., Kann, K., Yu, M., & Schütze, H. (2017). Comparative Study of CNN and RNN for Natural Language Processing. arXiv e-prints, arXiv-1702. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/hbo-2023-suissa-toward-period-specific-optimized-neural/