Normalizing without Modernizing: Keeping Historical Wordforms of Middle French while Reducing Spelling Variants

What it is about

The paper reduces spelling variation in sixteenth-century Middle French archival text without modernizing the words. Its gold data are about 1,600 segments normalized by experts from the Geneva Council Registers, aligned with their originals. A pretrained multilingual translation model is fine-tuned on this small corpus, which already gives a large gain: about 36 points more exact segment matches than copying the source. Two ways of producing more training data are then tested: sequence-level knowledge distillation, where the model's own outputs on unseen source text are added with a mixed loss, and generation followed by back-translation, where a language model writes target-side text that is then translated back to create source forms. The generation route gives the best scores, about two points of exact-match accuracy above distillation. Results are averaged over five folds and reported with five metrics, including exact segment accuracy, which counts how many segments need no revision.

Ideas

• I1. Keep the historical form as the target when the author uses it. The standard writes modern forms such as has for hath; an author delta can say the opposite. Measure exact segment matches on dev verses, since that is the number that says how much revision is needed.
• I2. Make artificial author text from the learned rules. Apply the delta's rules in reverse to modern verses, producing author-style sentences, and keep only those that pass the dev check. This is a count-based form of back-translation.
• I3. Use exact segment accuracy as the main number in exact mode. Report it per author, next to the residual, since the byte-match rate is what exact mode promises.
• I4. Mix the standard's counts and the author's counts with a single weight tuned on dev, the count version of the paper's interpolated loss. Never tune the weight on test.

For Mova

The code is tools/translate (profile.rs for the standard, the delta for the author). The neural models and back-translation are not for Mova, but the framing is: the standard should not modernize, and the author delta should carry the archaic forms (open problem 6). The exact-segment measure matches exact mode.

Topics: Machine translation

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/2024.findings-naacl.215

@inproceedings{rubino2024normalizing,
  title = {{Normalizing without Modernizing: Keeping Historical Wordforms of Middle French while Reducing Spelling Variants}},
  author = {Rubino, Raphael and Gerlach, Johanna and Mutal, Jonathan and Bouillon, Pierrette},
  booktitle = {Findings of the Association for Computational Linguistics: NAACL 2024},
  year = {2024},
  doi = {10.18653/v1/2024.findings-naacl.215},
  url = {https://doi.org/10.18653/v1/2024.findings-naacl.215}
}

References

31 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Alistair Baron, Paul Rayson, and DE Archer. 2009. Automatic standardization of spelling for historical text mining. Proceedings of Digital Humanities 2009. find
2. Rachel Bawden, Jonathan Poinhos, Eleni Kogkitsidou, Philippe Gambette, Benoît Sagot, and Simon Gabay. 2022. Automatic normalisation of early Modern French. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 3354– 3366, Marseille, France. European Language Resources Association. find
3. Nikolay Bogoychev and Rico Sennrich. 2019. Domain, translationese and noise in synthetic data for neural machine translation. arXiv preprint arXiv:1911.03362. ↗
4. Marcel Bollmann. 2013. Pos tagging for historical texts with sparse training data. In Proceedings of the 7th linguistic annotation workshop and interoperability with discourse, pages 11–18. find our card
5. Marcel Bollmann, Florian Petran, and Stefanie Dipper. 2011. Rule-based normalization of historical texts. In Proceedings of the Workshop on Language Technologies for Digital Humanities and Cultural Heritage, pages 34–42, Hissar, Bulgaria. Association for Computational Linguistics. find our card
6. Marcel Bollmann and Anders Søgaard. 2016. Improving historical spelling normalization with bidirectional LSTMs and multi-task learning. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 131–139, Osaka, Japan. find
7. The COL-ING 2016 Organizing Committee. find
8. Cristian Buciluǎ, Rich Caruana, and Alexandru Niculescu-Mizil. 2006. Model compression. In Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 535–541. find
9. Angela Fan, Shruti Bhosale, Holger Schwenk, Zhiyi Ma, Ahmed El-Kishky, Siddharth Goyal, Mandeep Baines, Onur Celebi, Guillaume Wenzek, Vishrav Chaudhary, et al. 2021. Beyond english-centric multilingual machine translation. Journal of Machine Learning Research, 22(107):1–48. find
10. Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A. Smith. 2020. Don’t stop pretraining: Adapt language models to domains and tasks. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8342–8360, Online. Association for Computational Linguistics. find
11. Junxian He, Jiatao Gu, Jiajun Shen, and Marc’Aurelio Ranzato. 2019. Revisiting self-training for neural sequence generation. In International Conference on Learning Representations. find
12. Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531. ↗
13. Yoon Kim and Alexander M. Rush. 2016. Sequence-level knowledge distillation. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1317–1327, Austin, Texas. Association for Computational Linguistics. find
14. Natalia Korchagina. 2017. Normalizing medieval german texts: from rules to deep learning. In Proceedings of the NoDaLiDa 2017 Workshop on Processing Historical Language, pages 12–17. find
15. Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2023. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys, 55(9):1–35. find
16. Ilya Loshchilov and Frank Hutter. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101. ↗
17. Benjamin Marie and Atsushi Fujita. 2021. Synthesizing monolingual data for neural machine translation. arXiv preprint arXiv:2101.12462. ↗
18. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318. find our card
19. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32. find
20. Eva Pettersson, Beáta Megyesi, and Joakim Nivre. 2014. A multilingual evaluation of three spelling normalisation methods for historical text. In Proceedings of the 8th workshop on language technology for cultural heritage, social sciences, and humanities (LaTeCH), pages 32–41. find
21. Eva Pettersson, Beáta Megyesi, and Jörg Tiedemann. 2013. An smt approach to automatic annotation of historical text. In Proceedings of the Workshop on Computational Historical Linguistics at NODALIDA 2013, NEALT Proceedings Series, volume 18, pages 54–69. find
22. Maja Popović. 2015. chrf: character n-gram f-score for automatic mt evaluation. In Proceedings of the tenth workshop on statistical machine translation, pages 392–395. find our card
23. Matt Post. 2018. A call for clarity in reporting BLEU scores. In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186– 191, Brussels, Belgium. Association for Computational Linguistics. find our card
24. Alexander Robertson and Sharon Goldwater. 2018. Evaluating historical text normalization systems: How well do they generalize? In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 720–725, New Orleans, Louisiana. Association for Computational Linguistics. find our card
25. Felipe Sánchez-Martínez, Isabel Martínez-Sempere, Xavier Ivars-Ribes, and Rafael C Carrasco. 2013. An open diachronic corpus of historical spanish: annotation criteria and automatic modernisation of spelling. arXiv preprint arXiv:1306.3692. ↗
26. Sarthak Mittal, Oleksii Hrinchuk, and Oleksii Kuchaiev. 2023. Leveraging synthetic targets for machine translation. In Findings of the Association for Computational Linguistics: ACL 2023, pages 9365–9379, Toronto, Canada. Association for Computational Linguistics. find
27. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Improving neural machine translation models with monolingual data. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 86–96, Berlin, Germany. Association for Computational Linguistics. find
28. Niklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M Saiful Bari, Sheng Shen, Zheng-Xin Yong, Hailey Schoelkopf, et al. 2022. Crosslingual generalization through multitask finetuning. arXiv preprint arXiv:2211.01786. ↗
29. Matthew Snover, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul. 2006. A study of translation edit rate with targeted human annotation. In Proceedings of the 7th Conference of the Association for Machine Translation in the Americas: Technical Papers, pages 223–231. find our card
30. Gongbo Tang, Fabienne Cap, Eva Pettersson, and Joakim Nivre. 2018. An evaluation of neural machine translation models on historical spelling normalization. In Proceedings of the 27th International Conference on Computational Linguistics, pages 1320–1331, Santa Fe, New Mexico, USA. Association for Computational Linguistics. find
31. Atnafu Lambebo Tonja, Olga Kolesnikova, Alexander Gelbukh, and Grigori Sidorov. 2023. Low-resource neural machine translation improvement using source-side monolingual data. Applied Sciences, 13(2):1201. Model Hand-crafted Acc ↑ *Table 3: Normalization results on the validation set averaged over 5 folds. Models under Synthetic Data were not finally fine-tuned on the hand-crafted data.* • Extended emphasis and accentuation based on modern usage find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/rubino-2024-normalizing-without-modernizing-keeping-historical/