AI-Driven Generation of Old English: A Framework for Low-Resource Languages

What it is about

The paper tries to enlarge the Old English corpus, which has about 3 million words, by generating new Old English text with an 8B language model. It is trained in stages: continued pre-training on English and Old English fragments (text completion, forward and back translation, cross-lingual definitions) with LoRA; then back-translation, where real Old English is translated into English and the pairs are used as new training data; then a dual-agent pipeline. One agent writes English sentences, using examples from the corpus, and a second translates them into Old English with greedy decoding. Evaluation uses BLEU, chrF and METEOR against reference translations, and expert linguists who score inflection, word order, lexical attestedness and coherence on a 1–10 scale; a text is added to the corpus only if it scores at least 7. BLEU for English to Old English rises from 25.9 for the base model to 60.0 after the first stage and 65.4 after back-translation. The expert averages are 9.0 for inflection, 9.0 for word order, 9.1 for lexical choice and 7.8 for coherence, with an overall 8.7. Common errors are looped output, untranslated segments, invented vocabulary and anachronisms. The paper does not say clearly whether the test pairs are kept apart from the pairs taken from the corpus.

Ideas

I1. en/src/emode.rs and tools/gutenberg: back-translation as a rule-based round trip. Generate Early Modern sentences from modern English with the closed form table, translate them back by the inverse table, and keep only pairs that return the source. Report the round-trip exact-match rate per form, with a negative control where the table is shuffled.
I2. Acceptance gate with per-criterion numbers: for each generated sentence, report the pronoun form, the verb ending and the lexical attestedness (is each word in the Early Modern lexicon?) as separate pass or fail values, and admit a sentence only when all pass. Mirrors the paper's threshold of 7 but is explainable.
I3. Loop detector: refuse a generated sentence that repeats a word pair within a short window, as in the looped output the paper reports, and count how many outputs it blocks on 14. Sample texts, read word by word.
I4. Person and number consistency across a passage: the paper's example shifts from second person singular to third person plural mid-passage. Check the pronoun features with coref/src/sieve.rs, and report violations per passage, with a clean passage as control.

For Mova

The Early Modern forms are in en/src/emode.rs, with the chapters 3. Pronouns, 4. Verb endings: -est, -st, -eth, -th and 5. Auxiliaries and modals. Generation and its checks belong to 12. How Mova reads this English today: pipeline, fixes, failures and 13. Concrete improvements for Mova's parser and lexicon. The corpus tool is tools/gutenberg, and the reference coreference check is coref/src/sieve.rs. Measures: round-trip exact match, lexicon attestedness rate, loop blocks, and pronoun-agreement violations per passage, all run twice for determinism.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. arXiv 2507.20111

@misc{alva2025ai,
  title = {{AI-Driven Generation of Old English: A Framework for Low-Resource Languages}},
  author = {Alva, Rodrigo Gabriel Salazar and Nunez, Matias and Lopez, Cristian and Arista, Javier Martin},
  year = {2025},
  eprint = {2507.20111},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2507.20111}
}

References

17 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

T. Sommerschield, Y. Assael, J. Pavlopoulos, V. Stefanak, A. Senior, C. Dyer, J. Bodel, J. Prag, I. Androutsopoulos, and N. de Freitas, “Machine learning for ancient languages: A survey,” Computational Linguistics, vol. 49, no. 3, pp. 703–747, 2023. ↗
J. P. W. A. diPaolo Healey and X. Xiang., “The dictionary of old english web corpus,” http://www.doe.utoronto.ca/, 2004. ↗
J. M. Arista, S. D. Barragán, L. F. Allo, L. G. Fernández, Y. H. Bghiyel, M. L. Palacios, R. M. Mendaza, C. N. Urraca, A. E. O. López, E. L. R. Narbona, R. T. Alonso, and R. V. Escarza, “Parcoroev3: An open access annotated parallel corpus old english-english,” Nerthus Project, Universidad de La Rioja, 2023, edited by J. Martı́n Arista. [Online]. Available: http://www.nerthusproject.com ↗
J. Bosworth and T. N. Toller, An Anglo-Saxon Dictionary. Oxford: Clarendon Press, 1972, supplement by T. N. Toller; Addenda and corrigenda by A. Campbell; Digital version by O. Tichy and M. Roceck. [Online]. Available: https://bosworthtoller.com ↗
K. P. Johnson, P. J. Burns, J. Stewart, T. Cook, C. Besnier, and W. J. B. Mattingly, “The Classical Language Toolkit: An NLP framework for pre-modern languages,” in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: System Demonstrations, H. Ji, J. C. Park, and R. Xia, Eds. Online: Association for Computational Linguistics, Aug. 2021, pp. 20–29. [Online]. Available: https://aclanthology.org/2021.acl-demo.3/ ↗
S. Gururangan, A. Marasović, S. Swayamdipta, K. Lo, I. Beltagy, D. Downey, and N. A. Smith, “Don’t stop pretraining: Adapt language models to domains and tasks,” 2020. [Online]. Available: https://arxiv.org/abs/2004.10964 ↗
Y. Xie, K. Aggarwal, and A. Ahmad, “Efficient continual pre-training for building domain specific large language models,” in Findings of the Association for Computational Linguistics: ACL 2024, L.-W. Ku, A. Martins, and V. Srikumar, Eds. Bangkok, Thailand: Association for Computational Linguistics, Aug. 2024, pp. 10 184–10 201. [Online]. Available: https://aclanthology.org/2024.findings-acl.606/ ↗
A. Dubey et al., “The llama 3 herd of models,” 2024. [Online]. Available: https://arxiv.org/abs/2407.21783 ↗
E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, “Lora: Low-rank adaptation of large language models,” 2021. find
S. Edunov, M. Ott, M. Auli, and D. Grangier, “Understanding back-translation at scale,” 2018. [Online]. Available: https://arxiv.org/abs/1808.09381 ↗
X.-P. Nguyen, S. M. Aljunied, S. Joty, and L. Bing, “Democratizing llms for low-resource languages by leveraging their english dominant abilities with linguistically-diverse prompts,” 2023. [Online]. Available: https://arxiv.org/abs/2306.11372v1 ↗
K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “Bleu: a method for automatic evaluation of machine translation,” in Proceedings of the 40th annual meeting of the Association for Computational Linguistics, 2002, pp. 311–318. ↗ our card
S. Banerjee and A. Lavie, “Meteor: An automatic metric for mt evaluation with improved correlation with human judgments,” in Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, 2005, pp. 65–72. find our card
M. Popović, “chrf: character n-gram f-score for automatic mt evaluation,” in Proceedings of the tenth workshop on statistical machine translation, 2015, pp. 392–395. ↗ our card
A. Q. Jiang, A. Sablayrolles, A. Mensch, C. Bamford, D. S. Chaplot, D. de las Casas, F. Bressand, G. Lengyel, G. Lample, L. Saulnier, L. R. Lavaud, M.-A. Lachaux, P. Stock, T. L. Scao, T. Lavril, T. Wang, T. Lacroix, and W. E. Sayed, “Mistral 7b,” 2023. [Online]. Available: https://arxiv.org/abs/2310.06825 ↗
P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W. tau Yih, T. Rocktäschel, S. Riedel, and D. Kiela, “Retrieval-augmented generation for knowledge-intensive nlp tasks,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 33, 2020, pp. 9459–9474. find
X. Lyu, S. Grafberger, S. Biegel, S. Wei, M. Cao, S. Schelter, and C. Zhang, “Improving retrieval-augmented large language models via data importance learning,” arXiv preprint arXiv:2307.03027, 2023. [Online]. Available: https://arxiv.org/abs/2307.03027 ↗

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/alva-2025-ai-driven-generation-old-english/