Context Volume Drives Performance: Tackling Domain Shift in Extremely Low-Resource Translation via RAG

What it is about

The authors translate Dhao, an Indonesian language with a New Testament but no digital footprint beyond it, into the Old Testament, which is a domain shift. An NLLB-600M model fine-tuned on 7,644 New Testament verses drops from 36.17 chrF++ in domain to 27.11 on the first 500 verses of Genesis. A large LLM (Gemini 2.5 Flash) then post-edits the NMT draft, with retrieved sentences from the Bible and a grammar book, and retrieved lexicon entries. The final system reaches 35.21 chrF++, close to the in-domain score, but spBLEU stays low. The main finding is that the number of retrieved examples drives the gain more than the retrieval algorithm (lexical, semantic, or per-word fuzzy matching), and the LLM also repairs catastrophic NMT failures such as repetition loops.

Ideas

• I1. Control for volume before claiming a retrieval gain. The paper shows that size of context can explain a result that looks like an algorithm effect. Any Mova retrieval experiment should match context size across the arms (an ablation by volume).
• I2. Per-word fuzzy lexicon lookup: for each source word, return the top entries whose normalised Levenshtein similarity is at least 0.5. For unpointed Hebrew, test this against the lex table of tools/hbo on Genesis, as a deterministic table lookup in Rust.
• I3. A deterministic check for repetition loops and hallucinated long repeats in teacher output (the paper's "safety net" failures). Mova can flag these in drafts from the large-model teacher before they are used as labels.

For Mova

I1 is a rule for any retrieval test in prag and world. I2 goes into The lexicon and tools/hbo (lexicon lookup). I3 belongs to the teacher pipeline, not to the model: it filters what the teacher writes before training. The paper is about machine translation with an LLM, so the rest of it has no direct use for Mova.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.loresmt-1.7

@inproceedings{setiawan2026context,
  title = {{Context Volume Drives Performance: Tackling Domain Shift in Extremely Low-Resource Translation via RAG}},
  author = {Setiawan, David Samuel and Merx, Raphaël and Lau, Jey Han},
  booktitle = {Proceedings for the Ninth Workshop on Technologies for Machine Translation of Low Resource Languages (LoResMT 2026)},
  year = {2026},
  doi = {10.18653/v1/2026.loresmt-1.7},
  url = {https://doi.org/10.18653/v1/2026.loresmt-1.7}
}

References

25 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Vesa Akerman, David Baines, Damien Daspit, Ulf Hermjakob, Taeho Jang, Colin Leong, Michael Martin, Joel Mathew, Jonathan Robie, and Marcus Schwarting. 2023. The ebible corpus: Data and model benchmarks for bible translation for low-resource languages. arXiv preprint arXiv:2304.09919. ↗ our card
2. Jermy Immanuel Balukh. 2020. A grammar of Dhao: An endangered Austronesian language in Eastern Indonesia. LOT dissertation series. Isaac Caswell, Elizabeth Nielsen, Jiaming Luo, Colin Cherry, Geza Kovacs, Hadar Shemtov, Partha Talukdar, Dinesh Tewari, Baba Mamadi Diane, Djibrila Diane, Solo Farabado Cissé, Koulako Moussa Doumbouya, Edoardo Ferrante, Alessandro Guasoni, Christopher Homan, Mamadou K. Keita, Sudhamoy DebBarma, Ali Kuzhuget, David Anugraha, and 5 others. 2025. SMOL: Professionally translated parallel data for 115 under-represented languages. In Proceedings of the Tenth Conference on Machine Translation, pages 1103–1123, Suzhou, China. of the Tenth Conference on Machine Translation, pages 355–413, Suzhou, China. Association for Computational Linguistics. find
3. Philipp Koehn and Rebecca Knowles. 2017. Six challenges for neural machine translation. In Proceedings of the First Workshop on Neural Machine Translation, pages 28–39, Vancouver. find
4. Chenhui Chu and Rui Wang. 2020. A survey of domain adaptation for machine translation. Journal of information processing, 28:413–426. find
5. Yue Li, Zhixue Zhao, and Carolina Scarton. 2025. It’s all about in-context learning! teaching extremely low-resource languages to LLMs. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 29532–29547, Suzhou, China. find
6. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, and 1 others. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv preprint arXiv:2507.06261. ↗
7. Dianqing Lin, Aruukhan, Hongxu Hou, Shuo Sun, Wei Chen, Yichen Yang, and Guodong Shi. 2025. Can large language models translate unseen languages in underrepresented scripts? In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 23148–23161, Suzhou, China. find
8. Marta Ruiz Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loïc Barrault, Gabriel Mejia Gonzalez, Prangthip Hansanti, John Hoffman, and 19 others. 2024. Scaling neural machine translation to 200 languages. Nature, 630:841 – 846. find
9. Ali Marashian, Enora Rice, Luke Gessler, Alexis Palmer, and Katharina von der Wense. 2025. From priest to doctor: Domain adaptation for low-resource neural machine translation. In Proceedings of the 31st International Conference on Computational Linguistics, pages 7087–7098, Abu Dhabi, UAE. find
10. Barry Haddow, Rachel Bawden, Antonio Valerio Miceli Barone, Jindřich Helcl, and Alexandra Birch. 2022. Survey of low-resource machine translation. Computational Linguistics, 48(3):673–732. find
11. Mathias Müller, Annette Rios, and Rico Sennrich. 2020. Domain robustness in neural machine translation. In Proceedings of the 14th Conference of the Association for Machine Translation in the Americas (Volume 1: Research Track), pages 151–164, Virtual. Association for Machine Translation in the Americas. find
12. Amr Hendy, Mohamed Abdelrehim, Amr Sharaf, Vikas Raunak, Mohamed Gabr, Hitokazu Matsushita, Young Jin Kim, Mohamed Afify, and Hany Hassan Awadalla. 2023. How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation. arXiv preprint. ArXiv:2302.09210 [cs]. ↗
13. Elizabeth Nielsen, Isaac Rayburn Caswell, Jiaming Luo, and Colin Cherry. 2025. Alligators all around: Mitigating lexical confusion in low-resource machine translation. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers), pages 206–221, Albuquerque, New Mexico. find
14. Junjie Hu, Mengzhou Xia, Graham Neubig, and Jaime Carbonell. 2019. Domain adaptation of neural machine translation by lexicon induction. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2989–3001, Florence, Italy. find
15. Eric Khiu, Hasti Toossi, David Anugraha, Jinyu Liu, Jiaxu Li, Juan Flores, Leandro Roman, A. Seza Doğruöz, and En-Shiun Lee. 2024. Predicting machine translation performance on low-resource languages: The role of domain similarity. In Findings of the Association for Computational Linguistics: EACL 2024, pages 1474–1486, St. Julian’s, Malta. find
16. Maja Popović. 2017. chrF++: words helping character n-grams. In Proceedings of the Second Conference on Machine Translation, pages 612–618, Copenhagen, Denmark. find our card
17. Surangika Ranathunga, En-Shiun Annie Lee, Marjana Prifti Skenduli, Ravi Shekhar, Mehreen Alam, and Rishemjit Kaur. 2023. Neural machine translation for low-resource languages: A survey. ACM Computing Surveys, 55(11):1–37. find
18. Tom Kocmi, Ekaterina Artemova, Eleftherios Avramidis, Rachel Bawden, Ondřej Bojar, Konstantin Dranch, Anton Dvorkovich, Sergey Dukanov, Mark Fishel, Markus Freitag, Thamme Gowda, Roman Grundkiewicz, Barry Haddow, Marzena Karpinska, Philipp Koehn, Howard Lakougna, Jessica Lundin, Christof Monz, Kenton Murray, and 10 others. 2025. Findings of the WMT25 general machine translation shared task: Time to stop evaluating on easy test sets. In Proceedings Vikas Raunak, Arul Menezes, and Marcin Junczys-Dowmunt. 2021. The curious case of hallucinations in neural machine translation. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1172–1183, Online. Association for Computational Linguistics. find
19. A Nathaniel Robinson, Perez Ogayo, David R. Mortensen, and Graham Neubig. 2023. ChatGPT MT: Competitive for high- (but not low-) resource languages. In Proceedings of the Eighth Conference on Machine Translation, pages 392–418, Singapore. Domain Shift Analysis To quantify the lexical divergence between the New Testament (NT) and Old Testament (OT), we analyzed the frequency of domain-specific terms and the Out-of-Vocabulary (OOV) rates. find
20. SIL International. 2025. Ethnologue: Languages of the world. https://www.ethnologue.com/. Accessed: August 14, 2025. ↗
21. Garrett Tanzer, Mirac Suzgun, Eline Visser, Dan Jurafsky, and Luke Melas-Kyriazi. 2024. A benchmark for learning to translate a new language from one grammar book. In International Conference on Representation Learning, volume 2024, pages 18955–18985. find
22. Raúl Vázquez, Yves Scherrer, Sami Virpioja, and Jörg Tiedemann. 2021. The Helsinki submission to the AmericasNLP shared task. In Proceedings of the First Workshop on Natural Language Processing for Indigenous Languages of the Americas, pages 255– 264, Online. find
23. Chaojun Wang and Rico Sennrich. 2020. On exposure bias, hallucination and domain shift in neural machine translation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3544–3552, Online. Association for Computational Linguistics. *Figure 5: The Out-of-Vocabulary (OOV) rate of the in-domain NT Validation set (8.1%) versus the out-of-domain OT Test set (25.9%). All rates are calculated relative to the NT training vocabulary.* Wycliffe Global Alliance. 2025. 2025 global scripture access. https://wycliffe.net/resources/ statistics/. Accessed: 2025-12-10. ↗
24. Shitao Xiao, Zheng Liu, Peitian Zhang, and Niklas Muennighoff. 2023. C-pack: Packaged resources to advance general chinese embedding. Preprint, arXiv:2309.07597. ↗
25. Armel Randy Zebaze, Benoît Sagot, and Rachel Bawden. 2025. Compositional translation: A novel LLM-based approach for low-resource machine translation. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 22328–22357, Suzhou, China. *Figure 6: Domain-specific term frequencies (normalized per 10k words) comparing the Old Testament (OT) and New Testament (NT) corpora. Note the prevalence of historical terms in the OT versus theological terms in the NT.* find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/lingbulk-2026-setiawan-context-volume-drives-performance-tackling/