Expanding the JHU Bible Corpus for Machine Translation of the Indigenous Languages of North America
What it is about
The paper extends the JHU Bible Corpus with more than thirty Bible translations in thirty Indigenous languages of North America, normalised by verse so that they can be aligned. The languages show syntactic and morphological phenomena that are little studied. The experiments train multilingual neural translation on three families (Algic, Athabaskan, Inuit-Aleut), with high-resource languages that have complete Bibles and a lower-resource one. Training many source languages into one target, English, produces what the authors call target overfitting: the model ignores the source and outputs an unrelated English sentence. Four training strategies counter it, among them limiting training to one source at a time, heterogeneous batching, and adding English-to-English copy examples. Multilingual training helps most for the poorest languages, those with only partial Bibles. The conclusion is that cross-lingual training is clearly beneficial in this setting.
Ideas
- I1. Use target overfitting as a test for the author-delta learner: a delta trained on many source translations into one target must still depend on the source. Add a shuffled-source control; a delta that scores the same with a shuffled source is memorising the target, and must be rejected.
- I2. Identity copy as a gate: read a verse into the meta-world and write it back in the same author. The residual must be zero. This is the copy example of the paper, made into a check with a red negative control.
- I3. Weight partial translations so that a small set is not drowned by a large one. The paper's heterogeneous batching is neural, but the principle matches the translator's rule of one vote per corpus (open problem 6).
For Mova
The usable parts are the control designs (shuffled source, identity copy) and the weighting principle, which map onto open problem 6 and the gates. The multilingual neural translation is not for Mova. Summary only is enough, since the paper's corpus is used through its description.
Cite
@inproceedings{nicolai2021expanding,
title = {{Expanding the JHU Bible Corpus for Machine Translation of the Indigenous Languages of North America}},
author = {Nicolai, Garrett and Coates, Edith and Zhang, Ming and Silfverberg, Miikka},
booktitle = {Proceedings of the 4th Workshop on the Use of Computational Methods in the Study of Endangered Languages Volume 1 (Papers)},
year = {2021},
url = {https://aclanthology.org/2021.computel-1.1/}
}References
- Roee Aharoni, Melvin Johnson, and Orhan Firat. 2019. Massively multilingual neural machine translation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 3874–3884, Minneapolis, Minnesota. Association for Computational Linguistics. find
- Ehsaneddin Asgari and Hinrich Schütze. 2017. Past, present, future: A computational investigation of the Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715– 1725, Berlin, Germany. Association for Computational Linguistics. typology of tense in 1000 languages. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 113–124, Copenhagen, Denmark. Association for Computational Linguistics. find
- Christos Christodouloupoulos and Mark Steedman. 2015. A massively parallel corpus: the Bible in 100 languages. Language resources and evaluation, 49(2):375–395. find
- John Enrico. 2003. Haida syntax, volume 1. U of Nebraska Press. find
- Chung-Chian Hsu, Yan-Ping Huang, and Keng-Wei Chang. 2008. Extended Naive Bayes classifier for mixed data. Expert Systems with Applications, 35(3):1080–1083. find
- Thomas Mayer and Michael Cysouw. 2014. Creating a massively parallel Bible corpus. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 3158– 3163, Reykjavik, Iceland. European Language Resources Association (ELRA). find
- Arya D. McCarthy, Rachel Wicks, Dylan Lewis, Aaron Mueller, Winston Wu, Oliver Adams, Garrett Nicolai, Matt Post, and David Yarowsky. 2020. The Johns Hopkins university bible corpus: 1600+ tongues for typological exploration. In Proceedings of the Twelfth International Conference on Language Resources and Evaluation (LREC 2020), Marseilles, France. European Language Resources Association (ELRA). find
- Aaron Mueller, Garrett Nicolai, Arya D. McCarthy, Dylan Lewis, Winston Wu, and David Yarowsky. 2020. An analysis of massively multilingual neural machine translation for low-resource languages. In Proceedings of the Twelfth International Conference on Language Resources and Evaluation (LREC 2020), Marseilles, France. European Language Resources Association (ELRA). find
- Elke Nowak. 2011. Transforming the images: Ergativity and transitivity in Inuktitut (Eskimo), volume 15. Walter de Gruyter. find
- Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019. Fairseq: A fast, extensible toolkit for sequence modeling. In Proceedings of NAACL-HLT 2019: Demonstrations. find
- Philip Resnik, Mari Broman Olsen, and Mona Diab. 1999. The Bible as a parallel corpus: Annotating the ‘book of 2000 tongues’. Computers and the Humanities, 33(1):129–153. find
- Katherine Schmirler, Antti Arppe, Trond Trosterud, and Lene Antonsen. 2018. Building a constraint grammar parser for Plains Cree verbs and arguments. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018). find
This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/nicolai-2021-expanding-jhu-bible-corpus-machine/