Handling Syntactic Divergence in Low-resource Machine Translation

What it is about

Back-translation needs a working reverse system, so it fails when parallel data is tiny and the languages differ in word order. The authors instead make pseudo-parallel data from monolingual target text in two steps. First they reorder each English sentence into the source language's order with a rule-based method (Isozaki et al. 2010): parse, move each phrase's head to the end of the phrase, and insert special tokens after subjects and objects to stand for the Japanese case markers. Then they translate the reordered sentence word by word with a bilingual dictionary. A small LSTM model is trained on the real and the pseudo pairs. On simulated low-resource Japanese→English (3k–6k pairs) and on real Uyghur→English (LORELEI), it beats supervised NMT, back-translation and SMT. A "No-Reorder" ablation (word by word in English order) is clearly worse, and the gap grows with sentence length and is also seen in RIBES, an order-sensitive metric.

Ideas

• I1. The paper's pseudo-source is a reorder-then-gloss translation. Its English-to-Hebrew mirror, which writes an English verse in Hebrew order and glosses it word by word, gives the Hebrew reader a synthetic source with known alignments. Use it as a sanity check of hebrew.rs and ibm.rs: the aligner should recover the identity alignment on it. Measure the alignment error per class.
• I2. A Hebrew→meta order normaliser written as head-movement rules, in the opposite direction to Isozaki's (open problem 7): move the verb of a wayyiqtol clause after its subject before reading into the meta-world. The paper shows such rules can be written by someone with basic knowledge of the target. Measure the reader's order class for wlc→jps with and without it, run twice.
• I3. The "No-Reorder" ablation is the right control for any order normaliser: the same pipeline with the order step switched off. Report both, and report the gain by verse length, since the paper finds it grows with length.
• I4. Inserted case-marker tokens are function words as features in miniature: the subject and object marks stand for material that the other language writes and this one does not. Add the same marks to the meta tokens of the reader (open problem 4), so the encoder can learn which authors write the, that or a comma there.

For Mova

The neural model is not for Mova. The two-step pseudo-data and the head-movement rules are deterministic and fit tools/translate (hebrew.rs, ibm.rs). The order-sensitive metric is a reminder to report the order class of the residual on its own, as measure.rs already does.

Topics: Machine translation

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/d19-1143

@inproceedings{zhou2019handling,
  title = {{Handling Syntactic Divergence in Low-resource Machine Translation}},
  author = {Zhou, Chunting and Ma, Xuezhe and Hu, Junjie and Neubig, Graham},
  booktitle = {Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)},
  year = {2019},
  doi = {10.18653/v1/d19-1143},
  url = {https://doi.org/10.18653/v1/d19-1143}
}

References

33 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Mikel Artetxe, Gorka Labaka, and Eneko Agirre. 2018a. Unsupervised statistical machine translation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium. Association for Computational Linguistics. find
2. Mikel Artetxe, Gorka Labaka, Eneko Agirre, and Kyunghyun Cho. 2018b. Unsupervised neural machine translation. In International Conference on Learning Representations (ICLR). find
3. Ibrahim Badr, Rabih Zbib, and James Glass. 2009. Syntactic phrase reordering for English-to-Arabic statistical machine translation. In Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics, pages 86– 93. Association for Computational Linguistics. find
4. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In International Conference on Learning Representations (ICLR). find
5. Luisa Bentivogli, Arianna Bisazza, Mauro Cettolo, and Marcello Federico. 2016. Neural versus phrase-based machine translation quality: a case study. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 257–267, Austin, Texas. Association for Computational Linguistics. find
6. Arianna Bisazza and Marcello Federico. 2016. A survey of word reordering in statistical machine translation: Computational models and language phenomena. Computational linguistics, 42(2):163–205. find
7. Ondřej Bojar, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Shujian Huang, Matthias Huck, Philipp Koehn, Qun Liu, Varvara Logacheva, et al. 2017. Findings of the 2017 conference on machine translation (WMT17). In Proceedings of the Second Conference on Machine Translation, pages 169–214. find
8. Yong Cheng, Wei Xu, Zhongjun He, Wei He, Hua Wu, Maosong Sun, and Yang Liu. 2016. Semi-supervised learning for neural machine translation. The 54th Annual Meeting of the Association for Computational Linguistics (ACL). find
9. Caitlin Christianson, Jason Duncan, and Boyan Onyshkevych. 2018. Overview of the DARPA LORELEI Program. Machine Translation, 32(1-2):3–9. find
10. Michael Collins, Philipp Koehn, and Ivona Kučerová. 2005. Clause restructuring for statistical machine translation. In Proceedings of the 43rd annual meeting on association for computational linguistics, pages 531–540. Association for Computational Linguistics. find
11. Anna Currey, Antonio Valerio Miceli Barone, and Kenneth Heafield. 2017. Copied monolingual data improves low-resource neural machine translation. In Proceedings of the Second Conference on Machine Translation, pages 148–156. find
12. Pratik Jawanpuria, Arjun Balgovind, Anoop Kunchukuttan, and Bamdev Mishra. 2019. Learning multilingual word embeddings in latent metric space: a geometric approach. Transactions of the Association for Computational Linguistics (TACL). find
13. Jinhua Du and Andy Way. 2017. Pre-Reordering for Neural Machine Translation: Helpful or Harmful? The Prague Bulletin of Mathematical Linguistics, 108(1):171–182. find
14. Philipp Koehn and Rebecca Knowles. 2017. Six challenges for neural machine translation. In Proceedings of the First Workshop on Neural Machine Translation, pages 28–39, Vancouver. Association for Computational Linguistics. find
15. Chris Dyer, Jonathan Weese, Hendra Setiawan, Adam Lopez, Ferhan Ture, Vladimir Eidelman, Juri Ganitkevitch, Phil Blunsom, and Philip Resnik. 2010. cdec: A decoder, alignment, and learning framework for finite-state and context-free translation models. In Proceedings of the ACL 2010 System Demonstrations, pages 7–12. Association for Computational Linguistics. find
16. Robert C Moore and William Lewis. 2010. Intelligent selection of language model training data. In Proceedings of the ACL 2010 conference short papers, pages 220–224. Association for Computational Linguistics. find
17. Edouard Grave, Piotr Bojanowski, Prakhar Gupta, Armand Joulin, and Tomas Mikolov. 2018. Learning word vectors for 157 languages. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC-2018). find
18. Francisco Guzmán, Peng-Jen Chen, Myle Ott, Juan Pino, Guillaume Lample, Philipp Koehn, Vishrav Chaudhary, and Marc’Aurelio Ranzato. 2019. Two New Evaluation Datasets for Low-Resource Machine Translation: Nepali-English and Sinhala-English. arXiv preprint arXiv:1902.01382. ↗
19. Nizar Habash. 2007. Syntactic preprocessing for statistical machine translation. Proceedings of the 11th MT Summit, 10. find
20. Di He, Yingce Xia, Tao Qin, Liwei Wang, Nenghai Yu, Tieyan Liu, and Wei-Ying Ma. 2016. Dual learning for machine translation. In Advances in Neural Information Processing Systems (NIPs), pages 820– 828. find
21. Ann Irvine and Chris Callison-Burch. 2013. Combining bilingual and comparable corpora for low resource machine translation. In Proceedings of the eighth workshop on statistical machine translation, pages 262–270. find
22. Hideki Isozaki, Tsutomu Hirao, Kevin Duh, Katsuhito Sudoh, and Hajime Tsukada. 2010a. Automatic evaluation of translation quality for distant language pairs. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 944–952. Association for Computational Linguistics. find
23. Hideki Isozaki, Katsuhito Sudoh, Hajime Tsukada, and Kevin Duh. 2010b. Head finalization: A simple reordering rule for SOV languages. In Proceedings of the Joint Fifth Workshop on Statistical Machine Translation and MetricsMATR, pages 244–251. Association for Computational Linguistics. find
24. Toshiaki Nakazawa, Manabu Yaguchi, Kiyotaka Uchimoto, Masao Utiyama, Eiichiro Sumita, Sadao Kurohashi, and Hitoshi Isahara. 2016. ASPEC: Asian Scientific Paper Excerpt Corpus. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC 2016), pages 2204–2208, Portorož, Slovenia. find
25. Jiri Navratil, Karthik Visweswariah, and Ananthakrishnan Ramanathan. 2012. A comparison of syntactic reordering methods for English-German machine translation. Proceedings of COLING 2012, pages 2043–2058. find
26. Graham Neubig and Junjie Hu. 2018. Rapid adaptation of neural machine translation to new languages. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 875–880. Association for Computational Linguistics. find
27. Franz Josef Och and Hermann Ney. 2003. A systematic comparison of various statistical alignment models. Computational Linguistics, 29(1):19–51. find
28. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Improving Neural Machine Translation Models with Monolingual Data. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 86–96. find
29. Yoshimasa Tsuruoka, Yusuke Miyao, and Jun’ichi Tsujii. 2004. Towards efficient probabilistic HPSG parsing: integrating semantic and syntactic preference to guide the parsing. In Proceedings of the IJCNLP-04 Workshop on Beyond Shallow Analyses. ACL. find
30. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems, pages 5998–6008. find
31. Fei Xia and Michael McCord. 2004. Improving a Statistical MT System with Automatically Learned Rewrite Patterns . In Proceedings of Coling 2004, pages 508–514, Geneva, Switzerland. COLING. find
32. Jiateng Xie, Zhilin Yang, Graham Neubig, Noah A Smith, and Jaime Carbonell. 2018. Neural cross-lingual named entity recognition with minimal resources. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 369–379. find
33. Chunting Zhou, Xuezhe Ma, Di Wang, and Graham Neubig. 2019. Density Matching for Bilingual Word Embedding. In Meeting of the North American Chapter of the Association for Computational Linguistics (NAACL), Minneapolis, USA. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/zhou-2019-handling-syntactic-divergence-low-resource/