Large-Scale Machine Translation between Arabic and Hebrew: Available Corpora and Initial Results

What it is about

The authors test Arabic-to-Hebrew machine translation on the WIT3 TED talks corpus (about 0.2M training sentences, 3.2M Arabic words), with the IWSLT 2016 tuning and test splits. They compare a phrase-based system (Moses) with an attention-based neural system, and add two kinds of morphology handling: external Arabic tokenizers (Farasa, MADAMIRA) and a character-level convolutional input. Tokenizers help phrase-based translation only slightly; the character model with unknown-word replacement gives the best BLEU (10.86, against 9.31 for the baseline) and the best Meteor (33.61). The paper also surveys the available Arabic-Hebrew corpora, which are mostly OpenSubtitles and small localisation files. Only the Arabic-to-Hebrew direction is tested.

Ideas

• I1. A character-level fallback for unknown tokens: the paper shows that character models can rival morphology-aware tokenizers. Mova's segmenter in tools/hbo can use a finite character n-gram model for tokens missing from its lex and pre tables. This stays deterministic and does not need a neural net.
• I2. Report significance with every score: the authors use paired significance tests on BLEU and note that Meteor and BLEU disagree. Mova's gains should be shown with the same kind of test, on two runs.
• I3. Use the paper's clitic list as a check: the Arabic and Hebrew clitics (conjunction, preposition, article) are separated the same way. Compare the prefix table of tools/hbo against that split on a sample.

For Mova

I1 and I3 apply to tools/hbo, where I3 checks the prefix tables against the clitic patterns in Pronouns, particles and prepositions. I2 is the measurement rule for any Hebrew result Mova reports. No data from the paper is carried into the repo.

Topics: Machine translation

Cite

The paper itself, not this card. BibTeX from the library's record of the work. arXiv 1609.07701

@misc{belinkov2016large,
  title = {{Large-Scale Machine Translation between Arabic and Hebrew: Available Corpora and Initial Results}},
  author = {Belinkov, Yonatan and Glass, James},
  year = {2016},
  eprint = {1609.07701},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1609.07701}
}

References

23 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ahmed Abdelali, Kareem Darwish, Nadir Durrani, and Hamdy Mubarak. 2016. Farasa: A find
2. Ibrahim Badr, Rabih Zbib, and James Glass. 2008. Segmentation for English-to-Arabic Statistical Machine Translation. In Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics on Human Language Technologies: Short Papers, HLT-Short ’08, pages 153–156, Columbus, Ohio. find
3. Mauro Cettolo, Christian Girardi, and Marcello Federico. 2012. WIT 3 : Web Inventory of Transcribed and Translated Talks. In Proceedings of the 16 th Conference of the European Association for Machine Translation (EAMT), pages 261–268, Trento, Italy. find
4. Mauro Cettolo, Jan Niehues, Sebastian Stüker, Luisa Bentivogli, and Marcello Federico. 2014. Report on the 11th IWSLT evaluation campaign, IWSLT 2014. In Proceedings of the International Workshop on Spoken Language Translation, Hanoi, Vietnam. find
5. Mauro Cettolo. 2016. An Arabic-Hebrew parallel corpus of TED talks. In Proceedings of the AMTA 2016 Workshop on Semitic Machine Translation (SeMaT), Austin, Texas. find our card
6. Ronan Collobert, Koray Kavukcuoglu, and Clément Farabet. 2011. Torch7: A Matlab-like Environment for Machine Learning. In BigLearn, NIPS Workshop. find
7. Michael Denkowski and Alon Lavie. 2014. Meteor Universal: Language Specific Translation Evaluation for Any Target Language. In Proceedings of the EACL 2014 Workshop on Statistical Machine Translation. find
8. Jacob Devlin, Rabih Zbib, Zhongqiang Huang, Thomas Lamar, Richard Schwartz, and John Makhoul. 2014. Fast and Robust Neural Network Joint Models for Statistical Machine Translation. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1370–1380, Baltimore, Maryland. find
9. Chris Dyer, Victor Chahuneau, and Noah A. Smith. 2013. A Simple, Fast, and Effective Reparameterization of IBM Model 2. In Proceedings [Koehn2004] Philipp Koehn. 2004. Statistical Sig-of the 2013 Conference of the North American Chapter nificance Tests for Machine Translation Evaluation. of the Association for Computational Linguistics: Hu-In Proceedings of EMNLP 2004, pages 388–395, man Language Technologies, pages 644–648, Atlanta, Barcelona, Spain. Georgia. find
11. Ahmed El Kholy and Nizar tani, Erik Peterson, and Katharina Probst. 2004. Habash. 2012. Orthographic and morphological pro-Rapid prototyping of a transfer-based Hebrew-to-cessing for English–Arabic statistical machine transla-English machine translation system. In 10th Internation. Machine Translation, 26(1):25–45. tional Conference on Theoretical and Methodological find
12. Ahmed El Kholy and Nizae Issues in Machine Translation (TMI). Habash. 2014. Alignment Symmetrization Optimiza- [Lembersky et al.2012] Gennadi Lembersky, Noam Ortion Targeting Phrase Pivot Statistical Machine Trans-dan, and Shuly Wintner. 2012. Language Models for lation. In Proceedings of The European Association Machine Translation: Original vs. Translated Texts. for Machine Translation (EAMT14). Computational Linguistics, 38(4):799–826. find
13. Ahmed El Kholy and Nizar [Lison and Tiedemann2016] Pierre Lison and Jörg Tiede-Habash. 2015. Morphological Constraints for Phrase mann. 2016. OpenSubtitles2016: Extracting Large Pivot Statistical Machine Translation. Proceedings of Parallel Corpora from Movie and TV Subtitles. In MT Summit XV, page 104. Proceedings of the Tenth International Conference on find
14. Nizar Habash and Fatiha Sadat. Language Resources and Evaluation (LREC 2016), 2006. Arabic Preprocessing Schemes for Statisti-Portoroz̆, Slovenia. cal Machine Translation. In Proceedings of the Hu- find
15. Arfath Pasha, Mohamed Alman Language Technology Conference of the NAACL, Badrashiny, Mona Diab, Ahmed El Kholy, Ramy Companion Volume: Short Papers, NAACL-Short ’06, Eskander, Nizar Habash, Manoj Pooleery, Owen pages 49–52, New York, New York. Rambow, and Ryan Roth. 2014. MADAMIRA: A find
16. Kenneth Heafield, Ivan Fast, Comprehensive Tool for Morphological Analysis Pouzyrevsky, Jonathan H. Clark, and Philipp and Disambiguation of Arabic. In Proceedings of Koehn. 2013. Scalable Modified Kneser-Ney Lan-the Ninth International Conference on Language guage Model Estimation. In Proceedings of the 51st Resources and Evaluation (LREC’14), Reykjavik, Annual Meeting of the Association for Computational Iceland. Linguistics, pages 690–696, Sofia, Bulgaria. [Riezler and Maxwell2005] Stefan Riezler and John T find
17. Sébastien Jean, Kyunghyun Cho, Maxwell. 2005. On some pitfalls in automatic evalu-Roland Memisevic, and Yoshua Bengio. 2015. On ation and significance testing for MT. In Proceedings Using Very Large Target Vocabulary for Neural of the ACL workshop on intrinsic and extrinsic evalua-Machine Translation. In Proceedings of the 53rd tion measures for machine translation and/or summa-Annual Meeting of the Association for Computational rization, pages 57–64. Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long [Sajjad et al.2013] Hassan Sajjad, Francisco Guzmán, Preslav Nakov, Ahmed Abdelali, Kenton Murray, Fa-Papers), pages 1–10, Beijing, China. had Al Obaidli, and Stephan Vogel. 2013. QCRI find
18. Yoon Kim, Yacine Jernite, David Sonat IWSLT 2013: Experiments in Arabic-English and tag, and Alexander Rush. 2016. Character-Aware English-Arabic Spoken Language Translation. In Pro-Neural Language Models. In AAAI Conference on Ar-ceedings of the 10th International Workshop on Spo-tificial Intelligence. ken Language Translation (IWSLT’13), volume 13, find
19. Yoon Kim. 2016. Seq2seq-attn. Heidelberg, Germany. https://github.com/harvardnlp/seq2seq-attn. [Sennrich et al.2016] Rico Sennrich, Barry Haddow, and ↗
20. Philipp Koehn, Hieu Hoang, Alexan-Alexandra Birch. 2016. Neural Machine Translation dra Birch, Chris Callison-Burch, Marcello Federico, of Rare Words with Subword Units. In Proceedings of Nicola Bertoldi, Brooke Cowan, Wade Shen, Chris-the 54th Annual Meeting of the Association for Com-tine Moran, Richard Zens, Chris Dyer, Ondřej Bo-putational Linguistics (Volume 1: Long Papers), pages jar, Alexandra Constantin, and Evan Herbst. 2007. 1715–1725, Berlin, Germany. Moses: Open Source Toolkit for Statistical Machine Translation. In Proceedings of the 45th Annual Meet- [Shilon et al.2012] Reshef Shilon, Nizar Habash, Alon ing of the ACL on Interactive Poster and Demonstra-Lavie, and Shuly Wintner. 2012. Machine Translation tion Sessions, ACL ’07, pages 177–180, Prague, Czech Between Hebrew and Arabic. Machine Translation, Republic. 26(1-2):177–195. find
21. Nimesh Singh and Nizar Habash. 2012. Hebrew Morphological Preprocessing for Statistical Machine Translation. In Proceedings of The European Association for Machine Translation (EAMT12). find our card
22. Jörg Tiedemann. 2009. News from OPUS - A Collection of Multilingual Parallel Corpora with Tools and Interfaces. In Recent Advances in Natural Language Processing, volume V, pages 237–248. Borovets, Bulgaria. find
23. Jörg Tiedemann. 2012. Parallel Data, Tools and Interfaces in OPUS. In Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC’12), Istanbul, Turkey. find
24. Jörg Tiedemann. 2016. Finding Alternative Translations in a Large Corpus of Movie Subtitle. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016), Portoroz̆, Slovenia. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/langstream-2016-belinkov-large-scale-machine-translation-between/