Mova0.5 pre-release⌕
☰
home / docs / papers / Machine translation / Translation Divergences in Chinese–English Machine Translation: An Empirical Investigation

Translation Divergences in Chinese–English Machine Translation: An Empirical Investigation

What it is about

The authors take about 10,000 Chinese–English sentence pairs that have manually built parse trees on both sides. They align the two trees hierarchically, word alignments and constituents together, so that a word alignment cannot contradict a parse and create spurious divergences. On this parallel treebank they find translation divergences semi-automatically and sort them into seven types, among them lexical encoding (one-to-many alignments), transitivity, unaligned function words, category mismatch, reordering and dropped elements. The results show that divergences are far more varied than the textbook cases. Head-switching, a frequently cited example, is rare, while unaligned function words and reordering are common. Syntax-based hierarchical phrase rules capture most of the divergences, but the flat structures of existing treebanks give long rules that rarely repeat. The authors conclude that some divergences can be bridged by a shared semantic representation, while others are open-ended.

Ideas

For Mova

The code is tools/translate/meta.rs for the alignment and measure.rs for the classes. I1 is open problem 1, I2 and I4 are open problems 4 and 8. The paper's numbers are for a different language pair, so any counts must come from Mova's own Bible pairs.

Topics: Treebanks · Machine translation

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.1162/coli_a_00292

@article{deng2017translation,
  title = {{Translation Divergences in Chinese–English Machine Translation: An Empirical Investigation}},
  author = {Deng, Dun and Xue, Nianwen},
  journal = {Computational Linguistics},
  year = {2017},
  doi = {10.1162/coli\_a\_00292},
  url = {https://doi.org/10.1162/coli_a_00292}
}

References

86 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Alshawi, Hiyan, Srinivas Bangalore, and Shona Douglas. 2000. Learning dependency translation models as collections of finite-state head transducers. Computational Linguistics, 26(1):45–60. find
  2. Ambati, Vamshi and Alon Lavie. 2008. Improving syntax driven translation models by re-structuring divergent and non-isomorphic parse tree structures. In Proceedings of the Eighth Conference of the Association for Machine Translation in the Americas, pages 235–244, Waikiki Beach, HI. find
  3. Banarescu, L., C. Bonial, S. Cai, M. Georgescu, K. Griffitt, U. Hermjakob, K. Knight, P. Koehn, M. Palmer, and N. Schneider. 2013. Abstract meaning representation for sembanking. In Proceedings of LAW VII - The 7th Linguistic Annotation Workshop, pages 178–186, Sofia, Bulgaria. find our card
  4. Baran, Elizabeth, Yaqin Yang, and Nianwen Xue. 2012. Annotating dropped pronouns in Chinese newswire text. In Proceedings of LREC-2012, pages 2795–2799, Istanbul, Turkey. find
  5. Bender, Emily M., Dan Flickinger, Stephan Oepen, Woodley Packard, and Ann Copestake. 2015. Layers of interpretation: On grammar and compositionality. In Proceedings of the 11th International Conference on Computational Semantics, 239–249, London. find
  6. Bies, Ann, Mark Ferguson, Karen Katz, Robert MacIntyre, Victoria Tredinnick, Grace Kim, Mary Ann Marcinkiewicz, and Britta Schasberger. 1995. Bracketing guidelines for treebank II style Penn treebank project. Tech report MS-CIS-95-06, University of Pennsylvania. find
  7. Brown, Peter F., Vincent J. Della Pietra, Stephen A. Della Pietra, and Robert L. Mercer. 1993. The mathematics of statistical machine translation: Parameter estimation. Computational Linguistics, 19(2):263–311. find
  8. Chao, Yuenren. 1968. A Grammar of Spoken Chinese, University of California Press. find
  9. Charniak, Eugene. 2000. A maximum-entropy-inspired parser. In Proceedings of the 1st North American Chapter of the Association for Computational Linguistics Conference, pages 132–139, Seattle, Washington. find
  10. Cherry, Colin and Dekang Lin. 2003. A probability model to improve word alignment. In Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics-Volume 1, pages 88–95, Sapporo, Japan. find
  11. Cherry, Colin and Dekang Lin. 2006. Soft syntactic constraints for word alignment through discriminative training. In Proceedings of the COLING/ACL on Main Conference Poster Sessions, pages 105–112, Sydney, Australia. find
  12. Chiang, David. 2005. A hierarchical phrase-based model for statistical machine translation. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics, pages 263–270, Ann Arbor, MI. find our card
  13. Chiang, David. 2007. Hierarchical phrase-based translation. Computational Linguistics, 33(2):201–228. find our card
  14. Copestake, Ann, Dan Flickinger, Rob Malouf, Susanne Riehemann, and Ivan Sag. 1995. Translation using minimal recursion semantics. In Proceedings of the Sixth International Conference on Theoretical and Methodological Issues in Machine Translation, pages 15–32, Leuven, Belgium. find our card
  15. Copestake, Ann, Dan Flickinger, Carl Pollard, and Ivan A Sag. 2005. Minimal recursion semantics: An introduction. Research on Language and Computation, 3(2-3):281–332. find
  16. Cowan, Brooke, Ivona Kučerová, and Michael Collins. 2006. A discriminative model for tree-to-tree translation. In Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, pages 232–241, Sydney, Australia. find
  17. Dave, Shachi, Jignashu Parikh, and Pushpak Bhattacharyya. 2001. Interlingua-based English–Hindi machine translation and language divergence. Machine Translation, 16(4):251–304. find
  18. DeNero, John and Dan Klein. 2007. Tailoring word alignments to syntactic machine translation. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 17–24, Prague. find
  19. Deng, Dun and Nianwen Xue. 2014a. Aligning Chinese-English parallel parse trees: Is it feasible? In Proceedings of LAW VIII - The 8th Linguistic Annotation Worshop, pages 29–37, Dublin. find
  20. Deng, Dun and Nianwen Xue. 2014b. Annotation guidelines for hierarchically aligning Chinese-English parallel parse trees (version 1.0). Technical report, Brandeis University. find
  21. Deng, Dun and Nianwen Xue. 2014c. Building a hierarchically aligned Chinese-English parallel treebank. In Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics, pages 1511–1520, Dublin. find
  22. Deng, Dun, Nianwen Xue, and Shiman Guo. 2015. Harmonizing word alignments and syntactic structures for extracting phrasal translation equivalents. In Proceedings of the Ninth Workshop on Syntax and Structure in Statistical Translation (SSST-9), pages 1–9. find
  23. Ding, Yuan and Martha Palmer. 2005. Machine translation using probabilistic synchronous dependency insertion grammars. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics, pages 541–548, Ann Arbor, MI. find our card
  24. Dorr, Bonnie J. 1994. Machine translation divergences: A formal description and proposed solution. Computational Linguistics, 20(4):597–633. find our card
  25. Dorr, Bonnie J., Lisa Pearl, Rebecca Hwa, and Nizar Habash. 2002. Duster: A method for unraveling cross-language divergences for statistical word-level alignment. In Machine Translation: From Research to Real Users: Proceedings of the 5th Conference of the Association for Machine Translation in the Americas, AMTA 2002, page 31, Tiburon, CA. find
  26. Dorr, Bonnie Jean. 1993. Machine Translation: A View from the Lexicon. MIT Press. find
  27. Eisner, Jason. 2003. Learning non-isomorphic tree mappings for machine translation. In Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics-Volume 2, pages 205–208, Sapporo, Japan. find our card
  28. Flickinger, Dan, Stephan Oepen, and Emily M. Bender. 2017. Sustainable development and refinement of complex linguistic annotations at scale. In Nancy Ide and James Pustejovsky, editors, Handbook of Linguistic Annotation Science. Springer, Netherlands. find
  29. Flickinger, Dan, Yi Zhang, and Valia Kordoni. 2012. Deepbank. A dynamically annotated treebank of the Wall Street Journal. In Proceedings of the 11th International Workshop on Treebanks and Linguistic Theories, pages 85–96, Lisbon, Portugal. find
  30. Fossum, Victoria L. 2010. Integrating parsing and word alignment in syntax-based machine translation. Ph.D. thesis, University of Michigan. find
  31. Galley, Michel, Jonathan Graehl, Kevin Knight, Daniel Marcu, Steve DeNeefe, Wei Wang, and Ignacio Thayer. 2006. Scalable inference and training of context-rich syntactic translation models. In Proceedings of the 21st International Conference on Computational Linguistics and the 44th Annual Meeting of the Association for Computational Linguistics, pages 961–968, Sydney, Australia. find our card
  32. Galley, Michel, Mark Hopkins, Kevin Knight, and Daniel Marcu. 2004. What’s in a translation rule? In HLT-NAACL 2004: Main Proceedings, pages 273–280, Boston, MA. find our card
  33. Gupta, Deepa and Niladri Chatterjee. 2001. Study of divergence for example based English-Hindi machine translation, Kanpur, India. In STRANS-2001, pages 43–51, Kanpur. find
  34. Gupta, Deepa and Niladri Chatterjee. 2003. Identification of divergence for English to Hindi EBMT. In Proceedings of MT Summit-IX, pages 141–148, New Orleans, LA. find
  35. Hearne, Mary, John Tinsley, Ventsislav Zhechev, and Andy Way. 2007. Capturing translational divergences with a statistical tree-to-tree aligner. In Proceedings of the 11th International Conference on Theoretical and Methodological Issues in Machine Translation (TMI ’07), pages 114–121, Skövde, Sweden. find our card
  36. Hermjakob, Ulf. 2009. Improved word alignment with statistics and linguistic heuristics. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 1, pages 229–237, Singapore. find
  37. Huang, L., K. Knight, and A. Joshi. 2006. Statistical syntax-directed translation with extended domain of locality. In Proceedings of AMTA, pages 66–73, Cambridge, MA. find our card
  38. Jackendoff, Ray. 1983. Semantics and cognition. MIT Press. find
  39. Jackendoff, Ray. 1992. Semantic structures. MIT Press. find
  40. Kamp, Hans and Uwe Reyle. 1993. From Discourse to Logic: Introduction to Model Theoretic Semantics of Natural Language, Formal Logic and Discourse Representation Theory. Kluwer, Dordrecht. find
  41. Koehn, Philipp. 2009. Statistical machine translation. Cambridge University Press. find
  42. Koehn, Philipp, Franz Josef Och, and Daniel Marcu. 2003. Statistical phrase-based translation. In Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology, volume 1, pages 48–54, Edmonton, Canada. find
  43. Lavie, Alon, Alok Parlikar, and Vamshi Ambati. 2008. Syntax-driven learning of sub-sentential translation equivalents and translation rules from parsed parallel corpora. In Proceedings of the Second Workshop on Syntax and Structure in Statistical Translation, pages 87–95, Columbus, OH. find
  44. Li, Audrey Y.-H. 1999. Plurality in a classifier language. Journal of East Asian Linguistics, 8:75–99. find
  45. Li, Xuansong, Niyu Ge, and Stephanie Strassel. 2009. Tagging guidelines for Chinese-English word alignment. Technical report, Linguistic Data Consortium. find
  46. Li, Xuansong, Stephanie Strassel, Stephen Grimes, Safa Ismael, Mohamed Maamouri, Ann Bies, and Nianwen Xue. 2012. Parallel aligned treebanks at LDC: New challenges interfacing existing infrastructures. In Proceedings of LREC-2012, Istanbul. find
  47. Lindop, Jeremy and Jun-ichi Tsujii. 1991. Complex Transfer in MT: A Survey of Examples. Centre for Computational Linguistics, UMIST. find
  48. Liu, Ding and Daniel Gildea. 2008. Improved tree-to-string transducer for machine translation. In Proceedings of the Third Workshop on Statistical Machine Translation, pages 62–69, Columbus, OH. find
  49. Liu, Yang, Yun Huang, Qun Liu, and Shouxun Lin. 2007. Forest-to-string statistical translation rules. In Proceedings of ACL, pages 704–711, Prague. find
  50. Liu, Yang, Qun Liu, and Shouxun Lin. 2006. Tree-to-string alignment template for statistical machine translation. In Proceedings of COLING/ACL, pages 609–616, Sydney, Australia. find our card
  51. Liu, Yang, Yajuan Lü, and Qun Liu. 2009. Improving tree-to-tree translation with packed forests. In Proceedings of ACL/IJCNLP, pages 558–566. find
  52. Lønning, Jan Tore, Stephan Oepen, Dorothee Beermann, Lars Hellan, John Carroll, Helge Dyvik, Dan Flickinger, Janne Bondi Johannessen, Paul Meurer, Torbjørn Nordgård, et al. 2004. Logon. A Norwegian MT effort. In Proceedings of the Workshop in Recent Advances in Scandinavian Machine Translation, 6, Uppsala, Sweden. find
  53. Marcus, Mitchell, Grace Kim, Mary Ann Marcinkiewicz, Robert MacIntyre, Ann Bies, Mark Ferguson, Karen Katz, and Britta Schasberger. 1994. The Penn treebank: Annotating predicate argument structure. In Proceedings of the workshop on Human Language Technology, pages 114–119, Plainsboro, NJ. find our card
  54. Marcus, Mitchell P., Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of English: The Penn treebank. Computational Linguistics, 19(2):313–330. find our card
  55. May, Jonathan and Kevin Knight. 2007. Syntactic re-alignment models for machine translation. In EMNLP-CoNLL, pages 360–368, Prague. find
  56. Melamed, I. Dan. 1998. Annotation style guide for the blinker project. University of Pennsylvania. Institute for Research in Cognitive Science Technical Report No. IRCS-98-06. find
  57. Mi, Haitao and Liang Huang. 2008. Forest-based translation rule extraction. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 206–214, Honolulu, HI. find
  58. Mi, Haitao, Liang Huang, and Qun Liu. 2008. Forest-based translation. In Proceedings of ACL-08: HLT, pages 192–199. find our card
  59. Mishra, Vimal and R. B. Mishra. 2009. Divergence patterns between English and Sanskrit machine translation. Journal of Computer Science, 8(3):62–71. find
  60. Och, Franz Josef. 1999. An efficient method for determining bilingual word classes. In Proceedings of the Ninth Conference of the European Chapter of the Association for Computational Linguistics (EACL ’99), pages 71–76, Bergen, Norway. find
  61. Oepen, Stephan, Kristina Toutanova, Stuart Shieber, Christopher Manning, Dan Flickinger, and Thorsten Brants. 2002. The lingo redwoods treebank motivation and preliminary applications. In Proceedings of the 19th International Conference on Computational Linguistics-Volume 2, pages 1–5, Taipei, Taiwan. find our card
  62. Oepen, Stephan, Erik Velldal, Jan Tore Lønning, Paul Meurer, Victoria Rosén, and Dan Flickinger. 2007. Towards hybrid quality-oriented machine translation— On linguistics and probabilities in MT. In Proceedings of the 11th Conference on Theoretical and Methodological Issues in Machine Translation, pages 144–153, Skövde, Sweden. find
  63. Petrov, Slav and Dan Klein. 2007. Improved inference for unlexicalized parsing. In Proceedings of HLT-NAACL, pages 404–411, Rochester, NY. find
  64. Riesa, Jason, Ann Irvine, and Daniel Marcu. 2011. Feature-rich language-independent syntax-based alignment for statistical machine translation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 497–507, Edinburgh, Scotland. find
  65. Riesa, Jason and Daniel Marcu. 2010. Hierarchical search for word alignment. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 157–166, Uppsala, Sweden. find
  66. Saboor, A. and M. A. Khan. 2010. Lexical-semantic divergence in Urdu-to-English example based machine translation. In 6th International Conference on Emerging Technologies (ICET), 2010, pages 316–320, Chengdu, China. find
  67. Shen, Libin, Jinxi Xu, and Ralph M. Weischedel. 2008. A new string-to-dependency machine translation algorithm with a target dependency language model. In Proceedings of ACL-08:HLT, pages 577–585, Columbus, OH. find our card
  68. Sinha, R., K. Mahesh, and Anil Thakur. 2005a. Divergence patterns in machine translation between Hindi and English. In Proceedings of the 10th Machine Translation Summit (MT Summit X), pages 346–353, Phuket. find
  69. Sinha, R., K. Mahesh, and Anil Thakur. 2005b. Translation divergence in English-Hindi MT. In Proceeding of EAMT Xth Annual Conference, pages 245–254, Budapest. find
  70. Stymne, Sara and Lars Ahrenberg. 2006. A bilingual grammar for translation of English-Swedish verb frame divergences. In Proceedings of EAMT, pages 9–18, Oslo, Norway. find
  71. Sun, Jun, Min Zhang, and Chew Lim Tan. 2010. Exploring syntactic structural features for sub-tree alignment using bilingual tree kernels. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 306–315, Uppsala, Sweden. find
  72. Tinsley, John, Ventsislav Zhechev, Mary Hearne, and Andy Way. 2007. Robust language pair-independent subtree alignment. In Proceedings of Machine Translation Summit XI, pages 467–474, Copenhagen. find
  73. Vauquois, Bernard. 1968. A survey of formal grammars and algorithms for recognition and transformation in machine translation. In Proceedings of the IFIP Congress-6, pages 254–260, Edinburgh, Scotland. find
  74. Wang, Zhiguo and Nianwen Xue. 2014. Joint POS tagging and transition-based constituent parsing in Chinese with non-local features. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 733–742, Baltimore, MD. find
  75. Wang, Zhiguo and Chengqing Zong. 2013. Large-scale word alignment using soft dependency cohesion constraints. Transactions of the Association for Computational Linguistics, 1:291–300. find
  76. Warner, Colin, Ann Bies, Christine Brisson, and Justin Mott. 2004. Addendum to the Penn treebank II style bracketing guidelines: Biomedical treebank annotation. Technical report, University of Pennsylvania. find
  77. Wu, Dekai. 1997. Stochastic inversion transduction grammars and bilingual parsing of parallel corpora. Computational Linguistics, 23(3):377–403. find our card
  78. Xue, Nianwen, Ondrej Bojar, Jan Hajic, Martha Palmer, Zdenka Uresova, and Xiuhong Zhang. 2014. Not an interlingua, but close: Comparison of English AMRS to Chinese and Czech. In LREC, pages 1765–1772, Reykjavik, Iceland. find our card
  79. Xue, Nianwen and Fei Xia. 1998. The bracketing guidelines for Penn Chinese treebank project. Technical Report IRCS-00-08. University of Pennsylvania. find
  80. Xue, Nianwen, Fei Xia, Fudong Chiou, and Martha Palmer. 2005. The Penn Chinese treebank: Phrase structure annotation of a large corpus. Natural Language Engineering, 11(2):207–238. find
  81. Yang, Yaqin and Nianwen Xue. 2010. Chasing the ghost: Recovering empty categories in the Chinese treebank. In Proceedings of the 23rd International Conference on Computational Linguistics: Posters, pages 1382–1390, Beijing, China. find
  82. Zhang, Min, Hongfei Jiang, Aiti Aw, Haizhou Li, Chew Lim Tan, and Sheng Li. 2008. A tree sequence alignment-based tree-to-tree translation model. In Proceedings of ACL-08:HLT, pages 559–567, Columbus, OH. find
  83. Zhang, Xiuhong and Nianwen Xue. 2012. Extending and scaling up the Chinese treebank annotation. In Proceedings of the Second CIPS-SIGHAN Joint Conference on Chinese Language Processing, pages 27–34, Tianjin. find
  84. Zhu, Jingbo, Qiang Li, and Tong Xiao. 2015. Improving syntactic rule extraction through deleting spurious links with translation span alignment. Natural Language Engineering, 21(2):227–249. find
  85. Zollmann, Andreas and Ashish Venugopal. 2006. Syntax augmented machine translation via chart parsing. In Proceedings of the SMT Workshop HLT-NAACL, New York, NY. find
  86. Zollmann, Andreas, Ashish Venugopal, Franz Josef Och, and Jay Ponte. 2008. A systematic comparison of phrase-based, hierarchical and syntax-augmented statistical MT. In Proceedings of COLING 2008, the 22nd International Conference on Computational Linguistics, volume 1, pages 1145–1152, Manchester, UK. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/deng-2017-translation-divergences-chinese-english-machine/