Automatic Cross-Lingual Similarization of Dependency Grammars for Tree-based Machine Translation

What it is about

Two languages may have dependency treebanks that annotate the same structure in different ways, so the trees are not isomorphic and tree-based translation suffers. The paper proposes "similarization": the annotation of each grammar is adjusted, one classification decision at a time, so that the two grammars agree more across a parallel corpus, while each grammar stays non-trivial. The consistency of a dependency relation across languages is measured with word alignments. Each step changes a decision only where the alignment-based score rises, and the two grammars are learned together in several rounds. On Chinese and English with the Penn Chinese Treebank, the cross-lingual similarity rises from about 34 per cent to about 64 per cent. A transfer classifier then corrects the output of the similarized grammar. In the tree-based MT experiment, the best configuration raises BLEU-4 from 24.6 to 26.2 by replacing the parser of the translator.

Ideas

• I1. Similarity as a measure for the meta-world: the paper's consistency score counts the share of dependency relations that agree across two languages under the alignment. Compute it for each verse pair and each author (KJV, WEB, ASV, JPS) as a residual-like table, to see where the meta-world's heads disagree (open problem 1).
• I2. Adjust decisions, not structures: the paper changes single edge decisions, not whole trees, and accepts a change only when the score rises. For the head patches of the meta-world, try single-edge changes with a count-based acceptance rule, and keep the patch only when it lowers the reader's residual bits.
• I3. Keep non-triviality as a constraint: the paper refuses grammars that become flat or single-branched. Add a check on the meta-world that refuses a reading where a verse loses all its internal heads, and count how often it fires.
• I4. Transfer classifier as a fallback: the paper uses a second classifier to correct the similarized parser. Use a count-based correction table for the reader on the verses where the reader's trees disagree with the Hebrew trees, and report the gain separately.

For Mova

The code is tools/roundtrip (the parse trees) and tools/translate/src/meta.rs (the heads of the meta-world). The method is a count-based consistency score with iterative adjustment, which fits Mova without a neural model. It also addresses open problem 1 directly, by making the head of a meta token agree across languages before the encoder reads it.

Topics: Dependency parsing · Treebanks · Machine translation

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/d16-1048

@inproceedings{jiang2016automatic,
  title = {{Automatic Cross-Lingual Similarization of Dependency Grammars for Tree-based Machine Translation}},
  author = {Jiang, Wenbin and Zhang, Wen and Xu, Jinan and Cai, Rangjia},
  booktitle = {Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing},
  year = {2016},
  doi = {10.18653/v1/d16-1048},
  url = {https://doi.org/10.18653/v1/d16-1048}
}

References

37 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Vamshi Ambati and Alon Lavie. 2008. Improving syntax driven translation models by re-structuring divergent and non-isomorphic parse tree structures. In Proceedings of Student Research Workshop of AMTA. find
2. David Burkett and Dan Klein. 2012. Transforming trees to improve syntactic convergence. In Proceedings of EMNLP-CNLL. find
3. David Chiang. 2005. A hierarchical phrase-based model for statistical machine translation. In Proceedings of the ACL. find our card
4. David Chiang. 2007. Hierarchical phrase-based translation. Computational Linguistics, pages 201–228. find our card
5. Koby Crammer, Ofer Dekel, Shai Shalev-Shwartz, and Yoram Singer. 2003. Online passive aggressive algorithms. In Proceedings of NIPS. find
6. Yuan Ding and Martha Palmer. 2005. Machine translation using probabilistic synchronous dependency insertion grammars. In Proceedings of the ACL. find our card
7. Jason M. Eisner. 1996. Three new probabilistic models for dependency parsing: An exploration. In Proceedings of COLING, pages 340–345. find our card
8. Michel Galley, Jonathan Graehl, Kevin Knight, and Daniel Marcu. 2006. Scalable inference and training of context-rich syntactic translation models. In Proceedings of the COLING-ACL. find our card
9. Bryant Huang and Kevin Knight. 2006. Relabeling syntax trees to improve syntax-based machine translation quality. In Proceedings of NAACL. find
10. Liang Huang, Kevin Knight, and Aravind Joshi. 2006. Statistical syntax-directed translation with extended domain of locality. In Proceedings of the AMTA. find our card
11. Rebecca Hwa, Philip Resnik, Amy Weinberg, and Okan Kolak. 2002. Evaluating translational correspondence using annotation projection. In Proceedings of the ACL. find
12. Philipp Koehn, Franz Och, and Daniel Marcu. 2003. Statistical phrase-based translation. In Proceedings of NAACL. find
13. Dekang Lin. 2004. A path-based transfer model for machine translation. In Proceedings of the COLING. find
14. Yang Liu, Qun Liu, and Shouxun Lin. 2006. Tree-tostring alignment template for statistical machine translation. In Proceedings of the ACL. find
15. Shujie Liu, Chi-Ho Li, Mu Li, and Ming Zhou. 2012. Re-training monolingual parser bilingually for syntactic smt. In Proceedings of EMNLP-CNLL. find
16. Mitchell P. Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of english: The penn treebank. In Computational Linguistics. find our card
17. Ryan McDonald, Koby Crammer, and Fernando Pereira. 2005. Online large-margin training of dependency parsers. In Proceedings of ACL, pages 91–98. find our card
18. Ryan McDonald, Joakim Nivre, Yvonne Quirmbach-Brundagez, Yoav Goldberg, Dipanjan Das, Kuzman Ganchev, Keith Hall, Slav Petrov, Hao Zhang, Oscar Täckström, Claudia Bedini, Nuria Bertomeu Castello, and Jungmee Leez. 2013. Universal dependency annotation for multilingual parsing. In Proceedings of ACL. find
19. Haitao Mi, Liang Huang, and Qun Liu. 2008. Forest-based translation. In Proceedings of the ACL. find our card
20. Makoto Morishita, Koichi Akabe, Yuto Hatakoshi, Graham Neubig, Koichiro Yoshino, and Satoshi Nakamura. 2015. Parser self-training for syntax-based machine translation. In Proceedings of IWSLT. find
21. Hwee Tou Ng and Jin Kiat Low. 2004. Chinese part-of-speech tagging: One-at-a-time or all-at-once? word-based or character-based? In Proceedings of EMNLP. find
22. Joakim Nivre, Johan Hall, Jens Nilsson, Gulsen Eryigit, and Svetoslav Marinov. 2006. Labeled pseudoprojective dependency parsing with support vector machines. In Proceedings of CoNLL, pages 221–225. find
23. Franz Och and Hermann Ney. 2003. A systematic comparison of various statistical alignment models. Computational Linguistics. find
24. Sinno Jialin Pan and Qiang Yang. 2010. A survey on transfer learning. IEEE TKDE. find
25. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the ACL. find our card
26. Slav Petrov, Dipanjan Das, and Ryan McDonald. 2012. A universal part-of-speech tagset. Proceedings of LREC. find
27. Chris Quirk, Arul Menezes, and Colin Cherry. 2005. Dependency treelet translation: Syntactically informed phrasal smt. In Proceedings of the ACL. find our card
28. Ratnaparkhi and Adwait. 1996. A maximum entropy part-of-speech tagger. In Proceedings of the Empirical Methods in Natural Language Processing Conference. find
29. Libin Shen, Jinxi Xu, and Ralph Weischedel. 2008. A new string-to-dependency machine translation algorithm with a target dependency language model. In Proceedings of ACL. find our card
30. David Smith and Jason Eisner. 2009. Parser adaptation and projection with quasi-synchronous grammar features. In Proceedings of EMNLP. find
31. Stolcke and Andreas. 2002. Srilm - an extensible language modeling toolkit. In Proceedings of the International Conference on Spoken Language Processing, pages 311–318. find
32. Wei Wang, Jonathan May, Kevin Knight, and Daniel Marcu. 2010. Re-structuring, re-labeling, and realignment for syntax-based machine translation. Computational Linguistics. find
33. Jun Xie, Haitao Mi, and Qun Liu. 2011. A novel dependency-to-string model for statistical machine translation. In Proceedings of EMNLP. find
34. Deyi Xiong, Qun Liu, and Shouxun Lin. 2007. A dependency treelet string correspondence model for statistical machine translation. In Proceedings of Workshop on SMT. find
35. Nianwen Xue, Fei Xia, Fu-Dong Chiou, and Martha Palmer. 2005. The penn chinese treebank: Phrase structure annotation of a large corpus. In Natural Language Engineering. find
36. H Yamada and Y Matsumoto. 2003. Statistical dependency analysis using support vector machines. In Proceedings of IWPT. find
37. Daniel Zeman, David Mareček, Martin Popel, Loganathan Ramasamy, Jan Štěpánek, Jan Hajič, and Zdeněk Žabokrtský. 2012. Hamledt: To parse or not to parse? Daniel Zeman, Ondřej Dušek, David Mareček, Martin Popel, Loganathan Ramasamy, Jan Štěpánek, Zdeněk Žabokrtský, and Jan Hajič. 2014. Hamledt: Harmonized multi-language dependency treebank. Language Resources & Evaluation. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/jiang-2016-automatic-cross-lingual-similarization-dependency/