A Hierarchical Phrase-Based Model for Statistical Machine Translation

What it is about

Chiang extends phrase-based translation with hierarchical phrases: phrase pairs that contain gaps filled by other phrase pairs. The model is a synchronous context-free grammar with one nonterminal, learned from a bitext with no syntactic annotation. Rules are extracted from word-aligned sentence pairs by taking phrase pairs and replacing sub-phrases by gaps, and the grammar gets a glue rule that combines phrases in order. The experiments are on Mandarin-English, with FBIS training data (about 16 million words), a trigram language model on 155 million words, and the 2002 and 2003 NIST sets for development and test. The baseline is the phrase system Pharaoh. The hierarchical system reaches BLEU-4 0.2877 against 0.2676 for Pharaoh, a relative gain of 7.5 percent. The grammar has 24 million rules, which the filter for the development set cuts to 2.2 million. Adding syntactic constituent information does not give a significant further gain.

Ideas

• I1. One gap in an encoder chunk. Allow a meta token's chunk to contain one gap filled by another meta token (the X of Y). This covers contexts that are unseen as full chunks and cuts the bare forms that morph supplies for unseen concepts (encoder, open problem 3).
• I2. Separate weights for the order-keeping rules and the reordering rules. The paper tunes the glue rule apart from the ordinary rules and finds it penalised less. The Mova analogue is a separate weight for the straight order against the Hebrew normalised order, tuned on dev only.
• I3. Keep only the rules that match the dev verses, as the paper does with its filter. For the residual-driven lexicon (open problem 2), keep a merge only if it occurs in the verses it is meant to read; the rest stay out of the table.
• I4. Report the gain with a significance test (bootstrap, as the paper does), on the same test split, and run the pipeline twice before any claim.

For Mova

The code is tools/translate, in the encoder (profile.rs) and the lexicon (meta.rs). The paper's grammar is a count-based synchronous grammar, so the gap rule can be stored as counts in ordered maps without any neural part. Measure first the number of encoder contexts that fall back to a bare form, which is what the gap rule would reduce.

Topics: Machine translation

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.3115/1219840.1219873

@inproceedings{chiang2005hierarchical,
  title = {{A Hierarchical Phrase-Based Model for Statistical Machine Translation}},
  author = {Chiang, David},
  booktitle = {Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL'05)},
  year = {2005},
  doi = {10.3115/1219840.1219873},
  url = {https://doi.org/10.3115/1219840.1219873}
}

References

22 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. A. V. Aho and J. D. Ullman. 1969. Syntax directed translations and the pushdown assembler. Journal of Computer and System Sciences, 3:37–56. find
2. Daniel M. Bikel and David Chiang. 2000. Two statistical parsing models applied to the Chinese Treebank. In Proceedings of the Second Chinese Language Processing Workshop, pages 1–6. find
3. Hans Ulrich Block. 2000. Example-based incremental synchronous interpretation. In Wolfgang Wahlster, editor, Verbmobil: Foundations of Speech-to-Speech Translation, pages 411–417. Springer-Verlag, Berlin. find
4. Peter F. Brown, Stephen A. Della Pietra, Vincent J. Della Pietra, and Robert L. Mercer. 1993. The mathematics of statistical machine translation: Parameter estimation. Computational Linguistics, 19:263–311. find
5. Stanley F. Chen and Joshua Goodman. 1998. An empirical study of smoothing techniques for language modeling. Technical Report TR-10-98, Harvard University Center for Research in Computing Technology. find
6. Philipp Koehn, Franz Josef Och, and Daniel Marcu. 2003. Statistical phrase-based translation. In Proceedings of HLT-NAACL 2003, pages 127–133. find
7. Philipp Koehn. 2003. Noun Phrase Translation. Ph.D. thesis, University of Southern California. find
8. Philipp Koehn. 2004a. Pharaoh: a beam search decoder for phrase-based statistical machine translation models. In Proceedings of the Sixth Conference of the Association for Machine Translation in the Americas, pages 115–124. find
9. Philipp Koehn. 2004b. Statistical significance tests for machine translation evaluation. In Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 388–395. find
10. Shankar Kumar, Yonggang Deng, and William Byrne. 2005. A weighted finite state transducer translation template model for statistical machine translation. Natural Language Engineering. To appear. find
11. Daniel Marcu and William Wong. 2002. A phrase-based, joint probability model for statistical machine translation. In Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 133–139. find
12. Franz Josef Och and Hermann Ney. 2000. Improved statistical alignment models. In Proceedings of the 38th Annual Meeting of the ACL, pages 440–447. find
13. Franz Josef Och and Hermann Ney. 2002. Discriminative training and maximum entropy models for statistical machine translation. In Proceedings of the 40th Annual Meeting of the ACL, pages 295–302. find
14. Franz Josef Och and Hermann Ney. 2004. The alignment template approach to statistical machine translation. Computational Linguistics, 30:417–449. find
15. Franz Josef Och, Ignacio Thayer, Daniel Marcu, Kevin Knight, Dragos Stefan Munteanu, Quamrul Tipu, Michel Galley, and Mark Hopkins. 2004. Arabic and Chinese MT at USC/ISI. Presentation given at NIST Machine Translation Evaluation Workshop. find
16. Franz Josef Och. 2003. Minimum error rate training in statistical machine translation. In Proceedings of the 41st Annual Meeting of the ACL, pages 160–167. find
17. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. B: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the ACL, pages 311–318. find
18. Andreas Stolcke. 2002. SRILM – an extensible language modeling toolkit. In Proceedings of the International Conference on Spoken Language Processing, volume 2, pages 901–904. find
19. Dekai Wu. 1997. Stochastic inversion transduction grammars and bilingual parsing of parallel corpora. Computational Linguistics, 23:377–404. find our card
20. Kenji Yamada and Kevin Knight. 2001. A syntax-based statistical translation model. In Proceedings of the 39th Annual Meeting of the ACL, pages 523–530. find our card
21. Richard Zens and Hermann Ney. 2004. Improvements in phrase-based statistical machine translation. In Proceedings of HLT-NAACL 2004, pages 257–264. find
22. Ying Zhang, Stephan Vogel, and Alex Waibel. 2004. Interpreting BLEU/NIST scores: How much improvement do we need to have a better system? In Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC), pages 2051– 2054. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/chiang-2005-hierarchical-phrase-based-model-statistical/