Feature-Rich Translation by Quasi-Synchronous Lattice Parsing

What it is about

The paper presents a log-linear translation model that puts phrase features, target n-gram and dependency-syntax features, and tree-to-tree configuration features into one score. The decoder is lattice parsing: a sausage lattice of candidate target words is parsed with a quasi-synchronous dependency grammar of the German source tree. Non-local features such as trigram and phrase features are handled by cube summing, an approximate method. On the BTEC German-English task with about 82K training sentences, adding phrase features to the base model raises BLEU from about 0.37 to 0.45, and adding target syntax reaches about 0.50. Adding the configuration features gives about 0.51. Restricting which configurations are allowed matters: strict synchronous alignment scores 0.40, and allowing parent-child and sibling configurations reaches the best value. Permitting every configuration does not help further. The lexicon keeps the top three translations for each source word, and 191 of 934 development sentences could not be reached by the model.

Ideas

• I1. Coverage as a checked count: the decoder cannot enforce that each source word is used once, so it needs coverage features. The translator's reader should report the same residual: the count of source tokens read zero times and read twice, per verse, as a hard check on the meta-world.
• I2. Keep the lexicon with a reachability count: the paper's top-three rule (ranked by p(s|t) times p(t|s)) leaves some sentences unreachable. For each merge in the translator's lexicon, report how many verses lose their reading when it is dropped, instead of only the merge count.
• I3. Configurations as separate features: count the configuration class of each meta-token pair and give each class its own weight in the encoder score, not a single penalty. This is a count version of the paper's tree-to-tree features and fits open problem 1.
• I4. Pseudolikelihood as a check, not a method: the paper trains with pseudolikelihood on a dev set. A count-based check of the same kind is to compare the encoder's score of the gold verse against the score of the verse with one token changed, over the Bible pairs, and to report the share where the gold wins.

For Mova

The code is tools/translate/src/profile.rs (the encoder score) and tools/translate/src/meta.rs (the reader). The decoder itself, with lattice parsing and cube summing, is not for Mova, since the translator has no search at test time that needs it. The feature grouping and the coverage check carry over as counts over tables that are ordered by key.

Topics: Machine translation

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.3115/1699510.1699539

@inproceedings{gimpel2009feature,
  title = {{Feature-Rich Translation by Quasi-Synchronous Lattice Parsing}},
  author = {Gimpel, Kevin and Smith, Noah A.},
  booktitle = {Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing},
  year = {2009},
  doi = {10.3115/1699510.1699539},
  url = {https://doi.org/10.3115/1699510.1699539}
}

References

44 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. H. Alshawi, S. Bangalore, and S. Douglas. 2000. Learning dependency translation modles as colec-tions of finite-state head transducers. Computational Linguistics, 26(1):45–60. find
2. S. Banerjee and A. Lavie. 2005. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In Proc. of ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for MT and/or Summarization. find our card
3. J. E. Besag. 1975. Statistical analysis of non-lattice data. The Statistician, 24:179–195. find
4. P. Blunsom and M. Osborne. 2008. Probabilistic inference for machine translation. In Proc. of EMNLP. find
5. P. Blunsom, T. Cohn, and M. Osborne. 2008. A discriminative latent variable model for statistical machine translation. In Proc. of ACL. find
6. P. F. Brown, S. A. Della Pietra, V. J. Della Pietra, and R. L. Mercer. 1993. The mathematics of statistical machine translation: Parameter estimation. Computational Linguistics, 19(2):263–311. find
7. S. Chen and J. Goodman. 1998. An empirical study of smoothing techniques for language modeling. Technical report 10-98, Harvard University. find
8. D. Chiang, Y. Marton, and P. Resnik. 2008. Online large-margin training of syntactic and structural translation features. In Proc. of EMNLP. find
9. D. Chiang. 2005. A hierarchical phrase-based model for statistical machine translation. In Proc. of ACL. find our card
10. D. Chiang. 2007. Hierarchical phrase-based translation. Computational Linguistics, 33(2):201–228. find our card
11. M. Collins. 1999. Head-Driven Statistical Models for Natural Language Parsing. Ph.D. thesis, U. Penn. find our card
12. D. Das and N. A. Smith. 2009. Paraphrase identification as probabilistic quasi-synchronous recognition. In Proc. of ACL-IJCNLP. find
13. Y. Ding and M. Palmer. 2005. Machine translation using probabilistic synchronous dependency insertion grammar. In Proc. of ACL. find
14. J. Eisner, E. Goldlust, and N. A. Smith. 2005. Compiling Comp Ling: Practical weighted dynamic programming and the Dyna language. In Proc. of HLT-EMNLP. find
15. J. Eisner. 1997. Bilexical grammars and a cubic-time probabilistic parser. In Proc. of IWPT. find
16. M. Galley, J. Graehl, K. Knight, D. Marcu, S. DeNeefe, W. Wang, and I. Thayer. 2006. Scalable inference and training of context-rich syntactic translation models. In Proc. of COLING-ACL. find our card
17. K. Gimpel and N. A. Smith. 2008. Rich source-side context for statistical machine translation. In Proc. of ACL-2008 Workshop on Statistical Machine Translation. find
18. K. Gimpel and N. A. Smith. 2009. Cube summing, approximate inference with non-local features, and dynamic programming without semirings. In Proc. of EACL. find
19. R. Haque, S. K. Naskar, Y. Ma, and A. Way. 2009. Using supertags as source language context in SMT. In Proc. of EAMT. find
20. L. Huang and D. Chiang. 2007. Forest rescoring: Faster decoding with integrated language models. In Proc. of ACL. find
21. A. Ittycheriah and S. Roukos. 2007. Direct translation model 2. In Proc. of HLT-NAACL. find
22. D. Klein and C. D. Manning. 2003. Fast exact inference with a factored model for natural language parsing. In Advances in NIPS 15. find
23. D. Klein and C. D. Manning. 2004. Corpus-based induction of syntactic structure: Models of dependency and constituency. In Proc. of ACL. find
24. P. Koehn, F. J. Och, and D. Marcu. 2003. Statistical phrase-based translation. In Proc. of HLT-NAACL. find
25. P. Koehn, H. Hoang, A. Birch, C. Callison-Burch, M. Federico, N. Bertoldi, B. Cowan, W. Shen, C. Moran, R. Zens, C. Dyer, O. Bojar, A. Constantin, and E. Herbst. 2007. Moses: Open source toolkit for statistical machine translation. In Proc. of ACL (demo session). find
26. T. Koo and M. Collins. 2005. Hidden-variable models for discriminative reranking. In Proc. of EMNLP. find
27. P. Liang, A. Bouchard-Côté, D. Klein, and B. Taskar. 2006. An end-to-end discriminative approach to machine translation. In Proc. of COLING-ACL. find
28. A. Lopez. 2009. Translation as weighted deduction. In Proc. of EACL. find
29. D. Marcu, W. Wang, A. Echihabi, and K. Knight. 2006. Statistical machine translation with syntactified target language phrases. In Proc. of EMNLP. find
30. H. Mi, L. Huang, and Q. Liu. 2008. Forest-based translation. In Proc. of ACL. find our card
31. F. J. Och and H. Ney. 2002. Discriminative training and maximum entropy models for statistical machine translation. In Proc. of ACL. find
32. F. J. Och and H. Ney. 2003. A systematic comparison of various statistical alignment models. Computational Linguistics, 29(1). find
33. F. J. Och. 2003. Minimum error rate training for statistical machine translation. In Proc. of ACL. find
34. K. Papineni, S. Roukos, and T. Ward. 1997. Feature-based language understanding. In EUROSPEECH. find
35. K. Papineni, S. Roukos, T. Ward, and W.J. Zhu. 2001. BLEU: a method for automatic evaluation of machine translation. In Proc. of ACL. find our card
36. F. C. N. Pereira and Y. Schabes. 1992. Inside-outside reestimation from partially bracketed corpora. In Proc. of ACL. find
37. C. Quirk, A. Menezes, and C. Cherry. 2005. Dependency treelet translation: Syntactically informed phrasal SMT. In Proc. of ACL. find our card
38. L. Shen, J. Xu, and R. Weischedel. 2008. A new string-to-dependency machine translation algorithm with a target dependency language model. In Proc. of ACL. find our card
39. D. A. Smith and J. Eisner. 2006. Quasi-synchronous grammars: Alignment by soft projection of syntactic dependencies. In Proc. of HLT-NAACL Workshop on Statistical Machine Translation. find our card
40. D. A. Smith and J. Eisner. 2009. Parser adaptation and projection with quasi-synchronous features. In Proc. of EMNLP. find
41. A. Stolcke. 2002. SRILM—an extensible language modeling toolkit. In Proc. of ICSLP. find
42. X. Sun and J. Tsujii. 2009. Sequential labeling with latent variables: An exact inference algorithm and its efficient approximation. In Proc. of EACL. find
43. M. Wang, N. A. Smith, and T. Mitamura. 2007. What is the Jeopardy model? a quasi-synchronous grammar for QA. In Proc. of EMNLP-CoNLL. find
44. K. Yamada and K. Knight. 2001. A syntax-based statistical translation model. In Proc. of ACL. find our card

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/gimpel-2009-feature-rich-translation-quasi-synchronous/