Scalable Inference and Training of Context-Rich Syntactic Translation Models

What it is about

The paper extends the rule extraction of Galley et al. (2004). Instead of keeping one minimal derivation per sentence pair, it builds a forest of many derivations, so that larger rules with more syntactic context and alternative readings of unaligned words are all available. It puts probabilities on these multi-level transfer rules, estimates them over very large rule sets from the forest, and shows how the forest is packed so that counts stay tractable. The English-Chinese corpus has 54 million words and 1.95 million sentence pairs, and the rule and derivation counts are reported with the running time for 50 EM iterations on 10 cluster nodes. In translation, a CKY decoder with beam search gives BLEU-4 results on the NIST sets. On NIST 2002, the rules with the largest context reach 28.1 BLEU, against 24.47 with minimal rules, which the abstract summarises as a 3.63 point gain. The decoder has no syntax-based language model, which the authors name as a limit.

Ideas

• I1. Fractional counts over alternative derivations. For the encoder tables, count each meta token's chunk under every chunking the reader allows, weighted by the current counts, instead of one best chunking. This is the EM idea in count form and bears on open problems 2 and 3.
• I2. Larger context in the encoder. The paper's gain comes from conditioning on more of the tree. The encoder already conditions on concept, tag and neighbours; test one extra level (the head concept) on dev verses, and keep it only if the exact-match residual falls on held-out verses.
• I3. A size sweep as in the paper's Table 6: minimal context against larger context, on the same verses, reported with the same split. Mova's gain must be shown the way the paper shows it, with the number and the split printed.
• I4. Probability of the target side given the source side, p(rhs given lhs), kept as a counted table per context. The paper's example of SVO to Arabic VSO is a test case for the Hebrew order normaliser (open problem 7).

For Mova

The work sits in tools/translate, in the encoder (profile.rs) and the Hebrew reader (hebrew.rs). The paper's probability model is EM over forests; Mova can take the counted-forest part, and it does not need a neural decoder. Measure first the residual of the encoder with minimal and with one-level-larger contexts on the same dev verses, and run it twice to check the output is identical.

Topics: Machine translation

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.3115/1220175.1220296

@inproceedings{galley2006scalable,
  title = {{Scalable Inference and Training of Context-Rich Syntactic Translation Models}},
  author = {Galley, Michel and Graehl, Jonathan and Knight, Kevin and Marcu, Daniel and DeNeefe, Steve and Wang, Wei and Thayer, Ignacio},
  booktitle = {Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics},
  year = {2006},
  doi = {10.3115/1220175.1220296},
  url = {https://doi.org/10.3115/1220175.1220296}
}

References

9 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. D. Chiang. 2005. A hierarchical phrase-based model for statistical machine translation. In Proc. of ACL. find our card
2. H. Fox. 2002. Phrasal cohesion and statistical machine translation. In Proc. of EMNLP, pages 304–311. find
3. M. Galley, M. Hopkins, K. Knight, and D. Marcu. 2004. What’s in a translation rule? In Proc. of HLT/NAACL-04. find our card
4. J. Graehl and K. Knight. 2004. Training tree transducers. In Proc. of HLT/NAACL-04, pages 105–112. find our card
5. F. Och and H. Ney. 2004. The alignment template approach to statistical machine translation. Computational Linguistics, 30(4):417–449. find
6. A. Poutsma. 2000. Data-oriented translation. In Proc. of COLING, pages 635–641. find
7. D. Wu. 1997. Stochastic inversion transduction grammars and bilingual parsing of parallel corpora. Computational Linguistics, 23(3):377–404. find our card
8. K. Yamada and K. Knight. 2001. A syntax-based statistical translation model. In Proc. of ACL, pages 523–530. find our card
9. H. Zhang, L. Huang, D. Gildea, and K. Knight. 2006. Synchronous binarization for machine translation. In Proc. of HLT/NAACL. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/galley-2006-scalable-inference-training-context-rich/