A Syntax-based Statistical Translation Model

What it is about

The paper builds a translation channel that takes an English parse tree and produces a Japanese string. At each node the channel makes three stochastic choices: it reorders the children, it inserts words left, right or nowhere, and it translates each leaf word. Insertion positions are indexed by the node label and its parent's label, and the inserted words themselves are chosen without context. The parameters are estimated with EM by summing over all operation sequences that give the same output, and a dynamic program keeps this polynomial. The training data is about 2,000 English-Japanese pairs from a dictionary, with English trees from a parser and node labels collapsed to head-word tags (30 labels, 474 child sequences). On the first 50 pairs, the Viterbi alignments score 0.582 against 0.431 for IBM Model 5 on a human scale, with 10 perfectly aligned sentence pairs against none for Model 5. The training perplexity is higher than Model 5's, which the authors attribute to Model 5 over-fitting on its small corpus.

Ideas

• I1. Reordering table keyed by the sequence of child labels (the r-table). For Hebrew-to-meta order, count the child-label sequences of the MACULA trees and the English order they take in the aligned verse, per parent label. This gives a table form of the Hebrew order normaliser (open problem 7).
• I2. Insertion table keyed by (node label, parent label), used for function words: definiteness, that, quotes and commas (open problem 4). Insert a function word at a meta token only when its count in that context passes a threshold.
• I3. Per-author delta as EM counts. Train the reorder, insert and translate tables on each translator's verse pairs and keep the author's table only where it differs from the standard by at least three tokens, as the delta rule already says.
• I4. Their perplexity check is a warning for Mova: on small corpora a model can get lower perplexity by over-fitting. Report the held-out residual, not the training perplexity, when comparing a per-author table with the standard.

For Mova

The place is tools/translate (profile.rs for the encoder tables, the delta rule for author overrides). The paper's model is a probabilistic rule system with EM, which Mova can use only in its count-based form: the tables are counts, the EM step is replaced by one exact count pass, and the output is checked for identical runs. Measure first the held-out residual of the order table on the Hebrew reader's output.

Topics: Machine translation

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.3115/1073012.1073079

@inproceedings{yamada2001syntax,
  title = {{A Syntax-based Statistical Translation Model}},
  author = {Yamada, Kenji and Knight, Kevin},
  booktitle = {Proceedings of the 39th Annual Meeting of the Association for Computational Linguistics},
  year = {2001},
  doi = {10.3115/1073012.1073079},
  url = {https://doi.org/10.3115/1073012.1073079}
}

References

7 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. H. Alshawi, S. Bangalore, and S. Douglas. 2000. Learning dependency translation models as collections of finite state head transducers. Computational Linguistics, 26(1). find
2. J. Baker. 1979. Trainable grammars for speech recognition. In Speech Communication Papers for the 97th Meeting of the Acoustical Sciety of America. find
3. A. Berger, P. Brown, S. Della Pietra, V. Della Pietra, J. Gillett, J. Lafferty, R. Mercer, H. Printz, and L. Ures. 1996. Language Translation Apparatus and Method Using Context-Based Translation Models. U.S. Patent 5,510,981. find
4. E. Brill. 1995. Transformation-based error-driven learning and natural language processing: A case study in part of specch tagging. Computational Linguistics, 21(4). find
5. P. Brown, J. Cocke, S. Della Pietra, F. Jelinek, R. Mercer, and P. Roossin. 1988. A statistical approach to language translation. In COLING-88. find
6. P. Brown, J. Cocke, S. Della Pietra, F. Jelinek, R. Mercer, and P. Roossin. 1991. Word-sense disambiguation using statistical methods. In ACL-91. find
7. P. Brown, S. Della Pietra, V. Della Pietra, and R. Mercer. 1993. The mathematics of statistical machine translation: Parameter estimation. Computational Linguistics, 19(2). YZ[]\$^ _`Z`\ _acbd`\$de e fOgih bafcZ[kj$al\_`ZSa6b8b$fnmoqp r8stcunvsw mxe y+z{ |}~bab$f6Z[Ta€(\$Obaƒ‚ „b†x[nynm find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/yamada-2001-syntax-based-statistical-translation-model/