Hidden Markov Tree Model in Dependency-based Machine Translation

What it is about

The paper applies Hidden Markov Tree Models (HMTM), a tree version of the hidden Markov chain, to the transfer step of a tree-based MT system. The source tree is the observed labelling, and the target tree is the hidden labelling. Each node carries a pair (lemma, formeme) on both sides. The emission probability is a backward translation model, and the transition probability is a model of the target tree given the parent, so the two parts can be estimated separately from a parallel and a monolingual corpus. A modified Viterbi algorithm finds the best target labelling over the whole tree in one pass. The model needs the source and target trees to have the same shape, which the paper argues is roughly true for deep-syntactic trees. Trained on 800 million words of Czech and a 60 million word parallel corpus, the model is tested on 2,700 WMT 2009 English-Czech sentences. It raises BLEU from 0.0898 to 0.1043 and NIST from 4.567 to 4.845 over the transfer step it replaces.

Ideas

• I1. A tree Viterbi over meta tokens: give the reader's meta-world tree a count-based emission (source concept and tag given target concept and tag) and a transition (child target given parent target, by relation). Find the best target labelling in one pass and compare with the greedy per-token choice in tools/translate/src/profile.rs, run twice for identical output.
• I2. Factor the encoder into channels, as the paper does: the lexical choice (concept), the surface form (function words and case, the formeme idea) and the grammatical categories (number, person from UD FEATS, open problems 4 and 5). Measure the residual bits per channel, so each factor is judged on its own.
• I3. Check the isomorphism assumption before using it: count the share of verse pairs (KJV against WEB, say) whose meta trees have the same shape after the reader. If the share is low, the transition model is not usable on those pairs.
• I4. Controls: run emission-only against emission plus transition on the same dev split. The paper's gain is small, so the transition term must show a gain on dev, not only in the sum of scores.

For Mova

The code is tools/translate/src/profile.rs (the encoder) and tools/translate/src/meta.rs (the meta tokens). The paper is statistical with simple count tables, which is the form Mova needs. Its HMTM is not a neural model, so the whole idea carries over. The main risk is the same-shape assumption, which is open problem 1 in another form.

Topics: Machine translation

Cite

The paper itself, not this card. BibTeX from the library's record of the work.

@inproceedings{zabokrtsku2009hidden,
  title = {{Hidden Markov Tree Model in Dependency-based Machine Translation}},
  author = {Žabokrtský, Zdeněk and Popel, Martin},
  booktitle = {Proceedings of the ACL-IJCNLP 2009 Conference Short Papers},
  year = {2009},
  url = {https://aclanthology.org/P09-2037/}
}

References

11 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Igor Boguslavsky, Leonid Iomdin, and Victor Sizov. 2004. Multilinguality in ETAP-3: Reuse of Lexical Resources. In Proceedings of Workshop Multilingual Linguistic Resources, COLING, pages 7–14. find
2. Ondřej Bojar. 2008. Exploiting Linguistic Data in Machine Translation. Ph.D. thesis, ÚFAL, MFF UK, Prague, Czech Republic. find
3. Peter E. Brown, Vincent J. Della Pietra, Stephen A. Della Pietra, and Robert L. Mercer. 1993. The mathematics of statistical machine translation: Parameter estimation. Computational Linguistics. find
4. Matthew Crouse, Robert Nowak, and Richard Baraniuk. 1998. Wavelet-based statistical signal processing using hidden markov models. IEEE Transactions on Signal Processing, 46(4):886–902. find
5. Michelangelo Diligenti, Paolo Frasconi, and Marco Gori. 2003. Hidden tree Markov models for document image classification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25:2003. find
6. Jean-Baptiste Durand, Paulo Goncalvès, and Yann Guédon. 2004. Computational methods for hidden Markov tree models - An application to wavelet trees. IEEE Transactions on Signal Processing, 52(9):2551–2560. find
7. Jan Hajič et al. 2006. Prague Dependency Treebank 2.0. Linguistic Data Consortium, LDC Catalog No.: LDC2006T01, Philadelphia. find
8. Philipp Koehn, Franz Josef Och, and Daniel Marcu. 2003. Statistical phrase based translation. In Proceedings of the HLT/NAACL. find
9. Arul Menezes and Stephen D. Richardson. 2001. A best-first alignment algorithm for automatic extraction of transfer mappings from bilingual corpora. In Proceedings of the workshop on Data-driven methods in machine translation, volume 14, pages 1–8. find
10. Petr Sgall. 1967. Generativnı́ popis jazyka a česká deklinace. Academia, Prague, Czech Republic. find
11. Zdeněk Žabokrtský, Jan Ptáček, and Petr Pajas. 2008. TectoMT: Highly Modular MT System with Tectogrammatics Used as Transfer Layer. In Proceedings of the 3rd Workshop on SMT, ACL. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/zabokrtsky-2009-hidden-markov-tree-model-dependency/