Formemes in English-Czech Deep Syntactic MT

What it is about

A formeme is a short label for the surface morpho-syntax of a node in a deep dependency tree. It has three parts: a syntactic part of speech, an optional preposition or conjunction, and a form such as a case number or a finite-clause marker. The paper revises the formemes used in the TectoMT English-Czech system to reduce data sparsity and to make the two languages more consistent. The changes are: the syntactic part of speech replaces the semantic one; numerals and adjectives used as nouns get the value a noun would get in the same position; Czech nominal attributes are marked only when case-congruent; and prepositions are chosen from the part-of-speech filter, not from dependency labels. A candidate change is kept only if it increases the mutual information of formemes between Czech and English trees, which rose from 1.598 to 1.687. Several changes were dropped because they lowered it. In the downstream tests, Czech-to-Czech synthesis BLEU rose from 0.6818 to 0.7092, and English-Czech BLEU on the WMT12 test rose from 0.1190 to 0.1199, a small gain trained on half the data.

Ideas

• I1. Value per meta token: give each meta token a short, readable value made of its coarse part of speech, its surface function word and its case or position (for example a subject noun, or a noun after "of"). This is the formeme idea for open problem 4, and it lets the encoder choose the surface form from the value alone.
• I2. A cheap gate before retraining: the paper scores a markup change by the mutual information between source and target values on aligned tokens, not by retraining the translation model. Use the same test on the Bible pairs in tools/translate to choose which function-word features to add, with counts kept in BTreeMap.
• I3. Keep a feature only when it raises the gate, as the paper drops changes that lower it. This mirrors the lexicon rule in open problem 2: a merge or a feature stays only if it lowers the residual.
• I4. Number words: the paper separates adjectival and nominal use of numerals (5 potatoes against Apollo 11). Count the same split for English and Hebrew number words in the Hebrew reader, which is the weakest part of the translator.

For Mova

The code is tools/translate/src/meta.rs (the features) and tools/translate/src/profile.rs (the encoder that chooses surface forms). The paper is count-based and rule-based, so it fits Mova without change. Its gains are small, so each feature should be judged by the residual bits in exact mode, not by a single BLEU number.

Topics: Part-of-speech tagging · Machine translation

Cite

The paper itself, not this card. BibTeX from the library's record of the work.

@inproceedings{dusek2012formemes,
  title = {{Formemes in English-Czech Deep Syntactic MT}},
  author = {Dušek, Ondřej and Žabokrtský, Zdeněk and Popel, Martin and Majliš, Martin and Novák, Michal and Mareček, David},
  booktitle = {Proceedings of the Seventh Workshop on Statistical Machine Translation},
  year = {2012},
  url = {https://aclanthology.org/W12-3132/}
}

References

27 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. O. Bojar and Z. Žabokrtský. 2009. CzEng 0.9: Large Parallel Treebank with Rich Annotation. Prague Bulletin of Mathematical Linguistics, 92. find
2. O. Bojar, D. Mareček, V. Novák, M. Popel, J. Ptáček, J. Rouš, and Z. Žabokrtský. 2009. English-Czech MT in 2008. In Proceedings of the Fourth Workshop on Statistical Machine Translation, pages 125–129. Association for Computational Linguistics. find
3. O. Bojar, J. Hajič, E. Hajičová, J. Panevová, P. Sgall, S. Cinková, E. Fučíková, M. Mikulová, P. Pajas, J. Popelka, J. Semecký, J. Šindlerová, J. Štěpánek, J. Toman, Z. Urešová, and Z. Žabokrtský. 2012a. Announcing Prague Czech-English Dependency Treebank 2.0. In Proceedings of LREC 2012, Istanbul, Turkey, May. ELRA, European Language Resources Association. In print. find
4. O. Bojar, Z. Žabokrtský, O. Dušek, P. Galuščáková, M. Majliš, D. Mareček, J. Maršík, M. Novák, M. Popel, and A. Tamchyna. 2012b. The Joy of Parallelism with CzEng 1.0. In Proceedings of LREC 2012, Istanbul, Turkey, May. ELRA, European Language Resources Association. In print. find
5. J. Hajič, J. Panevová, Z. Urešová, A. Bémová, V. Kolárová, and P. Pajas. 2003. PDT-VALLEX: Creating a large-coverage valency lexicon for treebank annotation. In Proceedings of The Second Workshop on Treebanks and Linguistic Theories, volume 9, pages 57–68. find
6. J. Hajič, J. Panevová, E. Hajičová, P. Sgall, P. Pajas, J. Štěpánek, J. Havelka, M. Mikulová, Z. Žabokrtský, and M. Ševčíková-Razímová. 2006. Prague Dependency Treebank 2.0. CD-ROM LDC2006T01, LDC, Philadelphia. find
7. G. Helbig and W. Schenkel. 1969. Wörterbuch zur Valenz und Distribution deutscher Verben. VEB Bibliographisches Institut, Leipzig. find
8. P. Koehn. 2004. Statistical Significance Tests for Machine Translation Evaluation. In Proceedings of EMNLP 2004, Barcelona, Spain. find
9. M. Lopatková, Z. Žabokrtský, V. Kettnerová, and K. Skwarska. 2008. Valenční slovník českých sloves. Karolinum, Prague. find
10. C.D. Manning and H. Schütze. 1999. Foundations of statistical natural language processing. MIT Press. find
11. D. Marcu, W. Wang, A. Echihabi, and K. Knight. 2006. SPMT: Statistical machine translation with syntactified target language phrases. In Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, pages 44–52. Association for Computational Linguistics. find
12. D. Mareček, M. Popel, and Z. Žabokrtský. 2010. Maximum entropy translation model in dependency-based MT framework. In Proceedings of the Joint Fifth Workshop on Statistical Machine Translation and Metrics (MATR), pages 201–206. Association for Computational Linguistics. find
13. A. Menezes and S. D. Richardson. 2001. A best-first alignment algorithm for automatic extraction of transfer mappings from bilingual corpora. In Proceedings of the workshop on Data-driven methods in machine translation - Volume 14, DMMT ’01, pages 1–8, Stroudsburg, PA. Association for Computational Linguistics. find
14. K. Papineni, S. Roukos, T. Ward, and W.J. Zhu. 2002. BLEU: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting on association for computational linguistics, pages 311–318. Association for Computational Linguistics. find our card
15. M. Popel and Z. Žabokrtský. 2010. TectoMT: modular NLP framework. Advances in Natural Language Processing, pages 293–304. find
16. M. Popel, D. Mareček, N. Green, and Z. Žabokrtský. 2011. Influence of parser choice on dependency-based MT. In Chris Callison-Burch, Philipp Koehn, Christof Monz, and Omar Zaidan, editors, Proceedings of the Sixth Workshop on Statistical Machine Translation, pages 433–439, Edinburgh, UK. Association for Computational Linguistics. find
17. J. Ptáček and Z. Žabokrtský. 2006. Synthesis of Czech sentences from tectogrammatical trees. In Text, Speech and Dialogue, pages 221–228. Springer. find
18. C. Quirk, A. Menezes, and C. Cherry. 2005. Dependency treelet translation: Syntactically informed phrasal SMT. In Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics, pages 271–279. Association for Computational Linguistics. find our card
19. M. Razímová and Z. Žabokrtský. 2006. Annotation of grammatemes in the Prague Dependency Treebank 2.0. In Proceedings of the LREC 2006 Workshop on Annotation Science, pages 12–19. find
20. M. Ševčíková-Razímová and Z. Žabokrtský. 2006. Systematic parameterized description of pro-forms in the Prague Dependency Treebank 2.0. In J. Hajič and J. Nivre, editors, Proceedings of the Fifth Workshop on Treebanks and Linguistic Theories (TLT), pages 175– 186, Prague. find
21. P. Sgall, E. Hajičová, J. Panevová, and J. Mey. 1986. The meaning of the sentence in its semantic and pragmatic aspects. Springer. find
22. P. Sgall. 1967. Generativní popis jazyka a česká dekli-nace. Academia, Prague. find
23. Z. Urešová. 2009. Building the PDT-VALLEX valency lexicon. In On-line proceedings of the fifth Corpus Linguistics Conference. University of Liverpool. find
24. Z. Žabokrtský, J. Ptáček, and P. Pajas. 2008. TectoMT: highly modular MT system with tectogrammatics used as transfer layer. In Proceedings of the Third Workshop on Statistical Machine Translation, StatMT ’08, pages 167–170, Stroudsburg, PA. Association for Computational Linguistics. find
25. Z. Žabokrtský. 2005. Valency Lexicon of Czech Verbs. Ph.D. thesis, Charles University in Prague. find
26. Z. Žabokrtský. 2010. From Treebanking to Machine Translation. Habilitation thesis, Charles University in Prague. find
27. Z. Žabokrtský and M. Popel. 2009. Hidden Markov Tree Model in Dependency-based Machine Translation. In Proceedings of the ACL-IJCNLP 2009 Conference Short Papers, pages 145–148, Suntec, Singapore. find our card

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/dusek-2012-formemes-english-czech-deep-syntactic/