Effective Use of Function Words for Rule Generalization in Forest-Based Translation

What it is about

English-to-Japanese forest-to-string translation on about one million abstract pairs. In a hand check of 100 GIZA++ alignments, 309 of 1,324 links were wrong, and 237 of those were target function words (particles and auxiliaries), which tend to align to a few English words such as "of" and "the". The fix: drop the alignments of target function words, then attach each one, as an unaligned item, to the frontier nodes of the source HPSG forest whose spans overlap the Japanese chunk (from a dependency chunker) that holds it. Rules are extracted with EM, and fractional counts are discounted by the parse probabilities. The composed rules generalise better (more small rules, as the node-count histogram shows), and BLEU rose from 27.07 to 28.89 over a minimal-rule forest baseline (significant at p < 0.01), with slower decoding.

Ideas

• I1. Attach, do not align. In the standard encoder (profile.rs), write a target function word (case marker, article, auxiliary) with the chunk of the content word it binds to, not through the word alignment. Measure the function-word class of the residual in exact mode on dev, run twice.
• I2. Remove function-word links before the IBM Model 1 step of hebrew.rs (ibm.rs): drop the links from Hebrew particles to meta tokens, then bind them back by chunk. Compare the reader's bits with and without the removal.
• I3. Keep several attachments, not one. The reader reads one parse per translation. Keep the k best attachments a parser gives and let the encoder choose the one that yields the right function words. Measure first how often the top attachment is wrong on function words; this answers open problem 1 directly.
• I4. Use the paper's diagnostic table: count the rules and their sizes per context level of the encoder chain (concept, tag, neighbours). A chain that produces many large entries generalises less.

For Mova

The paper is about Japanese particles, but the problem is open problem 4 in Mova's translator. The forest idea is a parse-level analogue: several readings of one verse, one choice per function word. Its rule-generalisation counts are a measure Mova can use on the encoder tables without any neural model.

Topics: Machine translation

Cite

The paper itself, not this card. BibTeX from the library's record of the work.

@inproceedings{wu2011effective,
  title = {{Effective Use of Function Words for Rule Generalization in Forest-Based Translation}},
  author = {Wu, Xianchao and Matsuzaki, Takuya and Tsujii, Jun'ichi},
  booktitle = {Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies},
  year = {2011},
  url = {https://aclanthology.org/P11-1003/}
}

References

28 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. David Chiang. 2005. A hierarchical phrase-based model for statistical machine translation. In Proceedings of ACL, pages 263–270, Ann Arbor, MI. find our card
2. David Chiang. 2010. Learning to translate with source and target syntax. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 1443–1452, Uppsala, Sweden, July. Association for Computational Linguistics. find our card
3. A. P. Dempster, N. M. Laird, and D. B. Rubin. 1977. Maximum likelihood from incomplete data via the em algorithm. Journal of the Royal Statistical Society, 39:1–38. find
4. Michel Galley, Mark Hopkins, Kevin Knight, and Daniel Marcu. 2004. What’s in a translation rule? In Proceedings of HLT-NAACL. find our card
5. Michel Galley, Jonathan Graehl, Kevin Knight, Daniel Marcu, Steve DeNeefe, Wei Wang, and Ignacio Thayer. 2006. Scalable inference and training of context-rich syntactic translation models. In Proceedings of COLING-ACL, pages 961–968, Sydney. find our card
6. Liang Huang and David Chiang. 2005. Better k-best parsing. In Proceedings of IWPT. find
7. Philipp Koehn, Franz Josef Och, and Daniel Marcu. 2003. Statistical phrase-based translation. In Proceedings of the Human Language Technology and North American Association for Computational Linguistics Conference (HLT/NAACL), Edomonton, Canada, May 27-June 1. find
8. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondřej Bojar, Alexandra Constantin, and Evan Herbst. 2007. Moses: Open source toolkit for statistical machine translation. In Proceedings of the ACL 2007 Demo and Poster Sessions, pages 177–180. find
9. Taku Kudo and Yuji Matsumoto. 2002. Japanese dependency analysis using cascaded chunking. In Proceedings of CoNLL-2002, pages 63–69. Taipei, Taiwan. find
10. Zhifei Li, Chris Callison-Burch, Chris Dyery, Juri Ganitkevitch, Sanjeev Khudanpur, Lane Schwartz, Wren N. G. Thornton, Jonathan Weese, and Omar F. Zaidan. 2009. Demonstration of joshua: An open source toolkit for parsing-based machine translation. In Proceedings of the ACL-IJCNLP 2009 Software Demonstrations, pages 25–28, August. find
11. Yang Liu, Qun Liu, and Shouxun Lin. 2006. Tree-to-string alignment templates for statistical machine transaltion. In Proceedings of COLING-ACL, pages 609–616, Sydney, Australia. find
12. Yang Liu, Yajuan Lü, and Qun Liu. 2009. Improving tree-to-tree translation with packed forests. In Proceedings of ACL-IJCNLP, pages 558–566, August. find
13. Samuel E. Martin. 1975. A Reference Grammar of Japanese. New Haven, Conn.: Yale University Press. find
14. Jonathan May and Kevin Knight. 2007. Syntactic realignment models for machine translation. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), pages 360–368, Prague, Czech Republic, June. Association for Computational Linguistics. find
15. Haitao Mi and Liang Huang. 2008. Forest-based translation rule extraction. In Proceedings of EMNLP, pages 206–214, October. find
16. Haitao Mi and Qun Liu. 2010. Constituency to dependency translation with forests. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 1433–1442, Uppsala, Sweden, July. Association for Computational Linguistics. find
17. Haitao Mi, Liang Huang, and Qun Liu. 2008. Forest-based translation. In Proceedings of ACL-08:HLT, pages 192–199, Columbus, Ohio. find our card
18. Yusuke Miyao and Jun’ichi Tsujii. 2008. Feature forest models for probabilistic hpsg parsing. Computational Lingustics, 34(1):35–80. find our card
19. Franz Josef Och and Hermann Ney. 2003. A systematic comparison of various statistical alignment models. Computational Linguistics, 29(1):19–51. find
20. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of ACL, pages 311–318. find our card
21. Carl Pollard and Ivan A. Sag. 1994. Head-Driven Phrase Structure Grammar. University of Chicago Press. find
22. Ivan A. Sag, Thomas Wasow, and Emily M. Bender. 2003. Syntactic Theory: A Formal Introduction. Number 152 in CSLI Lecture Notes. CSLI Publications. find
23. Andreas Stolcke. 2002. Srilm-an extensible language modeling toolkit. In Proceedings of International Conference on Spoken Language Processing, pages 901–904. find
24. Masao Utiyama and Hitoshi Isahara. 2007. A japanese-english patent parallel corpus. In Proceedings of MT Summit XI, pages 475–482, Copenhagen. find
25. Wei Wang, Jonathan May, Kevin Knight, and Daniel Marcu. 2010. Re-structuring, re-labeling, and realigning for syntax-based machine translation. Computational Linguistics, 36(2):247–277. find
26. Xianchao Wu, Takuya Matsuzaki, and Jun’ichi Tsujii. 2010. Fine-grained tree-to-string translation rule extraction. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 325–334, Uppsala, Sweden, July. Association for Computational Linguistics. find
27. Peng Xu, Jaeho Kang, Michael Ringgaard, and Franz Och. 2009. Using a dependency parser to improve smt for subject-object-verb languages. In Proceedings of HLT-NAACL, pages 245–253. find
28. Min Zhang, Hongfei Jiang, Ai Ti Aw, Jun Sun, Sheng Li, and Chew Lim Tan. 2007. A tree-to-tree alignment-based model for statistical machine translation. In Proceedings of MT Summit XI, pages 535–542, Copenhagen, Denmark, September. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/wu-2011-effective-use-function-words-rule/