Generation by Inverting a Semantic Parser that Uses Statistical Machine Translation

What it is about

Generation from meaning representations (RoboCup coach advice and GeoQuery database queries) is treated as the inverse of semantic parsing. The paper first applies a phrase-based SMT system (Pharaoh) to meaning-to-text, with the meaning tokens as the source language. It then inverts WASP, an SMT-based parser that learns synchronous context-free rules by word alignment from linearised production lists of the meaning language, not from flat tokens. The same rules serve both parsing and generation, and generation uses a noisy-channel model with an n-gram language model. A hybrid of the two systems scores best. On RoboCup, BLEU is 0.32 for Pharaoh and 0.60 for the hybrid WASP inversion, against 0.44 for WASP alone; on GeoQuery, 0.21 for Pharaoh and 0.54 for the hybrid. Human judges rate the hybrid higher on fluency and adequacy in RoboCup. The main lesson is that linearised tree productions, rather than flat tokens, give the alignment a usable structure, and that a meaning token with no meaning of its own (a bracket) confuses alignment.

Ideas

• I1. Read the meta-world as a list of productions, not as flat tokens. Each meta token contributes its concept, tag and relation as one production, and the encoder aligns productions to chunks. Measure the exact match per verse with the flat list and with the production list, run twice.
• I2. Use one rule table in both directions. WASP extracts rule pairs (source phrase and meta production) from alignments and inverts them for generation. Mova's mutual-partner merges in tools/translate/src/meta.rs are the same pairs; test using them, inverted, as the encoder's chunk rules on dev.
• I3. Drop meaningless tokens before alignment. The paper removes brackets and similar delimiters that disturb IBM alignment. The Hebrew reader in tools/translate/src/hebrew.rs aligns with IBM Model 1; try removing punctuation and empty meta tokens first, and measure alignment accuracy on a held-out verse set.
• I4. Keep the hybrid as a control. The gain of the hybrid over either part is a useful check that two count-based encoders combined beat each. Report the standard encoder, a second count-based one, and their per-context choice side by side.

For Mova

The code is tools/translate/src/hebrew.rs (alignment), tools/translate/src/meta.rs (lexicon pairs) and tools/translate/src/profile.rs (encoder). The ideas touch open problems 2 (merges), 3 (trees) and 7 (Hebrew alignment). The SMT machinery itself (phrase tables, n-gram scoring with EM) is not part of Mova's deterministic design; only the production view and the alignment fix are.

Topics: Machine translation

Cite

The paper itself, not this card. BibTeX from the library's record of the work.

@inproceedings{wong2007generation,
  title = {{Generation by Inverting a Semantic Parser that Uses Statistical Machine Translation}},
  author = {Wong, Yuk Wah and Mooney, Raymond},
  booktitle = {Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Conference},
  year = {2007},
  url = {https://aclanthology.org/N07-1022/}
}

References

32 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. A. V. Aho and J. D. Ullman. 1972. The Theory of Parsing, Translation, and Compiling. Prentice Hall, Englewood Cliffs, NJ. find
2. S. Bangalore, O. Rambow, and S. Whittaker. 2000. Evaluation metrics for generation. In Proc. INLG-00, pages 1–8, Mitzpe Ramon, Israel, July. find
3. A. Belz and E. Reiter. 2006. Comparing automatic and human evaluation of NLG systems. In Proc. EACL-06, pages 313–320, Trento, Italy, April. find
4. P. F. Brown, V. J. Della Pietra, S. A. Della Pietra, and R. L. Mercer. 1993. The mathematics of statistical machine translation: Parameter estimation. Computational Linguistics, 19(2):263–312, June. find
5. J. Carroll and S. Oepen. 2005. High efficiency realization for a wide-coverage unification grammar. In Proc. IJCNLP-05, pages 165–176, Jeju Island, Korea, October. find
6. J. Carroll, A. Copestake, D. Flickinger, and V. Poznański. 1999. An efficient chart generator for (semi-)lexicalist grammars. In Proc. EWNLG-99, pages 86–95, Toulouse, France. find
7. M. Chen et al. 2003. Users manual: RoboCup soccer server manual for soccer server version 7.07 and later. Available at http://sourceforge.net/projects/sserver/. ↗
8. D. Chiang. 2005. A hierarchical phrase-based model for statistical machine translation. In Proc. ACL-05, pages 263–270, Ann Arbor, MI, June. find our card
9. G. Doddington. 2002. Automatic evaluation of machine translation quality using n-gram co-occurrence statistics. In Proc. ARPA Workshop on Human Language Technology, pages 128–132, San Diego, CA. find
10. U. Germann. 2003. Greedy decoding for statistical machine translation in almost linear time. In Proc. HLT/NAACL-03, Edmonton, Canada. find
11. P. S. Jacobs. 1985. PHRED: A generator for natural language interfaces. Computational Linguistics, 11(4):219–242. find
12. R. J. Kate, Y. W. Wong, and R. J. Mooney. 2005. Learning to transform natural to formal languages. In Proc. AAAI-05, pages 1062–1068, Pittsburgh, PA, July. find
13. M. Kay. 1975. Syntactic processing and functional sentence perspective. In Theoretical Issues in Natural Language Processing— Supplement to the Proceedings, pages 12–15, Cambridge, MA, June. find
14. M. Kay. 1996. Chart generation. In Proc. ACL-96, pages 200–204, San Francisco, CA. find
15. K. Knight and V. Hatzivassiloglou. 1995. Two-level, many-paths generation. In Proc. ACL-95, pages 252–260, Cambridge, MA. find
16. P. Koehn and C. Monz. 2006. Manual and automatic evaluation of machine translation between European languages. In Proc. SMT-06 Workshop, pages 102–121, New York City, NY, June. find
17. P. Koehn, F. J. Och, and D. Marcu. 2003. Statistical phrase-based translation. In Proc. HLT/NAACL-03, Edmonton, Canada. find
18. G. Kuhlmann, P. Stone, R. J. Mooney, and J. W. Shavlik. 2004. Guiding a reinforcement learner with natural language advice: Initial results in RoboCup soccer. In Proc. of the AAAI-04 Workshop on Supervisory Control of Learning and Adaptive Systems, San Jose, CA, July. find
19. I. Langkilde-Geary. 2002. An empirical verification of coverage and correctness for a general-purpose sentence generator. In Proc. INLG-02, pages 17–24, Harriman, NY, July. find
20. R. C. Moore. 2002. A complete, efficient sentence-realization algorithm for unification grammar. In Proc. INLG-02, pages 41–48, Harriman, NY, July. find
21. H. Nakanishi, Y. Miyao, and J. Tsujii. 2005. Probabilistic models for disambiguation of an HPSG-based chart generator. In Proc. IWPT-05, pages 93–102, Vancouver, Canada, October. find
22. J. Nocedal. 1980. Updating quasi-Newton matrices with limited storage. Mathematics of Computation, 35(151):773–782, July. find
23. F. J. Och and H. Ney. 2003. A systematic comparison of various statistical alignment models. Computational Linguistics, 29(1):19–51. find
24. F. J. Och. 2003. Minimum error rate training in statistical machine translation. In Proc. ACL-03, pages 160–167, Sapporo, Japan, July. find
25. K. Papineni, S. Roukos, T. Ward, and W. J. Zhu. 2002. BLEU: a method for automatic evaluation of machine translation. In Proc. ACL-02, pages 311–318, Philadelphia, PA, July. find our card
26. S. M. Shieber. 1988. A uniform architecture for parsing and generation. In Proc. COLING-88, pages 614–619, Budapest, Hungary. find
27. S. M. Shieber. 1993. The problem of logical-form equivalence. Computational Linguistics, 19(1):179–190. find
28. A. Stolcke. 2002. SRILM—an extensible language modeling toolkit. In Proc. ICSLP-02, pages 901–904, Denver, CO. find
29. M. White and J. Baldridge. 2003. Adapting chart realization to CCG. In Proc. EWNLG-03, Budapest, Hungary, April. find
30. M. White. 2004. Reining in CCG chart realization. In Proc. INLG-04, New Forest, UK, July. find
31. P. Whitelock. 1992. Shake-and-bake translation. In Proc. COLING-92, pages 784–791, Nantes, France. find our card
32. Y. W. Wong and R. J. Mooney. 2006. Learning for semantic parsing with statistical machine translation. In Proc. HLT/NAACL-06, pages 439–446, New York City, NY, June. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/wong-2007-generation-inverting-semantic-parser-that/