Towards broad coverage surface realization with CCG

What it is about

A progress report on the first broad-coverage English surface realiser for Combinatory Categorial Grammar, built on OpenCCG. The grammar is extracted from CCGbank, a CCG version of the Penn Treebank. The authors revise the derivations, then attach logical forms to categories by XSLT transformations and templates. About 95% of development and 94% of test sentences get a logical form, and about two thirds of them have a single root. Realisation is a hybrid: a chart search, either anytime best-first with a time limit or two-stage packed, ranked by n-gram models over factored words (surface form, part of speech, supertag). Missing words are handled by a smoothing rule that copies the categories of the most frequent words of the same part of speech. On the non-blind development grammar, exact matches reach about 15% and BLEU about 0.66 for the best model; on the blind test section BLEU runs from about 0.37 to about 0.58 across models. The factored models beat word-only trigrams, while the part-of-speech-only and null scorers are weakest.

Ideas

• I1. Factored counts without a neural model. The translator's standard encoder chooses the most frequent chunk in the most specific context. Add a backoff over factors (word, coarse tag, role) as BTreeMap keys, and measure exact match on dev verses against the current choice.
• I2. A single-root rate per author. The share of verses whose meta-world is one connected tree is a cheap diagnostic of reader attachment noise (open problem 1). Report it per author next to the residual.
• I3. Better fallback for unseen concepts. The paper gives an out-of-vocabulary word the categories of frequent words of its part of speech. Mova gives an unseen concept a bare form from morph; try the most frequent chunk among concepts with the same coarse tag and role, and measure on held-out concepts.
• I4. Make the fallback visible. When search fails the paper greedily joins fragments. The translator should report each verse where its output comes from a fallback, so that such verses count against coverage and cannot pass as exact matches.

For Mova

The code is tools/translate/src/profile.rs (encoder), tools/translate/src/measure.rs (reporting) and en/src/morph.rs (bare forms). The ideas touch open problems 1 (attachment noise), 2 (lexicon noise on unseen concepts) and 6 (corpus weighting). The factor counts in I1 must be built from counts, not from a learned model, to stay within the deterministic rule.

Topics: Part-of-speech tagging · Machine translation

Cite

The paper itself, not this card. BibTeX from the library's record of the work.

@inproceedings{white2007broad,
  title = {{Towards broad coverage surface realization with CCG}},
  author = {White, Michael and Rajkumar, Rajakrishnan and Martin, Scott},
  booktitle = {Proceedings of the Workshop on Using corpora for natural language generation},
  year = {2007},
  url = {https://aclanthology.org/2007.mtsummit-ucnlg.4/}
}

References

31 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Jason Baldridge and Geert-Jan Kruijff. 2002. Coupling CCG and Hybrid Logic Dependency Semantics. In Proc. ACL-02. find
2. Jason Baldridge and Geert-Jan Kruijff. 2003. Multi-Modal Combinatory Categorial Grammar. In Proc. ACL-03. find
3. Jason Baldridge. 2002. Lexically Specified Derivational Control in Combinatory Categorial Grammar. Ph.D. thesis, School of Informatics, University of Edinburgh. find
4. Srinivas Bangalore and Owen Rambow. 2000. Exploiting a probabilistic hierarchical model for generation. In Proce. COLING-00. find
5. Jeff Bilmes and Katrin Kirchhoff. 2003. Factored language models and general parallelized backoff. In Proc. HLT-03. find
6. Alexandra Birch, Miles Osborne, and Philipp Koehn. 2007. CCG supertags in factored statistical machine translation. In Proc. SMT Workshop at ACL-07. find
7. Aoife Cahill and Josef van Genabith. 2006. Robust PCFG-based generation using automatically acquired LFG approximations. In Proc. COLING-ACL ’06. find
8. Charles Callaway. 2003. Evaluating coverage for large symbolic NLG grammars. In Proc. IJCAI-03. find
9. Chris Callison-Burch, Miles Osborne, and Philipp Koehn. 2006. Re-evaluating the role of BLEU in machine translation research. In Proc. EACL-06. find
10. John Carroll and Stefan Oepen. 2005. High efficiency realization for a wide-coverage unification grammar. In Proc. IJCNLP-05. find
11. John Carroll, Ann Copestake, Dan Flickinger, and Victor Poznański. 1999. An efficient chart generator for (semi-) lexicalist grammars. In Proc. ENLG-99. find
12. Stephen Clark and James R. Curran. 2004. The importance of supertagging for wide-coverage CCG parsing. In Proc. COLING-04. find our card
13. James R. Curran, Stephen Clark, and David Vadas. 2006. Multi-tagging for lexicalized-grammar parsing. In Proc. COLING-ACL ’06. find
14. Christine Doran. 1998. Incorporating Punctuation Into The Sentence Grammar: A Lexicalized Tree Adjoining Grammar Perspective. Ph.D. thesis, University of Pennsylvania. Technical Report IRCS-98-24. find
15. Martin C. Emele and Michael Dorna. 1998. Ambiguity preserving machine translation using packed representations. In Proc. of COLING-ACL ’98. find
16. Mary Ellen Foster and Michael White. 2007. Avoiding repetition in generated text. In Proc. ENLG-07. find
17. Julia Hockenmaier. 2003. Data and Models for Statistical Parsing with Combinatory Categorial Grammar. Ph.D. thesis, School of Informatics, University of Edinburgh. find
18. Martin Kay. 1996. Chart generation. In Proc. ACL-96. find our card
19. Irene Langkilde-Geary. 2002. An empirical verification of coverage and correctness for a general-purpose sentence generator. In Proc. INLG-02. find
20. Irene Langkilde. 2000. Forest-based statistical sentence generation. In Proc. NAACL-00. find
21. Jan Tore Lønning and Stephan Oepen. 2006. Reusable tools for precision machine translation. In Proc. COLING-ACL ’06 Demo Sessions. find
22. Hiroko Nakanishi, Yusuke Miyao, and Jun’ichi Tsujii. 2005. Probabilistic methods for disambiguation of an HPSG-based chart generator. In Proc. IWPT-05. find
23. Crystal Nakatsu and Michael White. 2006. Learning to say it well: Reranking realizations by predicted synthesis quality. In Proc. COLING-ACL ’06. find
24. Martha Palmer, Dan Gildea, and Paul Kingsbury. 2005. The proposition bank: A corpus annotated with semantic roles. Computational Linguistics, 31(1). find
25. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2001. Bleu: a Method for Automatic Evaluation of Machine Translation. Technical Report RC22176, IBM. find our card
26. Mark Steedman. 2000. The Syntactic Process. MIT Press. find
27. Amanda Stent, Matthew Marge, and Mohit Singhai. 2005. Evaluating evaluation methods for generation in the presence of variation. In Proc. CICLing-05. find
28. Andreas Stolcke. 2002. SRILM — An extensible language modeling toolkit. In Proc. ICSLP-02. find
29. Erik Velldal and Stephan Oepen. 2005. Maximum entropy models for realization ranking. In Proc. MT Summit X. find
30. Michael White. 2006a. CCG chart realization from disjunctive inputs. In Proceedings, INLG 2006. find
31. Michael White. 2006b. Efficient realization of coordinate structures in Combinatory Categorial Grammar. Research on Language and Computation, 4(1):39–75. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/white-2007-broad-coverage-surface-realization-ccg/