Enhancing PTB Universal Dependencies for Grammar-Based Surface Realization

What it is about

Grammar-based realisers work well when their input carries long-distance dependencies and coordination, but standard UD conversions from the Penn Treebank lose these, because the conversion ignores the traces. The authors take enhanced UD built from the PTB and add the dependencies found in CCGbank, which come from gold PTB traces. Then they adapt an existing CCG induction algorithm so that it can handle UD-style trees, where a phrase is often projected from a dependent such as a determiner rather than from its head. On the test data the induced derivations were complete for about 94 percent of the sentences. The paper reports no realisation scores. It is a progress report: the realisation experiments were still underway when it was written.

Ideas

• I1. Add shared-argument edges to the parser output. For a verb with a coordinated or long-distance argument, add the second governor as an extra edge in the UD tree, so the frames in tools/roundtrip hold the shared argument once for each verb. Count these cases on the English UD data first.
• I2. Report completeness the way the paper does: the share of sentences for which the frame and world reading covers every token. Track it per release, and keep the count deterministic with BTreeMap counts.
• I3. Test the coordination case in the translator. A coordinated subject shared by two verbs is one meta token with two heads, which is the head-attachment noise of open problem 1. Pick the coordination verses from the parallel Bible pairs and count how many change attachment across translations.

More ideas (a second reading)

• I4. en/src/convert.rs and the world builder: add enhanced edges (the controlled subject, the relative pronoun's antecedent, the free-relative head) as an optional MISC layer, not as a change to the basic tree; measure recall of unbounded-dependency edges on a set we can release, and the LAS on the basic tree, which must stay unchanged.
• I5. en/src/ud.rs or a treeagree rule: for a free relative clause, make the wh-word the head that the copula points to and remap ccomp or csubj to the relative; measure the ccomp and csubj error count on EWT dev before and after.
• I6. Precision protocol: sample 400 added edges, judge each by hand against UD rules, and report the share that the Mova registry gate (en/src/udreg.rs) accepts; the paper's 70% exact-label precision is the bar to beat.
• I7. en/src/lin.rs: take the enhanced edges as extra input to the word-order generator for relative clauses and free relatives; measure exact word-order match on held-out sentences given gold basic trees, with and without the extra edges.

For Mova

The paper bears on the parser side and the frames, not on the translator's encoder. The idea that carries over is the enhancement of trees with extra edges. The fit is en (parser output), prag (frames) and tools/roundtrip. Because no realisation result was published, any gain here must be measured from scratch on Mova's own data.

Topics: Universal Dependencies · Machine translation

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/w16-6638

@inproceedings{king2016enhancing,
  title = {{Enhancing PTB Universal Dependencies for Grammar-Based Surface Realization}},
  author = {King, David L. and White, Michael},
  booktitle = {Proceedings of the 9th International Natural Language Generation conference},
  year = {2016},
  doi = {10.18653/v1/w16-6638},
  url = {https://doi.org/10.18653/v1/w16-6638}
}

References

16 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Yoav Artzi and Luke Zettlemoyer. 2013. Weakly supervised learning of semantic parsers for mapping instructions to actions. TACL, 1:49–62. ↗
2. Anja Belz, Michael White, Dominic Espinosa, Eric Kow, Deirdre Hogan, and Amanda Stent. 2011. The first surface realisation shared task: Overview and evaluation results. In Proc. ENLG. find our card
3. Marie-Catherine de Marneffe, Miriam Connor, Natalia Silveira, Samuel R Bowman, Timothy Dozat, and Christopher D Manning. 2013. More Constructions, More Genres: Extending Stanford Dependencies. In Proc. DepLing 2013. find our card
4. Claire Gardent and Shashi Narayan. 2013. Generating elliptic coordination. In Proceedings of the 14th European Workshop on Natural Language Generation, pages 40–50. find
5. Julia Hockenmaier and Mark Steedman. 2007. CCGbank: A Corpus of CCG Derivations and Dependency Structures Extracted from the Penn Treebank. Computational Linguistics, 33(3):355–396. ↗ our card
6. Tom Kwiatkowski, Luke Zettlemoyer, Sharon Goldwater, and Mark Steedman. 2011. Lexical generalization in CCG grammar induction for semantic parsing. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 1512–1523. find
7. Mitchell P. Marcus, Mary Ann Marcinkiewicz, and Beatrice Santorini. 1993. Building a large annotated corpus of English: The Penn Treebank. Computational linguistics, 19(2):313–330. ↗ our card
8. Shashi Narayan and Claire Gardent. 2012. Error mining with suspicion trees: Seeing the forest for the trees. In Proc. COLING. find
9. Luan Nguyen, Marten Van Schijndel, and William Schuler. 2012. Accurate unbounded dependency recovery using generalized categorial grammars. In Proceedings of COLING 2012, pages 2125– 2140. find
10. Joakim Nivre, Marie-Catherine de Marneffe, Filip Ginter, Yoav Goldberg, Jan Hajič, Christopher Manning, Ryan McDonald, Slav Petrov, Sampo Pyysalo, Natalia Silveira, et al. 2016. Universal dependencies v1: A multilingual treebank collection. In Proceedings of the 10th International Conference on Language Resources and Evaluation (LREC 2016). find our card
11. Stephan Oepen, Marco Kuhlmann, Yusuke Miyao, Daniel Zeman, Dan Flickinger, Jan Hajic, Angelina Ivanova, and Yi Zhang. 2014. Semeval 2014 task 8: Broad-coverage semantic dependency parsing. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 63–72. find
12. Laura Rimell, Stephen Clark, and Mark Steedman. 2009. Unbounded Dependency Recovery for Parser Evaluation. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 2-Volume 2, pages 813–821. find our card
13. Sebastian Schuster and Christopher D. Manning. 2016. Enhanced english universal dependencies: An improved representation for natural language understanding tasks. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016). find our card
14. Mark Steedman and Jason Baldridge. 2011. Combinatory Categorial Grammar. Non-Transformational Syntax: Formal and Explicit Models of Grammar. Wiley-Blackwell. ↗
15. Mark Steedman. 2000. The syntactic process. MIT Press, Cambridge, MA, USA. ↗
16. Michael White. 2014. Towards surface realization with CCGs induced from dependencies. In Proceedings of the 8th International Natural Language Generation Conference (INLG), pages 147–151. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/king-2016-enhancing-ptb-universal-dependencies-grammar/