Underspecified Universal Dependency Structures as Inputs for Multilingual Surface Realisation

What it is about

The paper describes how the input data of the SR'18 task were derived from UD v2.0 treebanks. The shallow inputs are made by removing word order and lemmatising tokens, which is a set of replacement rules. The deep inputs are harder to make. Auxiliaries, functional prepositions and conjunctions are removed, and surface morphology is dropped. Where the annotation does not allow a decision about an argument slot, the slot is left undefined instead of guessed. The authors measured the deep inputs against human-corrected versions. For English the labelled agreement was about 80. The paper also asks whether NLU annotations, which were built for understanding and not for generation, give a realiser enough of the right information. Its answer is only partly: UD structures were not designed for NLG, and they are a close but imperfect approximation of meaning representations.

Ideas

• I1. Make underspecification explicit in the meta-world. When the reader cannot decide a role, mark the slot as undefined instead of attaching the token to a guessed head. Count how often this happens per author and per language, and report it next to the residual.
• I2. Measure the reader against a corrected reference, as the paper does with human-corrected deep inputs. Take a sample of verses, have a person fix the frames read by the roundtrip stack, and report the labelled agreement for the English and Hebrew readers. Hold the sample fixed across runs.
• I3. Run a deep-track style ablation on the meta-world. Remove the function words from the meta tokens and measure how much the standard can recover. This shows how much of the function-word information (open problem 4) the meta-world must carry.

For Mova

The paper is about input design, so it touches the reader side of the translator. It fits tools/roundtrip (tokens to tree to frames), prag and tools/translate (meta.rs, hebrew.rs). The explicit undefined slot is a small change to the frame format and can be counted with a plain table. The agreement check against corrected frames is the cheapest honest measure of the reader.

Topics: Dependency parsing · Machine translation

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/w18-6527

@inproceedings{mille2018underspecified,
  title = {{Underspecified Universal Dependency Structures as Inputs for Multilingual Surface Realisation}},
  author = {Mille, Simon and Belz, Anja and Bohnet, Bernd and Wanner, Leo},
  booktitle = {Proceedings of the 11th International Conference on Natural Language Generation},
  year = {2018},
  doi = {10.18653/v1/w18-6527},
  url = {https://doi.org/10.18653/v1/w18-6527}
}

References

23 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Miguel Ballesteros, Bernd Bohnet, Simon Mille, and Leo Wanner. 2015. Data-driven deep-syntactic dependency parsing. Natural Language Engineering, pages 1–36. find
2. John Bateman. 1997. Enabling technology for multilingual natural language generation: The KPML development environment. Natural Language Engineering Journal, 3(1):15–55. find
3. Anja Belz, Michael White, Dominic Espinosa, Eric Kow, Deirdre Hogan, and Amanda Stent. 2011. The first surface realisation shared task: Overview and evaluation results. In Proceedings of the 13th European Workshop on Natural Language Generation, ENLG ’11, pages 217–226, Stroudsburg, PA, USA. Association for Computational Linguistics. find our card
4. Adam Meyers, R. Reeves, C. Macleod, R. Szekely, V. Zielinska, B. Young, and R. Grishman. 2004. The NomBank project: An interim report. In HLT-NAACL 2004 Workshop: Frontiers in Corpus Annotation, Boston, MA, May 2004, pages 24–31. find
5. Alena Böhmová, Silvie Cinková, and Eva Hajičová. 2005. A manual for tectogrammatical layer annotation of the Prague Dependency Treebank (English translation). Technical report, ÚFAL MFF UK, Prague, Czech Republic. find
6. Simon Mille, Anja Belz, Bernd Bohnet, Yvette Graham, Emily Pitler, and Leo Wanner. 2018. The First Multilingual Surface Realisation Shared Task (SR’18): Overview and Evaluation Results. In Proceedings of the 1st Workshop on Multilingual Surface Realisation (MSR), 56th Annual Meeting of the Association for Computational Linguistics (ACL), pages 1–12, Melbourne, Australia. find our card
7. Bernd Bohnet and Leo Wanner. 2010. Open soucre graph transducer interpreter and grammar development environment. Valletta, Malta. find
8. Aoife Cahill and Josef van Genabith. 2006. Robust PCFG-based generation using automatically acquired LFG approximations. In Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics, pages 1033– 1040. Association for Computational Linguistics. find
9. Simon Mille, Roberto Carlini, Ivan Latorre, and Leo Wanner. 2017. UPF at EPE 2017: Transduction-based deep analysis. In Shared Task on Extrinsic Parser Evaluation (EPE 2017), pages 80–88, Pisa, Italy. find
10. M. Elhadad and J. Robin. 1996. An overview of SURGE: a reusable comprehensive syntactic realization component. Technical report, Ben Gurion University in the Negev. find our card
11. Simon Mille and Leo Wanner. 2015. Towards large-coverage detailed lexical resources for data-to-text generation. In Proceedings of the First International Workshop on Data-to-text Generation, Edinburgh, Scotland. find
12. Claire Gardent, Anastasia Shimorina, Shashi Narayan, and Laura Perez-Beltrachini. 2017. Creating training corpora for micro-planners. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vancouver, Canada. Association for Computational Linguistics. find
13. Hiroko Nakanishi, Ysuke Miyao, and Jun’ichi Tsujii. 2005. Probabilistic models for disambiguation of an HPSG-based chart generator. In Proceedings of the International Workshop on Parsing Technologies. find
14. Albert Gatt and Ehud Reiter. 2009. Simplenlg: A realisation engine for practical applications. In Proceedings of the 12th European Workshop on Natural Language Generation, pages 90–93. Association for Computational Linguistics. find
15. Jekaterina Novikova, Ondrej Dušek, and Verena Rieser. 2017. The E2E dataset: New challenges for end-to-end generation. In Proceedings of the 18th Annual Meeting of the Special Interest Group on Discourse and Dialogue, Saarbrücken, Germany. ArXiv:1706.09254. ↗
16. Irene Langkilde-Geary. 2002. An empirical verification of coverage and correctness for a general-purpose sentence generator. In Proceedings of the International Natural Language Generation Conference (INLG’02). find
17. Martha Palmer, Daniel Gildea, and Paul Kingsbury. 2005. The proposition bank: An annotated corpus of semantic roles. Computational Linguistics, 31(1):71–105. find our card
18. Percy Liang, Michael I Jordan, and Dan Klein. 2009. Learning semantic correspondences with less supervision. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 1-Volume 1, pages 91–99. Association for Computational Linguistics. find
19. Mihai Surdeanu, Richard Johansson, Adam Meyers, Lluı́s Màrquez, and Joakim Nivre. 2008. The CoNLL-2008 shared task on joint parsing of syntactic and semantic dependencies. In Proceedings of the International Conference on Computational Linguistics and the Annual Meeting of the Association for Computational Linguistics (COLING-ACL’08). find our card
20. Marie-Catherine de Marneffe, Timothy Dozat, Natalia Silveira, Katri Haverinen, Filip Ginter, Joakim Nivre, and Christopher D. Manning. 2014. Universal stanford dependencies: A cross-linguistic typology. pages 4585–4592, Reykjavik, Iceland. European Language Resources Association (ELRA). ACL Anthology Identifier: L14-1045. ↗ our card
21. Michael White and Rajakrishnan Rajkumar. 2009. Perceptron reranking for CCG realization. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’09). find
22. Daniel Zeman, Martin Popel, Milan Straka, Jan Hajic, Joakim Nivre, Filip Ginter, Juhani Luotolahti, Sampo Pyysalo, Slav Petrov, Martin Potthast, et al. 2017. Conll 2017 shared task: multilingual parsing from raw text to universal dependencies. Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pages 1–19. find our card
23. Huayan Zhong and Amanda Stent. 2005. Building surface realizers automatically from corpora using general-purpose tools. In Proceedings of the Workshop on Using Corpora for Natural Language Generation (UCNLG). find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/mille-2018-underspecified-universal-dependency-structures-as/