'The parse is darc and full of errors: Universal dependency parsing with transition-based and graph-based algorithms'

What it is about

The authors built two simple parsers for the CoNLL 2017 UD task: darc, a transition-based arc-standard parser with a swap action and a feedforward network, which was the official system and ranked 12th of 33, and mstnn, a graph-based parser that was not ranked. The paper is a post-mortem: it asks why the simple parser lands just above the baseline. The explanations are that the pipeline errors snowball, that using lemmas where they exist helped the lexical space, that treating FEATS as atomic symbols causes sparsity, and that training on the full training sets beat the baseline's split. mstnn lost about four points more from UAS to LAS than darc did, which the authors read as the cost of its separate labeller in a pipeline.

Ideas

• I1. Tagger-cost measure: run en parse-eval on EWT dev with gold UPOS and FEATS and with predicted ones; the LAS gap is what the tagging stage costs the parser, and should be reported before any parser change, following the snowball argument.
• I2. Lemma as a second lexical space in en/src/parse.rs: add the en/src/morph.rs lemma as a feature beside FORM and split the feature table between them, as darc does; measure EWT dev LAS against the current FORM-only features.
• I3. FEATS as key-value backoff: in en/src/ud.rs back off from a full FEATS bundle to single key-value pairs; count types and hapax on EWT train, then measure UFeats and LAS on EWT dev (test only once per release).

For Mova

The parser is en/src/parse.rs (arc-standard, averaged perceptron, beam 8), the morphology is en/src/morph.rs and en/src/ud.rs, and the evaluation is en parse-eval. The chapter is 13. Verbs for the FEATS side and 20. Syntax: the Universal Dependencies frame for the parser. Measure first the tagger-cost gap on EWT dev.

Topics: Dependency parsing

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/k17-3013

@inproceedings{yu2017parse,
  title = {{'The parse is darc and full of errors: Universal dependency parsing with transition-based and graph-based algorithms'}},
  author = {Yu, Kuan and Sofroniev, Pavel and Schill, Erik and Hinrichs, Erhard},
  booktitle = {Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies},
  year = {2017},
  doi = {10.18653/v1/k17-3013},
  url = {https://doi.org/10.18653/v1/k17-3013}
}

References

2 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Our representation is explained in Section 3.2. We experimented with normalizing the FEATS vectors into unit vectors by their L 2 -norm, or into probability distributions by their L 1 -norm as in Alberti et al. (2015). But simple binary indicators seemed to work the best. find
2. Martı́n Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenbe… find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/ud-2017-yu-parse-is-darc-full-errors/