The DISRPT 2019 Shared Task on Elementary Discourse Unit Segmentation and Connective Detection

What it is about

Discourse parsing needs a first step: cutting text into elementary discourse units (EDUs), usually clauses, and finding explicit connectives such as "because" or "in order to". The paper reports the first shared task on both steps. It covers 15 datasets in 10 languages, about 2.6 million manually annotated tokens. Twelve datasets are for segmentation and three for connective detection. Systems were scored by micro-averaged precision, recall and F1 within each dataset, macro-averaged across datasets, in two scenarios: treebanked input with gold sentence splits and POS tags, and plain tokens only. Four systems were submitted. The best, ToNy, is a sequence labeller on contextual BERT embeddings with whole-document chunks. It averages F1 90.11 on treebanked segmentation and 86.38 on plain tokens. On RST-DT it reaches 96.04 (treebanked), above the prior 93.7, and 93.43 on plain text, above 89.5. GumDrop averages 88.11 and IXA 87.18. For connectives, ToNy averages 79.25 on treebanked data and 83.63 on plain, and gets 88.60 F1 on the English PDTB in the treebanked setting. The authors note that segmentation is easy at sentence starts, and that hard cases lie inside sentences.

Ideas

• I1. en/src/parse.rs and en/src/tok.rs: a rule-based EDU segmenter from the UD tree. Cut at advcl, ccomp, conj clauses and relative clauses, keep complement and restrictive clauses together (the paper's definition). Score on any open EDU data for which the licence allows evaluation, and compare the plain-token setting with the parse setting to measure what the parse adds.
• I2. Connective detection as a lexicon with a disambiguation rule: an inventory of explicit connectives (BTreeMap), each with the frame that makes it a connective ("in order to" before a verb, not "in order" as a noun). The paper says the task needs memorised vocabulary, so count coverage first.
• I3. Measure the gap between treebanked and plain input on Mova's own text: run the segmenter with and without the parse and report the difference, as the paper does, so the parser's share is visible.
• I4. Use the paper's precision-recall pattern as a diagnostic: GumDrop has higher recall, ToNy higher precision. Mova's rules can be tuned on the same axis, and the trade-off reported as a table.

For Mova

The chapters are 36. Discourse relations and coherence (connectives and relations) and 24. Coordination and subordination (clause boundaries). The code is en/src/parse.rs for the tree, en/src/tok.rs for tokens, and prag/src/frames.rs for the discourse layer. The measure: EDU boundary F1 on treebanked and plain input, run twice for determinism, and the connective coverage count. Note: the datasets that use LDC or Wall Street Journal text are for evaluation only and never enter a build.

Topics: Discourse analysis

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/w19-2713

@inproceedings{zeldes2019disrpt,
  title = {{The DISRPT 2019 Shared Task on Elementary Discourse Unit Segmentation and Connective Detection}},
  author = {Zeldes, Amir and Das, Debopam and Maziero, Erick Galani and Antonio, Juliano and Iruskieta, Mikel},
  booktitle = {Proceedings of the Workshop on Discourse Relation Parsing and Treebanking 2019},
  year = {2019},
  doi = {10.18653/v1/w19-2713},
  url = {https://doi.org/10.18653/v1/w19-2713}
}

References

4 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Stergos Afantenos, Nicholas Asher, Farah Benamara, Myriam Bras, Ccile Fabre, Mai Ho-dac, Anne Le Draoulec, Philippe Muller, Marie-Paule Pry-Woodley, Laurent Prvot, Josette Rebeyrolle, Ludovic Tanguy, Marianne Vergez-Couret, and Laure Vieu. 2012. An empirical resource for discovering cognitive principles of discourse organisation: The ANNODIS corpus. In Proceedings of LREC 2012, pages 2727–2734, Istanbul, Turkey. find
2. Stergos Afantenos, Pascal Denis, Philippe Muller, and Laurence Danlos. 2010. Learning recursive segments for discourse parsing. arXiv preprint arXiv:1003.5372. ↗
3. Nicholas Asher. 1993. Reference to Abstract Objects in Discourse. Kluwer, Dordrecht. ↗
4. Nicholas Asher, Julie Hunter, Mathieu Morey, Farah Benamara, and Stergos D. Afantenos. 2016. Discourse structure and dialogue acts in multiparty dialogue: the STAC corpus. In Proceedings of LREC 2016, pages 2721–2727, Portorož, Slovenia. find our card

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/zeldes-2019-disrpt-2019-shared-task-elementary/