GumDrop at the DISRPT2019 Shared Task: A Model Stacking Approach to Discourse Unit Segmentation and Connective Detection

What it is about

GumDrop is a shared-task system for sentence splitting, elementary discourse unit segmentation and connective detection across 15 corpora in 10 languages (RST, SDRT and PDTB styles). Each task has an ensemble of heterogeneous modules — a character-feature logistic regression, a dependency-subtree classifier (the candidate token's relation, parent and grandparent, nearest and farthest children's relations, span lengths, clause brackets), an RNN, a bag-of-words counter of segments per sentence, rule-based baselines — whose out-of-fold predictions feed a tree-based metalearner. Out-of-fold training (5 folds, each module predicting data it did not see) keeps the metalearner from over-trusting modules. It set a new best segmentation on RST-DT (F 0.957 gold syntax) and was robust across corpora; on GUM with predicted syntax segmentation drops to F 0.85. Errors cluster on words that are both preposition and subordinator (to, by, after, before) and on coordination, where corpora disagree whether coordinated VPs are separate units.

Ideas

• I1. Replace the hand-written agreer between Mova's three prag models with stacking done the paper's way: train perceptron, IGTree and k-NN with 5-fold out-of-fold predictions on the silver, then a small transparent metalearner (a decision list or IGTree over their labels, confidences and two or three context slots) learns when to trust which. Compare against the current agreement rule and the «disagreement → teacher» trigger in prag — Mova pragmatics: LLM silver and a transparent SLM on act and indirect macro-F1, and check that the metalearner's choices are explainable.
• I2. For Mova's frame splitting the ambiguous words are the same as GumDrop's: after/before/until/since as preposition vs subordinator. A frame (and a next link) should come only from the clausal reading (mark of an advcl), never from case of an obl («after dinner»). Add minimal-pair tests in prag/src/frames.rs («After dinner he slept» vs «After he ate, he slept») and count misparses on SLM trees.

For Mova

36. Discourse relations and coherence §10 (segmentation and parsing), 31-speech-acts.md §9 (how acts are labelled). Code: prag/src/model.rs, prag/src/agree.rs (stacking), prag/src/frames.rs (clausal vs prepositional tests). Measures: act/indirect macro-F1 in prag eval; minimal-pair test results; frame bench next precision.

Topics: Discourse analysis

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/w19-2717

@inproceedings{yu2019gumdrop,
  title = {{GumDrop at the DISRPT2019 Shared Task: A Model Stacking Approach to Discourse Unit Segmentation and Connective Detection}},
  author = {Yu, Yue and Zhu, Yilun and Liu, Yang and Liu, Yan and Peng, Siyao and Gong, Mackenzie and Zeldes, Amir},
  booktitle = {Proceedings of the Workshop on Discourse Relation Parsing and Treebanking 2019},
  year = {2019},
  doi = {10.18653/v1/w19-2717},
  url = {https://doi.org/10.18653/v1/w19-2717}
}

References

32 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Nicholas Asher. 1993. Reference to Abstract Objects in Discourse. Kluwer, Dordrecht. ↗
2. Steven Bird, Edward Loper, and Ewan Klein. 2009. Natural Language Processing with Python. O’Reilly, Sebastopol, CA. find
3. Daniel Marcu. 2000. The Theory and Practice of Discourse Parsing and Summarization. MIT Press, Cambridge, MA. ↗
4. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. Enriching word vectors with subword information. TACL, 5:135–146. ↗
5. Daniel Marcu, Estibaliz Amorrortu, and Magdalena Romera. 1999. Experiments in constructing a corpus of discourse trees. In Proceedings of the ACL Workshop Towards Standards and Tools for Discourse Tagging, pages 48–57, College Park, MD. find
6. Chloé Braud, Maximin Coavoux, and Anders Søgaard. 2017a. Cross-lingual RST discourse parsing. In Proceedings of EACL 2017, pages 292–304, Valencia, Spain. ↗
7. Mitchell P. Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of English: The Penn Treebank. Special Issue on Using Large Corpora, Computational Linguistics, 19(2):313–330. ↗ our card
8. Chloé Braud, Ophélie Lacroix, and Anders Søgaard. 2017b. Does syntax help discourse segmentation? not so much. In Proceedings of EMNLP 2017, pages 2432–2442, Copenhagen. find
9. Mathieu Morey, Philippe Muller, and Nicholas Asher. 2017. How much progress have we made on RST discourse parsing? a replication study of recent results on the RST-DT. In Proceedings of EMNLP 2017, pages 1319–1324, Copenhagen, Denmark. find our card
10. Lynn Carlson, Daniel Marcu, and Mary Ellen Okurowski. 2003. Building a discourse-tagged corpus in the framework of Rhetorical Structure Theory. In Current and New Directions in Discourse and Dialogue, Text, Speech and Language Technology 22, pages 85–112. Kluwer, Dordrecht. ↗ our card
11. Gary Patterson and Andrew Kehler. 2013. Predicting the presence of discourse connectives. In Proceedings of EMNLP 2013, pages 914–923. find
12. Tianqi Chen and Carlos Guestrin. 2016. XGBoost: A scalable tree boosting system. In KDD ’16 Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 785–794, San Francisco, CA. ↗
13. Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, and Vincent Dubourg. 2011. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830. ↗
14. Kevin Clark and Christopher D. Manning. 2015. Entity-centric coreference resolution with model stacking. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (ACL-IJCNLP 2015), pages 1405–1415, Beijing. find
15. Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. GloVe: Global vectors for word representation. In Proceedings of EMNLP 2014, pages 1532–1543, Doha, Qatar. find
16. Emily Pitler and Ani Nenkova. 2009. Using syntax to disambiguate explicit discourse connectives in text. In Proceedings of the ACL-IJCNLP 2009 Conference Short Papers, pages 13–16, Suntec, Singapore. find our card
17. Jasper Friedrichs, Debanjan Mahata, and Shubham Gupta. 2017. InfyNLP at SMM4H task 2: Stacked ensemble of shallow convolutional neural networks for identifying personal medication intake from Twitter. In Proceedings of SMM4H@AMIA 2017, Washington, DC. ↗
18. Balaji Polepalli Ramesh, Rashmi Prasad, Tim Miller, Brian Harrington, and Hong Yu. 2012. Automatic discourse connective detection in biomedical text. Journal of the American Medical Informatics Association, 19(5):800–808. ↗
19. Hugo Hernault, Helmut Prendinger, David A. duVerle, and Mitsuru Ishizuka. 2010. HILDA: A discourse parser using support vector machine classification. Dialogue and Discourse, 1(3):1–33. find
20. Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Miltsakaki, Livio Robaldo, Aravind Joshi, and Bonnie Webber. 2008. The Penn Discourse Treebank 2.0. In Proceedings of the 6th International Conference on Language Resources and Evaluation (LREC 2008), pages 2961–2968, Marrakesh, Morocco. find our card
21. Alistair Knott. 1996. A Data-Driven Methodology for Motivating a Set of Coherence Relations. Ph.D. thesis, University of Edinburgh, University of Edinburgh. find
22. Rashmi Prasad, Susan McRoy, Nadya Frid, Aravind Joshi, and Hong Yu. 2011. The Biomedical Discourse Relation Bank. BMC bioinformatics, 12(1):188. ↗
23. Qi Li, Tianshi Li, and Baobao Chang. 2016. Discourse parsing with attention-based hierarchical neural networks. In Proceedings of EMNLP 2016, pages 362– 371, Austin, TX. find
24. Rashmi Prasad, Bonnie Webber, and Aravind Joshi. 2014. Reflections on the Penn Discourse Treebank, comparable corpora, and complementary annotation. Computational Linguistics, 40(4):921–950. ↗ our card
25. William C. Mann and Sandra A. Thompson. 1988. Rhetorical Structure Theory: Toward a functional theory of text organization. Text, 8(3):243–281. ↗ our card
26. Helmut Schmid. 1994. Probabilistic part-of-speech tagging using decision trees. In Proceedings of the Conference on New Methods in Language Processing, pages 44–49, Manchester, UK. find
27. Radu Soricut and Daniel Marcu. 2003. Sentence level discourse parsing using syntactic and lexical information. In Proceedings of HLT-NAACL 2003, pages 149–156, Edmonton. find our card
28. Caroline Sporleder and Mirella Lapata. 2005. Discourse chunking and its application to sentence compression. In Proceedings of EMNLP 2005, pages 257–264, Vancouver. find
29. Huong Le Thanh, Geetha Abeysinghe, and Christian Huyck. 2004. Generating discourse structures for written text. In Proceedings of COLING 2004, pages 329–335, Geneva, Switzerland. find
30. David H. Wolpert. 1992. Stacked generalization. Neural Networks, 5(2):241–259. find
31. Jie Yang and Yue Zhang. 2018. NCRF++: An open-source neural sequence labeling toolkit. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, pages 74–79, Melbourne. find
32. Amir Zeldes. 2016. rstWeb - a browser-based annotation interface for Rhetorical Structure Theory and discourse relations. In Proceedings of NAACL-HLT 2016 System Demonstrations, pages 1–5, San Diego, CA. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/yu-2019-gumdrop-disrpt2019-shared-task-model/