Comparing PTB and UD information for PDTB discourse connective identification

What it is about

Deciding whether a connective candidate (when, and, since, instead) is used as a discourse connective is the first step of shallow discourse parsing. Classic feature-based classifiers on PDTB used Penn Treebank constituents: the connective string alone already gives about 86% accuracy, and syntactic context (the category covering the connective, its parent and siblings, whether the right sibling contains a VP; later the neighbouring words and their tags, and the path to the root) brings it to about 96–99%. The authors reproduce Pitler & Nenkova 2009, Lin et al. 2014 and Li et al. 2016 with logistic regression, then replace the PTB information by Universal Dependencies (coarser tags, dependency relations, automatically parsed). UD information turns out to be enough: with predicted UD parses they obtain slightly better results than with gold PTB tags, which makes the method usable wherever only UD parsers exist.

Ideas

• I1. Write the connective-vs-non-connective decision in Mova as explicit UD rules rather than a learned classifier, since UD alone suffices: a subordinator counts when it is mark of an advcl (or advmod of an advcl for when/where), a coordinator when it is cc of a conjoined clause or VP, an adverbial (instead, then, so, however) when it is advmod of the main predicate at clause edge; when/where inside a relative or a wh-question are not connectives. Each rule is a named test that prints why. Measure link precision on the frame bench and against GUM's eRST discourse markers on dev (evaluation only).
• I2. Use the paper's feature set as a check-list for the cases rules miss: the token before the connective, the dependency relation of the connective and of its head, and whether the head is a verb. A small perceptron over these, trained on licence-clean silver (teacher-labelled tale sentences), can back up the rules for the ambiguous and, as, since, so; report it only if it beats the rules on held-out silver.
• I3. Measure on SLM trees as well as gold trees: the paper's point is that predicted UD parses do not hurt. The frame bench already runs on SLM trees; split link errors into parser errors (wrong advcl/mark) and rule errors.

For Mova

36. Discourse relations and coherence §6 (connectives and their syntax), §10, §11.1. Code: prag/src/frames.rs (connective tests), Subordinating conjunctions — mark. Measures: frame bench cause/next precision, connective precision vs GUM eRST markers on dev, error split by tree source.

Topics: Discourse analysis · Universal Dependencies

Cite

The paper itself, not this card. BibTeX from the library's record of the work.

@inproceedings{han2020comparing,
  title = {{Comparing PTB and UD information for PDTB discourse connective identification}},
  author = {Han, Kelvin and Leavitt, Phyllicia and Balard, Srilakshmi},
  booktitle = {Actes de la 6e conférence conjointe Journées d'Études sur la Parole (JEP, 33e édition), Traitement Automatique des Langues Naturelles (TALN, 27e édition), Rencontre des Étudiants Chercheurs en Informatique pour le Traitement Automatique des Langues (RÉCITAL, 22e édition). Volume 3 : Rencontre des Étudiants Chercheurs en Informatique pour le TAL},
  year = {2020},
  url = {https://aclanthology.org/2020.jeptalnrecital-recital.10/}
}

References

21 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. B RAUD C., L ACROIX O. & S ØGAARD A. (2017). Does syntax help discourse segmentation ? not so much. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, p. 2432–2442, Copenhagen, Denmark : Association for Computational Linguistics. doi : 10.18653/v1/D17-1258. ↗
2. C ARLSON L., M ARCU D. & O KUROVSKY M. E. (2001). Building a discourse-tagged corpus in the framework of rhetorical structure theory. In Proceedings of the Second SIGdial Workshop on Discourse and Dialogue. doi : https ://doi.org/10.3115/1118078.1118083. ↗ our card
3. D ANLOS L., C OLINET M. & S TEINLIN J. (2015). Fdtb1, première étape du projet « french discourse treebank » : repérage des connecteurs de discours en corpus. Discours, 17. doi : 10.4000/discours.9065. ↗
4. J OHANNSEN A. & S ØGAARD A. (2013). Disambiguating explicit discourse connectives without oracles. In Proceedings of the Sixth International Joint Conference on Natural Language Processing, p. 997–1001, Nagoya, Japan : Asian Federation of Natural Language Processing. find
5. L I Z., Z HAO H., P ANG C., W ANG L. & W ANG H. (2016). A constituent syntactic parse tree based discourse parser. In Proceedings of the CoNLL-16 shared task, p. 60–64, Berlin, Germany : Association for Computational Linguistics. doi : 10.18653/v1/K16-2008. ↗
6. L IN Z., N G H. T. & K AN M.- Y . (2014). A pdtb-styled end-to-end discourse parser. Natural Language Engineering, 20(2), 151–184. doi : 10.1017/S1351324912000307. ↗
7. M ANNING C. & K LEIN D. (2003). Optimization, maxent models, and conditional estimation without magic. USA : Association for Computational Linguistics. find
8. M ARCUS M. P., S ANTORINI B. & M ARCINKIEWICZ M. A. (1993). Building a large annotated corpus of English : The Penn Treebank. Computational Linguistics, 19(2), 313–330. find our card
9. N IE A., B ENNETT E. & G OODMAN N. (2019). DisSent : Learning sentence representations from explicit discourse relations. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, p. 4497–4510, Florence, Italy : Association for Computational Linguistics. doi : 10.18653/v1/P19-1442. ↗
10. N IVRE J., DE M ARNEFFE M.-C., G INTER F., G OLDBERG Y., H AJI Č J., M ANNING C. D., M C D O -NALD R., P ETROV S., P YYSALO S., S ILVEIRA N., T SARFATY R. & Z EMAN D. (2016). Universal dependencies v1 : A multilingual treebank collection. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), p. 1659–1666, Portorož, Slovenia : European Language Resources Association (ELRA). find our card
11. P EDREGOSA F., V AROQUAUX G., G RAMFORT A., M ICHEL V., T HIRION B., G RISEL O., B LON -DEL M., P RETTENHOFER P., W EISS R., D UBOURG V., V ANDERPLAS J., P ASSOS A., C OURNA -PEAU D., B RUCHER M., P ERROT M. & D UCHESNAY E. (2011). Scikit-learn : Machine learning in Python. Journal of Machine Learning Research, 12, 2825–2830. find
12. P ENG S. & Z ELDES A. (2018). All roads lead to UD : Converting Stanford and Penn parses to English universal dependencies with multilayer annotations. In Proceedings of the Joint Workshop on Linguistic Annotation, Multiword Expressions and Constructions (LAW-MWE-CxG-2018), p. 167–177, Santa Fe, New Mexico, USA : Association for Computational Linguistics. find our card
13. P ITLER E. & N ENKOVA A. (2009). Using syntax to disambiguate explicit discourse connectives in text. In Proceedings of the ACL-IJCNLP 2009 Conference Short Papers, p. 13–16, Suntec, Singapore : Association for Computational Linguistics. find our card
14. P OLAKOVA L., M ÍROVSKÝ J. & S YNKOVÁ P. (2017). Signalling implicit relations : A pdtb - rst comparison. Dialogue and Discourse, 8. doi : 10.5087/dad.2017.210. ↗
15. P RASAD R., D INESH N., L EE A., M ILTSAKAKI E., R OBALDO L., J OSHI A. & W EBBER B. (2008). The Penn discourse TreeBank 2.0. In Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC’08), Marrakech, Morocco : European Language Resources Association (ELRA). find our card
16. S ILEO D., V AN D E C RUYS T., P RADEL C. & M ULLER P. (2019). Mining discourse markers for unsupervised sentence representation learning. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics : Human Language Technologies, Volume 1 (Long and Short Papers), p. 3477–3486, Minneapolis, Minnesota : Association for Computational Linguistics. doi : 10.18653/v1/N19-1351. ↗
17. S TEPANOV E. & R ICCARDI G. (2016). UniTN end-to-end discourse parser for CoNLL 2016 shared task. In Proceedings of the CoNLL-16 shared task, p. 85–91, Berlin, Germany : Association for Computational Linguistics. doi : 10.18653/v1/K16-2012. ↗
18. W EISS G. & B AJEC M. (2018). Sense classification of shallow discourse relations with focused rnns. In PloS one. doi : https ://doi.org/10.1371/journal.pone.0206057. ↗
19. X UE N., N G H. T., P RADHAN S., P RASAD R., B RYANT C. & R UTHERFORD A. (2015). The CoNLL-2015 shared task on shallow discourse parsing. In Proceedings of the Nineteenth Conference on Computational Natural Language Learning - Shared Task, p. 1–16, Beijing, China : Association for Computational Linguistics. doi : 10.18653/v1/K15-2001. ↗
20. X UE N., N G H. T., P RADHAN S., R UTHERFORD A., W EBBER B., W ANG C. & W ANG H. (2016). CoNLL 2016 shared task on multilingual shallow discourse parsing. In Proceedings of the CoNLL-16 shared task, p. 1–19, Berlin, Germany : Association for Computational Linguistics. doi : 10.18653/v1/K16-2001. ↗ our card
21. Z HOU Y. & X UE N. (2015). The chinese discourse treebank : a chinese corpus annotated with discourse relations. Language Resources and Evaluation, 49. doi : 10.1007/s10579-014-9290-3. ↗

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/han-2020-comparing-ptb-ud-information-pdtb/