The Penn Discourse Treebank

What it is about

The paper introduces the PDTB project. It adds a discourse layer to the Penn Treebank and Propbank: discourse connectives, their two arguments (Arg1 and Arg2), and, for implicit relations, an inserted connective. Connectives are of four kinds: subordinating and coordinating conjunctions, adverbials, and implicit ones. The paper reports a pilot. Two annotators marked 2,717 tokens of 10 explicit connectives and 386 implicit tokens. For the argument spans of explicit connectives, exact-match agreement is 90.2% (4,900 of 5,434 argument tokens). It is 92.4% for subordinating conjunctions and 71.8% for adverbials. Of the disagreements, 79% are partial overlaps, mostly over a governing verb, a dependent clause or a parenthetical. For implicit connectives, argument agreement is 85.1% (657 of 772 tokens), and agreement on the class of the inserted connective is 72% (307 tokens). The authors do not use kappa, because the spans do not fall into fixed categories.

Ideas

• I1. Report two agreement numbers for any Mova annotation: exact span and partial overlap. The paper's partial-match figure (94.5%) shows how much the choice of measure changes the result.
• I2. Store each argument as a minimal span and a separate supplementary span. A rule can then use the minimal span for sense and the full span for retrieval. The same split fits the frames in prag/src/frames.rs.
• I3. Use five coarse classes for inserted connectives (additive, cause-effect, temporal, contrast, restatement), as the paper does, and measure agreement on the same kind of sample.
• I4. Report the adverbial and subordinating classes apart. The gap in the paper (92.4% vs 71.8%) shows where a rule-based argument finder is likely to fail, and the same split can be measured on Mova's output.

For Mova

The chapters are 36. Discourse relations and coherence and 38. Dialogue, discourse markers and reported speech. The code is prag/src/frames.rs. The measure is a two-annotator agreement check on a sample of Mova's own lesson annotations, with the two span measures and the per-class split above. Run it twice. The paper is a 2004 pilot with small counts, so the numbers are a guide, not a target.

Topics: Discourse analysis · Treebanks

Cite

The paper itself, not this card. BibTeX from the library's record of the work.

@inproceedings{miltsakaki2004penn,
  title = {{The Penn Discourse Treebank}},
  author = {Miltsakaki, Eleni and Prasad, Rashmi and Joshi, Aravind and Webber, Bonnie},
  booktitle = {Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC'04)},
  year = {2004},
  url = {https://aclanthology.org/L04-1386/}
}

References

11 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Asher, Nicholas, 1993. Reference to Abstract Objects in Discourse. Kluwer Academic Publishers. find
2. Carlson, Lynn, Daniel Marcu, and Mary Ellen Okurowski, 2003. Current Directions in Discourse and Dialogue, chapter Building a Discourse-Tagged Corpus in the Framework of Rhetorical Structure Theory. Kluwer Academic Publishers. find our card
3. Gaizauskas, Rob, Patrick Hanks, James Pustejovsky, Roser Sauri, Andrew See, Andrea Setzer, Lisa Ferro, and Beth Sundheim., 2003. The timebank corpus. In Corpus Linguistics 2003.. Lancaster, U.K. find
4. Kingsbury, Paul and Martha Palmer, 2002. From Treebank to Propbank. In Third International Conference on Language Resources and Evaluation, LREC-02, Las Palmas, Canary Islands, Spain. find our card
5. Knott, Alistair, 1996. A Data-Driven Methodology for Motivating a Set of Coherence Relations. Ph.D. thesis, University of Edinburgh. find
6. Marcus, Mitch, Beatrice Santorini, and Mary Ann Marcinkiewicz, 1993. Building a large annotated corpus of english: the Penn Treebank. Computational Linguistics, 19:313–330. find our card
7. Miltsakaki, Eleni, Rashmi Prasad, Aravind Joshi, and Bonnie Webber, 2004. Annotating discourse connectives and their arguments. In Proceedings of the Workshop on Frontiers in Corpus Annotation. Boston, Massachussetts. find
8. Mitkov, Ruslan, R. Evans, C. Orasan, C. Barbu, L. Jones, and V. Sotirova, 2000. Coreference and anaphora: developing annotating tools, annotated resources and annotation strategies. In Proceedings of the Discourse Anaphora and Anaphora Resolution Colloquium (DAARC 2000), Lancaster, U.K.. find
9. Muller, Christoph, Stefan Rapp, and Michael Strube, 2002. Applying co-training to reference resolution. In Proceedings of the Annual Meeting of the Association for Computational Linguistics, Philadelpha PA. find
10. Siegel, Sidney and N. J. Castellan, 1988. Nonparamateric Statistics for the Behavioral Sciences. McGraw-Hill, 2nd edition. find
11. Webber, Bonnie, Matthew Stone, Aravind Joshi, and Alistair Knott, 2003. Anaphora and discourse structure. Computational Linguistics, 29:545–587. find our card

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/miltsakaki-2004-penn-discourse-treebank/