The CommitmentBank: Investigating projection in naturally occurring discourse

What it is about

The paper builds a corpus of 1,200 naturally occurring discourses, from the Wall Street Journal, the fiction part of the BNC and Switchboard dialogue. In each one, the last sentence has a clause-embedding predicate (such as "know", "think", "tell") under negation, a modal, a question or a conditional, and the question is whether the speaker is still committed to the content of the complement. The workers on Mechanical Turk (at least eight per item) answered on a scale from -3 (speaker is certain the complement is false) to +3 (certain it is true), and 982 epistemic-modal items with 45 predicates remain for the analysis. The factive/nonfactive split explains some of the variance (a coefficient of 1.60 for factivity, with a Nagelkerke R² of 0.126), but the predicate lemma explains more (R² 0.228). Subject person (0.124), tense (0.078) and genre (0.062) add little alone, and all factors together reach R² 0.35. Context-only plausibility, from 558 items, explains very little (R² 0.036). The main finding is that projection is gradient and item-specific, so the factive class is a weak predictor on its own.

Ideas

• I1. A projection score for complements in prag. For each clause complement (UD ccomp, csubj) of a predicate under negation, a question, a modal or a conditional, compute a commitment value on the paper's -3 to +3 scale from the lexical and structural features, and store it with the explanation (which feature gave what). Measure the rank correlation with teacher ratings on open dialogue and tales, by two-fold cv. The teacher is Opus and only labels; the score must come from Rust.
• I2. A predicate table with projection rates per lemma. The paper's result is that the lemma beats the factive class, so en could hold, for each clause-taking verb, a learned mean and spread, with the factive class as the default for unseen lemmas. Learn the table only from the teacher labels on open text (not from the CommitmentBank itself, because the corpus is not ours to build with), and test on held-out lemmas.
• I3. Person and tense as slot features. The paper finds that a first-person subject lowers the commitment for most predicates, and that the effect interacts with tense (Figures 4 and 5). Add subject_person and predicate_tense as slots in prag/src/feats.rs, and run the SLM with and without them. Report the ablation (remove the slot, measure the change), and keep the slot only if the drop is larger than the run-to-run spread.
• I4. A negative control from context. The paper shows that the plausibility from the preceding context hardly predicts the score. Mova can check the same thing on its own tales: a complement that the story already states should project more, and a complement that the story never mentions should not. If the context does not change the score, the world is not being used, and that is a bug to report.

For Mova

Chapters: 33. Implicature and presupposition (projection and presupposition), 27. Negation (the negated embedding), 26. Questions (complements under a question), and 31. Speech acts (commitment). Code: prag/src/feats.rs, prag/src/lens.rs, the narration frames in world/src/story.rs (a complement becomes a claim of the speaker in a frame, not a fact of the main line), en/src/parse.rs for the complement clauses. Licence note: the paper is CC BY 4.0, so short quotes with the citation are fine. The CommitmentBank data is a separate resource on GitHub, and its licence has to be checked before any use. Until then, treat it as a test set only.

Topics: Discourse analysis

Cite

The paper itself, not this card. BibTeX from the library's record of the work.

@inproceedings{marneffe2019commitmentbank,
  title = {{The CommitmentBank: Investigating projection in naturally occurring discourse}},
  author = {Marneffe, Marie-Catherine de and Simons, Mandy and Tonhauser, Judith},
  booktitle = {Proceedings of Sinn und Bedeutung 23 (2019), vol. 2, pp. 107–124},
  year = {2019},
  url = {https://ojs.ub.uni-konstanz.de/sub/index.php/sub/article/view/601}
}

References

16 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Anand, P. and V. Hacquard (2014). Factivity, belief and discourse. In The Art and Craft of Semantics: A Festschrift for Irene Heim, pp. 69–90. Cambridge, MA: MIT Working Papers in Linguistics. find
2. Beaver, D. (2010). Have you noticed that your belly button lint colour is related to the colour of your clothing? In R. Bäuerle, U. Reyle, and T. E. Zimmermann (Eds.), Presuppositions and Discourse: Essays Offered to Hans Kamp, pp. 65–99. Leiden, The Netherlands: Brill. find
3. Chierchia, G. and S. McConnell-Ginet (1990). Meaning and Grammar. Cambridge, MA: MIT Press. find
4. Christensen, R. H. B. (2018). “ordinal”: Regression models for ordinal data. R package version 2018.8-25. http://www.cran.r-project.org/package=ordinal/. de Marneffe, M.-C., B. MacCartney, and C. D. Manning (2006). Generating typed dependency parses from phrase structure parses. In Proceedings of the 5th International Conference on Language Resources and Evaluation, pp. 449–454. ↗ our card
5. Gazdar, G. (1979). A solution to the projection problem. In C.-K. Oh and D. A. Dinneen (Eds.), Syntax and Semantics, Volume 11: Presupposition, pp. 57–89. New York: Academic Press. find
6. Heim, I. (1983). On the projection problem for presuppositions. In M. Barlow, D. Flickinger, and M. Westcoat (Eds.), West Coast Conference on Formal Linguistics 2, pp. 114–125. find
7. Karttunen, L. (1971). Some observations on factivity. Papers in Linguistics 4, 55–69. find
8. Kiparsky, P. and C. Kiparsky (1970). Fact. In M. Bierwisch and K. E. Heidolph (Eds.), Progress in linguistics, pp. 143–173. The Hague, Paris: Mouton. find
9. Klein, D. and C. D. Manning (2003). Accurate unlexicalized parsing. In Proceedings of the 41st Meeting of the Association of Computational Linguistics, pp. 423–430. find our card
10. R Development Core Team (2016). R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing. find
11. Schlenker, P. (2010). Local contexts and local meanings. Philosophical Studies 151, 115–142. find
12. Simons, M. (2007). Observations on embedding verbs, evidentiality, and presupposition. Lingua 117(6), 1034–1056. find
13. Smith, E. A. S. and K. C. Hall (2014). The relationship between projection and embedding environment. In 48th Annual Meeting of the Chicago Linguistic Society, Volume 48(1), pp. 563–578. find
14. Stalnaker, R. C. (1974). Pragmatic presuppositions. In M. Munitz and P. Unger (Eds.), Semantics and Philosophy, pp. 197–213. New York: New York University Press. find
15. Tonhauser, J. (2016). Prosodic cues to presupposition projection. Semantics and Linguistic Theory 26, 934–960. find
16. Tonhauser, J., D. Beaver, and J. Degen (2018). How projective is projective content? Gradience in projectivity and at-issueness. Journal of Semantics 35, 495–542. van der Sandt, R. (1992). Presupposition projection as anaphora resolution. Journal of Semantics 9, 333–377. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/enprag-2019-marneffe-commitmentbank-investigating-projection-naturally/