Fine-grained Information Status Classification Using Discourse Context-Aware BERT

What it is about

Information status (IS) says how accessible a mention's referent is: old (mentioned before), new, or mediated — not mentioned but accessible through world knowledge (mostly proper names), syntax (their father, a noun linked to an old mention), coordination (aggregate), a rise/fall function value, comparison (further attacks, another), or bridging (the door after a house). On ISNotes (50 Wall Street Journal texts, 10,980 mentions: old 29.5%, mediated 33.8%, new 36.7%; bridging only 6%) the paper builds a «pseudo sentence» per mention: the mention, its own sentence, and the earlier mentions it overlaps with lexically, and fine-tunes BERT on it. Accuracy is 83.7%, 4.8 points above the earlier feature-based collective classifier, and bridging recognition improves by 10.5 F1 without hand-made semantic features. Attention analysis shows the model attends to what linguists expect: earlier same-head mentions for old, comparative premodifiers, possessives for syntactic mediated.

Ideas

• I1. Block same-head links when the mention is marked as comparative or contrastive, a cheap attack on the second main error of coref («same text or head, different entities», exact 69.6%, strict_a 54.7%): a mention with other, another, further, different, second/next (or an ordinal not matching the candidate's) is mediated/comparative and is not linked by exact, relaxed or strict_* to an earlier same-head mention. Add it to coref/src/sieve.rs with the blocker printed in coref explain; measure link precision of those sieves and CoNLL F1 on GUM dev (gold and SLM trees).
• I2. Label each mention with a rule-based IS class from what coref/src/mention.rs and the sieves already know: old = linked; syntactic = possessive or a nominal dependent pointing to an old mention; aggregate = coordination with an old member; world knowledge = proper name not linked; new = indefinite; bridging candidate = a definite common noun that is none of these. Output it in the CorefUD MISC and use the bridging candidates as the input set for a later bridging step (see roesiger-2018-bridging-resolution-task-definition-corpus). Report the distribution on GUM dev against ISNotes' proportions as a sanity check.
• I3. Use IS in the frame reader's salience (prag/src/frames.rs Ctx): a new (indefinite) mention in subject position introduces a likely topic; a definite that is a bridging candidate should not be resolved to an earlier same-head entity of a different scene. Measure the frame bench coref role.

For Mova

35. Reference and coreference §2 (givenness, accessibility) and §3.2 (bridging), 29-ellipsis-word-order-information-structure.md. Code: coref/src/sieve.rs (comparative blocker), coref/src/mention.rs and coref/src/corefud.rs (IS label), prag/src/frames.rs. Measures: sieve precision and CoNLL F1 on GUM dev; frame bench coref F1.

Topics: Discourse analysis

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.coling-main.537

@inproceedings{hou2020fine,
  title = {{Fine-grained Information Status Classification Using Discourse Context-Aware BERT}},
  author = {Hou, Yufang},
  booktitle = {Proceedings of the 28th International Conference on Computational Linguistics},
  year = {2020},
  doi = {10.18653/v1/2020.coling-main.537},
  url = {https://doi.org/10.18653/v1/2020.coling-main.537}
}

References

33 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Vaswani Ashish, Shazeer Noam, Parmar Niki, Uszkoreit Jakob, Jones Llion, Gomez Aidan N., Kaiser Lukasz, and Polosukhin Illia. 2017. Attention is all you need. In Advances in Neural Information Processing Systems 30 (NIPS 2017), pages 1–11. MIT Press. find
2. Betty J. Birner and Gregory Ward. 1998. Information Status and Noncanonical Word Order in English. John Benjamins, Amsterdam, The Netherlands. find
3. Anders Björkelund, Kerstin Eckart, Arndt Riester, Nadja Schauffler, and Katrin Schweitzer. 2014. The extended DIRNDL corpus as a resource for coreference and bridging resolution. In Proceedings of the 9th International Conference on Language Resources and Evaluation, Reykjavik, Iceland, 26–31 May 2014, pages 3222–3228. find
4. Aoife Cahill and Arndt Riester. 2012. Automatically acquiring fine-grained information status distinctions in German. In Proceedings of the SIGdial 2012 Conference: The 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue, Seoul, Korea, 5–6 July 2012, pages 232–236. find
5. Kevin Clark, Urvashi Khandelwal, Omer Levy, and Christopher D. Manning. 2019. What does BERT look at? An analysis of BERT’s attention. In Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, Florence, Italy, 28 July–2 August 2019, pages 276–286. find
6. Herbert H. Clark. 1975. Bridging. In Proceedings of the Conference on Theoretical Issues in Natural Language Processing, Cambridge, Mass., June 1975, pages 169–174. find
7. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, USA, 2–7 June 2019, pages 4171–4186. find
8. Jeanette K. Gundel, Nancy Hedberg, and Ron Zacharski. 1993. Cognitive status and the form of referring expressions in discourse. Language, 69:274–307. find
9. M. A. K. Halliday. 1967. Notes on transitivity and theme in English, Part 2. Journal of Linguistics, 3:199–244. find
10. Yufang Hou, Katja Markert, and Michael Strube. 2013a. Cascading collective classification for bridging anaphora recognition using a rich linguistic feature set. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, Seattle, Wash., 18–21 October 2013, pages 814–820. find
11. Yufang Hou, Katja Markert, and Michael Strube. 2013b. Global inference for bridging anaphora resolution. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Atlanta, Georgia, 9–14 June 2013, pages 907–917. find our card
12. Yufang Hou, Katja Markert, and Michael Strube. 2014. A rule-based system for unrestricted bridging resolution: Recognizing bridging anaphora and finding links to antecedents. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, Doha, Qatar, 25–29 October 2014, pages 2082–2093. find
13. Yufang Hou, Katja Markert, and Michael Strube. 2018. Unrestricted bridging resolution. Computational Linguistics, 44(2):237–284. find our card
14. Yufang Hou. 2016a. Incremental fine-grained information status classification using attention-based LSTMs. In Proceedings of the 26th International Conference on Computational Linguistics, Osaka, Japan, 11–16 December 2016, pages 1880–1890. find
15. Yufang Hou. 2016b. Unrestricted Bridging Resolution. Ph.D. thesis, Heidelberg University. find our card
16. Yufang Hou. 2018a. A deterministic algorithm for bridging anaphora resolution. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, 31 October– 4 November 2018, pages 1938–1948. find
17. Yufang Hou. 2018b. Enhanced word representations for bridging anaphora resolution. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, New Orleans, Louisiana, 1–6 June 2018, pages 1–7. find
18. Yufang Hou. 2020. Bridging anaphora resolution as question answering. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Seattle, Wash., 5–10 July 2020, pages 1428–1438. find our card
19. Ganesh Jawahar, Benoı̂t Sagot, and Djamé Seddah. 2019. What does BERT learn about the structure of language? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, 28 July–2 August 2019, pages 3651–3657. find
20. Mandar Joshi, Omer Levy, Luke Zettlemoyer, and Daniel Weld. 2019. BERT for coreference resolution: Baselines and analysis. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China, 3 – 7 November 2018, pages 5803–5808. find
21. Olga Kovaleva, Alexey Romanov, Anna Rogers, and Anna Rumshisky. 2019. Revealing the dark secrets of BERT. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China, 3 – 7 November 2018, pages 4365–4374. find
22. Ivana Kruijff-Korbayová and Mark Steedman. 2003. Discourse and information structure. Journal of Logic, Language and Information. Special Issue on Discource and Information Structure, 12(3):149–259. find
23. Knud Lambrecht. 1994. Information Structure and Sentence Form. Cambridge, U.K.: Cambridge University Press. find
24. Heeyoung Lee, Yves Peirsman, Angel Chang, Nathanael Chambers, Mihai Surdeanu, and Dan Jurafsky. 2011. Stanford’s multi-pass sieve coreference resolution system at the CoNLL-2011 shared task. In Proceedings of the Shared Task of the 15th Conference on Computational Natural Language Learning, Portland, Oreg., 23–24 June 2011, pages 28–34. find our card
25. Katja Markert, Yufang Hou, and Michael Strube. 2012. Collective classification for fine-grained information status. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics, Jeju Island, Korea, 8–14 July 2012, pages 795–804. find
26. Malvina Nissim, Shipara Dingare, Jean Carletta, and Mark Steedman. 2004. An annotation scheme for information status in dialogue. In Proceedings of the 4th International Conference on Language Resources and Evaluation, Lisbon, Portugal, 26–28 May 2004, pages 1023–1026. find
27. Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, New Orleans, Louisiana, 1–6 June 2018, pages 2227–2237. find
28. Ellen F. Prince. 1981. Towards a taxonomy of given-new information. In P. Cole, editor, Radical Pragmatics, pages 223–255. Academic Press, New York, N.Y. find
29. Ellen F. Prince. 1992. The ZPG letter: Subjects, definiteness, and information-status. In W.C. Mann and S.A. Thompson, editors, Discourse Description. Diverse Linguistic Analyses of a Fund-Raising Text, pages 295–325. John Benjamins, Amsterdam. find
30. Altaf Rahman and Vincent Ng. 2012. Learning the fine-grained information status of discourse entities. In Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics, Avignon, France, 23–27 April 2012, pages 798–807. find
31. Arndt Riester, David Lorenz, and Nina Seemann. 2010. A recursive annotation scheme for referential information status. In Proceedings of the 7th International Conference on Language Resources and Evaluation, La Valetta, Malta, 17–23 May 2010, pages 717–722. find
32. Ina Rösiger. 2019. Computational modelling of coreference and bridging resolution. Ph.D. thesis, University of Stuttgart. find
33. Ralph Weischedel, Martha Palmer, Mitchell Marcus, Eduard Hovy, Sameer Pradhan, Lance Ramshaw, Nianwen Xue, Ann Taylor, Jeff Kaufman, Michelle Franchini, Mohammed El-Bachouti, Robert Belvin, and Ann Houston. 2011. OntoNotes release 4.0. LDC2011T03, Philadelphia, Penn.: Linguistic Data Consortium. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/hou-2020-fine-grained-information-status-classification/