Easy Victories and Uphill Battles in Coreference Resolution

What it is about

The paper builds a learning-based coreference system, the Berkeley system, that uses only a small set of homogeneous feature templates over shallow mention properties (head match, exact string match, and syntactic and discourse features) instead of many hand-built heuristics. A log-linear mention-ranking model chooses one antecedent for each mention or starts a new cluster. Mentions are extracted by simple rules (named-entity chunks, PRP pronouns, nominal NP projections), so the system has many spurious mentions that the model must reject. Evaluation uses the CoNLL 2011 data with system mentions, scored with MUC, B3 and CEAF-e and their average, the CoNLL metric. The system beats the Stanford system (the CoNLL 2011 winner) by 3.5 points and the IMS system by 1.9 points on the CoNLL metric. The authors report that the features win "easy victories" on syntax and discourse, but semantic features (shallow lexical semantics from the literature) add nothing with system mentions; they help only with gold mentions, which they explain by the number of spurious mentions that weak evidence cannot overcome. I read the abstract, the introduction and the setup; I did not read the later feature tables.

Ideas

I1. Replace hand-written mention-pair heuristics in coref/src/sieve.rs with a small set of template features over UD heads and the three mention types (proper, pronominal, nominal), trained as a perceptron, with every feature's weight written out for explanation.
I2. Measure the semantic-feature effect separately with gold and with system mentions, as the paper does: if a semantic rule helps only on gold mentions, the bottleneck is mention detection, not the rule (coref/src/mention.rs).
I3. Keep the spurious-mention count as a measure: the paper's recall-first mention extractor leaves many false mentions. Report the count of rejected mentions per document in coref/src/eval.rs, so precision work on mentions is visible.

For Mova

Code: coref/src/mention.rs (three mention types from UD PRON, PROPN, NP heads), coref/src/sieve.rs (pairwise features), coref/src/eval.rs (MUC, B3, CEAF, CoNLL average). Chapter: 35. Reference and coreference. Measure: the CoNLL average on a dev split with system mentions and with gold mentions, run twice; the gap between the two is the mention-detection share.

Topics: Coreference resolution

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/d13-1203

@inproceedings{durrett2013easy,
  title = {{Easy Victories and Uphill Battles in Coreference Resolution}},
  author = {Durrett, Greg and Klein, Dan},
  booktitle = {Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing},
  year = {2013},
  doi = {10.18653/v1/d13-1203},
  url = {https://doi.org/10.18653/v1/d13-1203}
}

References

37 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

Amit Bagga and Breck Baldwin. 1998. Algorithms for Scoring Coreference Chains. In Proceedings of the Conference on Language Resources and Evaluation Workshop on Linguistics Coreference. Avg. F 1 F 1 47.07 46.66 49.00 50.09 57.67 58.13 60.06 61.58 45.17 45.35 47.79 56.65 58.26 60.13 find our card
Mohit Bansal and Dan Klein. 2012. Coreference Semantics from Web Features. In Proceedings of the Association for Computational Linguistics. find
Eric Bengtson and Dan Roth. 2008. Understanding the Value of Features for Coreference Resolution. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. find
Shane Bergsma and Dekang Lin. 2006. Bootstrapping Path-Based Pronoun Resolution. In Proceedings of the Conference on Computational Linguistics and the Association for Computational Linguistics. find
Anders Björkelund and Richárd Farkas. 2012. Data-driven Multilingual Coreference Resolution using Resolver Stacking. In Proceedings of the Joint Conference on Empirical Methods in Natural Language Proceedings and Conference on Computational Natural Language Learning - Shared Task. find
Anders Björkelund and Pierre Nugues. 2011. Exploring Lexicalized Features for Coreference Resolution. In Proceedings of the Conference on Computational Natural Language Learning: Shared Task. find
Jie Cai and Michael Strube. 2010. Evaluation Metrics for End-to-End Coreference Resolution Systems. In Proceedings of the Special Interest Group on Discourse and Dialogue. find
Chen Chen and Vincent Ng. 2012. Combining the Best of Two Worlds: A Hybrid Approach to Multilingual Coreference Resolution. In Proceedings of the Joint Conference on Empirical Methods in Natural Language Proceedings and Conference on Computational Natural Language Learning - Shared Task. find
Pascal Denis and Jason Baldridge. 2008. Specialized Models and Ranking for Coreference Resolution. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. find
John Duchi, Elad Hazan, and Yoram Singer. 2011. Adaptive Subgradient Methods for Online Learning and Stochastic Optimization. Journal of Machine Learning Research, 12:2121–2159, July. find
Greg Durrett, David Hall, and Dan Klein. 2013. Decentralized Entity-Level Modeling for Coreference Resolution. In Proceedings of the Association for Computational Linguistics. find
Eraldo Rezende Fernandes, Cı́cero Nogueira dos Santos, and Ruy Luiz Milidiú. 2012. Latent Structure Perceptron with Feature Induction for Unrestricted Coreference Resolution. In Proceedings of the Joint Conference on Empirical Methods in Natural Language Proceedings and Conference on Computational Natural Language Learning - Shared Task. find
Kevin Gimpel and Noah A. Smith. 2010. Softmax-Margin CRFs: Training Log-Linear Models with Cost Functions. In Proceedings of the North American Chapter for the Association for Computational Linguistics. find
David Graff, Junbo Kong, Ke Chen, and Kazuaki Maeda. 2007. English Gigaword Third Edition. Linguistic Data Consortium, Catalog Number LDC2007T07. find
Barbara J. Grosz, Scott Weinstein, and Aravind K. Joshi. 1995. Centering: A Framework for Modeling the Local Coherence of Discourse. Computational Linguistics, 21(2):203–225, June. find our card
Aria Haghighi and Dan Klein. 2009. Simple Coreference Resolution with Rich Syntactic and Semantic Features. In Proceedings of Empirical Methods in Natural Language Processing. find our card
Aria Haghighi and Dan Klein. 2010. Coreference Resolution in a Modular, Entity-Centered Model. In Proceedings of the North American Chapter of the Association for Computational Linguistics. find
Iris Hendrickx and Walter Daelemans, 2007. Adding Semantic Information: Unsupervised Clusters for Coreference Resolution. find
Eduard Hovy, Mitchell Marcus, Martha Palmer, Lance Ramshaw, and Ralph Weischedel. 2006. OntoNotes: The 90% Solution. In Proceedings of the North American Chapter of the Association for Computational Linguistics: Short Papers. find
Emmanuel Lassalle and Pascal Denis. 2013. Improving Pairwise Coreference Models Through Feature Space Hierarchy Learning. In Proceedings of the Association for Computational Linguistics. find
Heeyoung Lee, Yves Peirsman, Angel Chang, Nathanael Chambers, Mihai Surdeanu, and Dan Jurafsky. 2011. Stanford’s Multi-Pass Sieve Coreference Resolution System at the CoNLL-2011 Shared Task. In Proceedings of the Conference on Computational Natural Language Learning: Shared Task. find our card
Xiaoqiang Luo, Abe Ittycheriah, Hongyan Jing, Nanda Kambhatla, and Salim Roukos. 2004. A Mention-Synchronous Coreference Resolution Algorithm Based on the Bell Tree. In Proceedings of the Association for Computational Linguistics. find
Xiaoqiang Luo. 2005. On Coreference Resolution Performance Metrics. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. find our card
Sebastian Martschat, Jie Cai, Samuel Broscheit, Éva Mújdricza-Maydt, and Michael Strube. 2012. A Multigraph Model for Coreference Resolution. In Proceedings of the Joint Conference on Empirical Methods in Natural Language Proceedings and Conference on Computational Natural Language Learning - Shared Task. find
Simone Paolo Ponzetto and Michael Strube. 2006. Exploiting Semantic Role Labeling, WordNet and Wikipedia for Coreference Resolution. In Proceedings of the North American Chapter of the Association of Computational Linguistics. find
Sameer Pradhan, Lance Ramshaw, Mitchell Marcus, Martha Palmer, Ralph Weischedel, and Nianwen Xue. 2011. CoNLL-2011 Shared Task: Modeling Unrestricted Coreference in OntoNotes. In Proceedings of the Conference on Computational Natural Language Learning: Shared Task. find our card
Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Olga Uryupina, and Yuchen Zhang. 2012. CoNLL-2012 Shared Task: Modeling Multilingual Unrestricted Coreference in OntoNotes. In Joint Conference on EMNLP and CoNLL - Shared Task. find our card
Altaf Rahman and Vincent Ng. 2009. Supervised Models for Coreference Resolution. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. find
Altaf Rahman and Vincent Ng. 2011a. Coreference Resolution with World Knowledge. In Proceedings of the Association for Computational Linguistics: Human Language Technologies. find
Altaf Rahman and Vincent Ng. 2011b. Narrowing the Modeling Gap: A Cluster-Ranking Approach to Coreference Resolution. Journal of Artificial Intelligence Research, 40(1):469–521, January. find
Marta Recasens, Matthew Can, and Daniel Jurafsky. 2013a. Same Referent, Different Words: Unsupervised Mining of Opaque Coreferent Mentions. In Proceedings of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. find
Marta Recasens, Marie-Catherine de Marneffe, and Christopher Potts. 2013b. The Life and Death of Discourse Entities: Identifying Singleton Mentions. In Proceedings of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. find
Wee Meng Soon, Hwee Tou Ng, and Daniel Chung Yong Lim. 2001. A Machine Learning Approach to Coreference Resolution of Noun Phrases. Computational Linguistics, 27(4):521–544, December. find our card
Veselin Stoyanov, Nathan Gilbert, Claire Cardie, and Ellen Riloff. 2009. Conundrums in Noun Phrase Coreference Resolution: Making Sense of the State-of-the-Art. In Proceedings of the Association for Computational Linguistics. find our card
Veselin Stoyanov, Claire Cardie, Nathan Gilbert, Ellen Riloff, David Buttler, and David Hysom. 2010. Coreference Resolution with Reconcile. In Proceedings of the Association for Computational Linguistics: Short Papers. find
Yannick Versley, Simone Paolo Ponzetto, Massimo Poesio, Vladimir Eidelman, Alan Jern, Jason Smith, Xiaofeng Yang, and Alessandro Moschitti. 2008. BART: A Modular Toolkit for Coreference Resolution. In Proceedings of the Association for Computational Linguistics: Demo Session. find
Marc Vilain, John Burger, John Aberdeen, Dennis Connolly, and Lynette Hirschman. 1995. A Model-Theoretic Coreference Scoring Scheme. In Proceedings of the Conference on Message Understanding. find our card

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/durrett-2013-easy-victories-uphill-battles-coreference/