Mova0.5 pre-release⌕
☰
home / docs / papers / English / CoNLL-2011 Shared Task: Modeling Unrestricted Coreference in OntoNotes

CoNLL-2011 Shared Task: Modeling Unrestricted Coreference in OntoNotes

What it is about

The paper describes the CoNLL-2011 shared task on general coreference in OntoNotes. OntoNotes links identical reference (IDENT) for pronouns, names and definite NPs, and for events, not only NPs or a fixed set of entity types. Appositives are tagged separately as attributes. Singletons are not annotated, so mention detection and coreference cannot be scored apart. The task had two tracks: closed (the provided layers, plus WordNet and a fixed gender and number table) and open (any external resource). Scores were given under three conditions: predicted mentions, predicted with gold boundaries, and gold mentions. The official score is the arithmetic mean of the F-scores of MUC, B-cubed and CEAF-e. Eighteen closed-track and five open-track systems were scored, and the paper discusses their approaches and results. Appositive heads follow a specificity scale: proper name, pronoun, definite NP, indefinite specific NP, non-specific NP. On a tie, the left member is the head.

Ideas

For Mova

The code is coref/src/eval.rs (the three metrics and their mean), coref/src/sieve.rs (the appositive rule), coref/src/mention.rs and coref/src/corefud.rs. The chapter is 35. Reference and coreference. The measure is the official-style mean on the Mova dev split, run twice with identical outputs.

Topics: Coreference resolution

Cite

The paper itself, not this card. BibTeX from the library's record of the work.

@inproceedings{pradhan2011conll,
  title = {{CoNLL-2011 Shared Task: Modeling Unrestricted Coreference in OntoNotes}},
  author = {Pradhan, Sameer and Ramshaw, Lance and Marcus, Mitchell and Palmer, Martha and Weischedel, Ralph and Xue, Nianwen},
  booktitle = {Proceedings of the Fifteenth Conference on Computational Natural Language Learning: Shared Task},
  year = {2011},
  url = {https://aclanthology.org/W11-1901/}
}

References

49 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Olga Babko-Malaya, Ann Bies, Ann Taylor, Szuting Yi, Martha Palmer, Mitch Marcus, Seth Kulick, and Libin Shen. 2006. Issues in synchronizing the English treebank and propbank. In Workshop on Frontiers in Linguistically Annotated Corpora 2006, July. find
  2. Amit Bagga and Breck Baldwin. 1998. Algorithms for Scoring Coreference Chains. In The First International Conference on Language Resources and Evaluation Workshop on Linguistics Coreference, pages 563–566. find our card
  3. Shane Bergsma and Dekang Lin. 2006. Bootstrapping path-based pronoun resolution. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, pages 33–40, Sydney, Australia, July. find
  4. Jie Cai and Michael Strube. 2010. Evaluation metrics for end-to-end coreference resolution systems. In Proceedings of the 11th Annual Meeting of the Special Interest Group on Discourse and Dialogue, SIGDIAL ’10, pages 28–36. find
  5. Eugene Charniak and Mark Johnson. 2001. Edit detection and parsing for transcribed speech. In Proceedings of the Second Meeting of North American Chapter of the Association of Computational Linguistics, June. find
  6. Eugene Charniak and Mark Johnson. 2005. Coarse-to-fine n-best parsing and maxent discriminative reranking. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL), Ann Arbor, MI, June. find our card
  7. Nancy Chinchor and Beth Sundheim. 2003. Message understanding conference (MUC) 6. In LDC2003T13. find
  8. Nancy Chinchor. 2001. Message understanding conference (MUC) 7. In LDC2001T02. find
  9. Aron Culotta, Michael Wick, Robert Hall, and Andrew McCallum. 2007. First-order probabilistic models for coreference resolution. In HLT/NAACL, pages 81–88. find
  10. Pascal Denis and Jason Baldridge. 2007. Joint determination of anaphoricity and coreference resolution using integer programming. In Proceedings of HLT/NAACL. find
  11. Pascal Denis and Jason Baldridge. 2009. Global joint models for coreference resolution and named entity classification. Procesamiento del Lenguaje Natural, (42):87–96. find
  12. Charles Fillmore, Christopher Johnson, and Miriam R. L. Petruck. 2003. Background to framenet. International Journal of Lexicography, 16(3). find
  13. G. G. Doddington, A. Mitchell, M. Przybocki, L. Ramshaw, S. Strassell, and R. Weischedel. 2000. The automatic content extraction (ACE) program-tasks, data, and evaluation. In Proceedings of LREC. find
  14. Aria Haghighi and Dan Klein. 2010. Coreference resolution in a modular, entity-centered model. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 385–393, Los Angeles, California, June. find
  15. Jan Hajič, Massimiliano Ciaramita, Richard Johansson, Daisuke Kawahara, Maria Antònia Martı́, Lluı́s Màrquez, Adam Meyers, Joakim Nivre, Sebastian Padó, Jan Štěpánek, Pavel Straňák, Mihai Surdeanu, Nianwen Xue, and Yi Zhang. 2009. The CoNLL-2009 shared task: Syntactic and semantic dependencies in multiple languages. In Proceedings of the Thirteenth Conference on Computational Natural Language Learning (CoNLL 2009): Shared Task, pages 1–18, Boulder, Colorado, June. find our card
  16. Sanda M. Harabagiu, Razvan C. Bunescu, and Steven J. Maiorano. 2001. Text and knowledge mining for coreference resolution. In NAACL. find
  17. L. Hirschman and N. Chinchor. 1997. Coreference task definition (v3.0, 13 jul 97). In Proceedings of the Seventh Message Understanding Conference. find
  18. Eduard Hovy, Mitchell Marcus, Martha Palmer, Lance Ramshaw, and Ralph Weischedel. 2006. OntoNotes: The 90% solution. In Proceedings of HLT/NAACL, pages 57–60, New York City, USA, June. Association for Computational Linguistics. find
  19. Karin Kipper, Anna Korhonen, Neville Ryant, and Martha Palmer. 2000. A large-scale classification of english verbs. Language Resources and Evaluation, 42(1):21 – 40. find
  20. Xiaoqiang Luo. 2005. On coreference resolution performance metrics. In Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing, pages 25– 32, Vancouver, British Columbia, Canada, October. find our card
  21. Mitchell P. Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of English: The Penn treebank. Computational Linguistics, 19(2):313–330, June. find our card
  22. Andrew McCallum and Ben Wellner. 2004. Conditional models of identity uncertainty with application to noun coreference. In Advances in Neural Information Processing Systems (NIPS). find
  23. Joseph McCarthy and Wendy Lehnert. 1995. Using decision trees for coreference resolution. In Proceedings of the Fourteenth International Conference on Artificial Intelligence, pages 1050–1055. find
  24. Thomas S. Morton. 2000. Coreference for nlp applications. In Proceedings of the 38th Annual Meeting of the Association for Computational Linguistics, October. find
  25. Vincent Ng. 2007. Shallow semantics for coreference resolution. In Proceedings of the IJCAI. find
  26. Vincent Ng. 2010. Supervised noun phrase coreference research: The first fifteen years. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 1396–1411, Uppsala, Sweden, July. find our card
  27. Martha Palmer, Daniel Gildea, and Paul Kingsbury. 2005. The Proposition Bank: An annotated corpus of semantic roles. Computational Linguistics, 31(1):71– 106. ↗ our card
  28. Martha Palmer, Hoa Trang Dang, and Christiane Fellbaum. 2007. Making fine-grained and coarse-grained sense distinctions, both manually and automatically. find
  29. R. Passonneau. 2004. Computing reliability for coreference annotation. In Proceedings of LREC. find
  30. Massimo Poesio and Ron Artstein. 2005. The reliability of anaphoric annotation, reconsidered: Taking ambiguity into account. In Proceedings of the Workshop on Frontiers in Corpus Annotations II: Pie in the Sky. find
  31. Massimo Poesio. 2004. The mate/gnome scheme for anaphoric annotation, revisited. In Proceedings of SIGDIAL. find
  32. Simone Paolo Ponzetto and Massimo Poesio. 2009. State-of-the-art nlp approaches to coreference resolution: Theory and practical recipes. In Tutorial Abstracts of ACL-IJCNLP 2009, page 6, Suntec, Singapore, August. find
  33. Simone Paolo Ponzetto and Michael Strube. 2005. Semantic role labeling for coreference resolution. In Companion Volume of the Proceedings of the 11th Meeting of the European Chapter of the Association for Computational Linguistics, pages 143–146, Trento, Italy, April. find
  34. Simone Paolo Ponzetto and Michael Strube. 2006. Exploiting semantic role labeling, WordNet and Wikipedia for coreference resolution. In Proceedings of the HLT/NAACL, pages 192–199, New York City, N.Y., June. find
  35. Sameer Pradhan, Kadri Hacioglu, Valerie Krugler, Wayne Ward, James Martin, and Dan Jurafsky. 2005. Support vector learning for semantic argument classification. Machine Learning Journal, 60(1):11–39. find
  36. Sameer Pradhan, Eduard Hovy, Mitchell Marcus, Martha Palmer, Lance Ramshaw, and Ralph Weischedel. 2007a. OntoNotes: A Unified Relational Semantic Representation. International Journal of Semantic Computing, 1(4):405–419. find
  37. Sameer Pradhan, Lance Ramshaw, Ralph Weischedel, Jessica MacBride, and Linnea Micciulla. 2007b. Unrestricted Coreference: Indentifying Entities and Events in OntoNotes. In in Proceedings of the IEEE International Conference on Semantic Computing (ICSC), September 17-19. find
  38. Altaf Rahman and Vincent Ng. 2009. Supervised models for coreference resolution. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 968–977, Singapore, August. Association for Computational Linguistics. find
  39. W. M. Rand. 1971. Objective criteria for the evaluation of clustering methods. Journal of the American Statistical Association, 66(336). find
  40. Marta Recasens and Eduard Hovy. 2011. Blanc: Implementing the rand index for coreference evaluation. Natural Language Engineering. find
  41. Marta Recasens, Lluı́s Màrquez, Emili Sapena, M. Antònia Martı́, Mariona Taulé, Véronique Hoste, Massimo Poesio, and Yannick Versley. 2010. Semeval-2010 task 1: Coreference resolution in multiple languages. In Proceedings of the 5th International Workshop on Semantic Evaluation, pages 1–8, Uppsala, Sweden, July. find
  42. W. Soon, H. Ng, and D. Lim. 2001. A machine learning approach to coreference resolution of noun phrase. Computational Linguistics, 27(4):521–544. find
  43. Veselin Stoyanov, Nathan Gilbert, Claire Cardie, and Ellen Riloff. 2009. Conundrums in noun phrase coreference resolution: Making sense of the state-of-the-art. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, pages 656–664, Suntec, Singapore, August. Association for Computational Linguistics. find our card
  44. Mihai Surdeanu, Richard Johansson, Adam Meyers, Lluı́s Màrquez, and Joakim Nivre. 2008. find
  45. The CoNLL 2008 shared task on joint parsing of syntactic and semantic dependencies. In CoNLL 2008: Proceedings of the Twelfth Conference on Computational Natural Language Learning, pages 159–177, Manchester, England, August. find our card
  46. Yannick Versley. 2007. Antecedent selection techniques for high-recall coreference resolution. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL). find
  47. M. Vilain, J. Burger, J. Aberdeen, D. Connolly, and L. Hirschman. 1995. A model theoretic coreference scoring scheme. In Proceedings of the Sixth Message Undersatnding Conference (MUC-6), pages 45–52. find our card
  48. Ralph Weischedel and Ada Brunstein. 2005. BBN pronoun coreference and entity type corpus LDC catalog no.: LDC2005T33. BBN Technologies. find
  49. Ralph Weischedel, Eduard Hovy, Martha Palmer, Mitch Marcus, Robert Belvin, Sameer Pradhan, Lance Ramshaw, and Nianwen Xue. 2011. OntoNotes: A Large Training Corpus for Enhanced Processing. In Joseph Olive, Caitlin Christianson, and John McCary, editors, Handbook of Natural Language Processing and Machine Translation. Springer. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/pradhan-2011-conll-2011-shared-task-modeling/