Mova0.5 pre-release⌕
☰
home / docs / papers / English / CoNLL-2012 Shared Task: Modeling Multilingual Unrestricted Coreference in OntoNotes

CoNLL-2012 Shared Task: Modeling Multilingual Unrestricted Coreference in OntoNotes

What it is about

The paper describes the CoNLL-2012 shared task: coreference in English, Chinese and Arabic on OntoNotes 5.0. The corpus covers general anaphoric coreference (not only noun phrases, and not limited to a set of entity types), and it comes with integrated layers of parses, semantic roles, word senses and named entities. The task has a closed track (only the provided data, WordNet allowed for English, and a fixed table of number and gender predictions from Bergsma and Lin) and an open track (external resources allowed). Each track has ablation settings: gold mention boundaries, gold mentions, and gold parses, with the main task using no gold mention information. The paper states the format, the preprocessing and the evaluation criteria, and reports the participating systems in sections 5 to 8. I read the abstract, the introduction and the track definitions. I did not read the score tables, so the card gives no shared-task numbers.

Ideas

For Mova

Code: coref/src/eval.rs (settings and tracks), coref/src/mention.rs (mention boundaries from UD), en/src/dict.rs (the lexicon, for any resource that would count as open track), coref/src/corefud.rs (CoNLL-format input). Chapter: 35. Reference and coreference. Measure: the main score and the three gold-setting scores on one corpus, run twice, to find which layer costs most.

Topics: Coreference resolution

Cite

The paper itself, not this card. BibTeX from the library's record of the work.

@inproceedings{pradhan2012conll,
  title = {{CoNLL-2012 Shared Task: Modeling Multilingual Unrestricted Coreference in OntoNotes}},
  author = {Pradhan, Sameer and Moschitti, Alessandro and Xue, Nianwen and Uryupina, Olga and Zhang, Yuchen},
  booktitle = {Joint Conference on EMNLP and CoNLL - Shared Task},
  year = {2012},
  url = {https://aclanthology.org/W12-4501/}
}

References

70 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Olga Babko-Malaya, Ann Bies, Ann Taylor, Szuting Yi, Martha Palmer, Mitch Marcus, Seth Kulick, and Libin Shen. 2006. Issues in synchronizing the English treebank and propbank. In Workshop on Frontiers in Linguistically Annotated Corpora 2006, July. find
  2. Amit Bagga and Breck Baldwin. 1998. Algorithms for Scoring Coreference Chains. In The First International Conference on Language Resources and Evaluation Workshop on Linguistics Coreference, pages 563–566. find our card
  3. Elizabeth Baran and Nianwen Xue. 2011. Singular or Plural? Exploiting Parallel Corpora for Chinese Number Prediction. In Proceedings of Machine Translation Summit XIII. find
  4. Shane Bergsma and Dekang Lin. 2006. Bootstrapping path-based pronoun resolution. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, pages 33–40, Sydney, Australia, July. find
  5. Jie Cai and Michael Strube. 2010. Evaluation metrics for end-to-end coreference resolution systems. In Proceedings of the 11th Annual Meeting of the Special Interest Group on Discourse and Dialogue, SIGDIAL ’10, pages 28–36. find
  6. Jie Cai, Eva Mujdricza-Maydt, and Michael Strube. 2011a. Unrestricted coreference resolution via global hypergraph partitioning. In Proceedings of the Fifteenth Conference on Computational Natural Language Learning: Shared Task, pages 56–60, Portland, Oregon, USA, June. Association for Computational Linguistics. find
  7. Shu Cai, David Chiang, and Yoav Goldberg. 2011b. Language-independent parsing with empty elements. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 212–216, Portland, Oregon, USA, June. Association for Computational Linguistics. find
  8. Wendy W Chapman, Prakash M Nadkarni, Lynette Hirschman, Leonard W D’Avolio, Guergana K Savova, and Ozlem Uzuner. 2011. Overcoming barriers to NLP for clinical text: the role of shared tasks and the need for additional creative solutions. Journal of American Medical Informatics Association, 18(5), September. find
  9. Eugene Charniak and Mark Johnson. 2001. Edit detection and parsing for transcribed speech. In Proceedings of the Second Meeting of North American Chapter of the Association of Computational Linguistics, June. find
  10. Eugene Charniak and Mark Johnson. 2005. Coarse-to-fine n-best parsing and maxent discriminative reranking. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL), Ann Arbor, MI, June. find our card
  11. Nancy Chinchor and Beth Sundheim. 2003. Message understanding conference (MUC) 6. In LDC2003T13. find
  12. Nancy Chinchor. 2001. Message understanding conference (MUC) 7. In LDC2001T02. find
  13. Aron Culotta, Michael Wick, Robert Hall, and Andrew McCallum. 2007. First-order probabilistic models for coreference resolution. In HLT/NAACL, pages 81–88. find
  14. Pascal Denis and Jason Baldridge. 2007. Joint determination of anaphoricity and coreference resolution using integer programming. In Proceedings of HLT/NAACL. find
  15. Pascal Denis and Jason Baldridge. 2009. Global joint models for coreference resolution and named entity classification. Procesamiento del Lenguaje Natural, (42):87–96. find
  16. George Doddington, Alexis Mitchell, Mark Przybocki, Lance Ramshaw, Stephanie Strassel, and Ralph Weischedel. 2004. The automatic content extraction (ACE) program-tasks, data, and evaluation. In Proceedings of LREC. find
  17. Charles Fillmore, Christopher Johnson, and Miriam R. L. Petruck. 2003. Background to FrameNet. International Journal of Lexicography, 16(3). find
  18. Ryan Gabbard. 2010. Null Element Restoration. Ph.D. thesis, University of Pennsylvania. find
  19. Aria Haghighi and Dan Klein. 2010. Coreference resolution in a modular, entity-centered model. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 385–393, Los Angeles, California, June. find
  20. Jan Hajič, Massimiliano Ciaramita, Richard Johansson, Daisuke Kawahara, Maria Antònia Martı́, Lluı́s Màrquez, Adam Meyers, Joakim Nivre, Sebastian Padó, Jan Štěpánek, Pavel Straňák, Mihai Surdeanu, Nianwen Xue, and Yi Zhang. 2009. The CoNLL-2009 shared task: Syntactic and semantic dependencies in multiple languages. In Proceedings of the Thirteenth Conference on Computational Natural Language Learning (CoNLL 2009): Shared Task, pages 1–18, Boulder, Colorado, June. find our card
  21. Sanda M. Harabagiu, Razvan C. Bunescu, and Steven J. Maiorano. 2001. Text and knowledge mining for coreference resolution. In NAACL. find
  22. Lynette Hirschman and Nancy Chinchor. 1997. Coreference task definition (v3.0, 13 jul 97). In Proceedings of the Seventh Message Understanding Conference. find
  23. Lynette Hirschman, Patricia Robinson, John Burger, and Marc Vilain. 1998. Automating coreference: The role of annotated training data. In Proceedings of AAAI Spring Symposium on Applying Machine Learning to Discourse Processing. find
  24. Eduard Hovy, Mitchell Marcus, Martha Palmer, Lance Ramshaw, and Ralph Weischedel. 2006. OntoNotes: The 90% solution. In Proceedings of HLT/NAACL, pages 57–60, New York City, USA, June. Association for Computational Linguistics. find
  25. Karin Kipper, Anna Korhonen, Neville Ryant, and Martha Palmer. 2000. A large-scale classification of english verbs. Language Resources and Evaluation, 42(1):21 – 40. find
  26. Heeyoung Lee, Yves Peirsman, Angel Chang, Nathanael Chambers, Mihai Surdeanu, and Dan Jurafsky. 2011. Stanford’s multi-pass sieve coreference resolution system at the conll-2011 shared task. In Proceedings of the Fifteenth Conference on Computational Natural Language Learning: Shared Task, pages 28–34, Portland, Oregon, USA, June. Association for Computational Linguistics. find our card
  27. Xiaoqiang Luo. 2005. On coreference resolution performance metrics. In Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing, pages 25– 32, Vancouver, British Columbia, Canada, October. find our card
  28. Mohamed Maamouri and Ann Bies. 2004. Developing an arabic treebank: Methods, guidelines, procedures, and tools. In Ali Farghaly and Karine Megerdoomian, editors, COLING 2004 Computational Approaches to Arabic Script-based Languages, pages 2–9, Geneva, Switzerland, August 28th. COLING. find our card
  29. Mitchell P. Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of English: The Penn treebank. Computational Linguistics, 19(2):313–330, June. find our card
  30. Andrew McCallum and Ben Wellner. 2004. Conditional models of identity uncertainty with application to noun coreference. In Advances in Neural Information Processing Systems (NIPS). find
  31. Joseph McCarthy and Wendy Lehnert. 1995. Using decision trees for coreference resolution. In Proceedings of the Fourteenth International Conference on Artificial Intelligence, pages 1050–1055. find
  32. Thomas S. Morton. 2000. Coreference for nlp applications. In Proceedings of the 38th Annual Meeting of the Association for Computational Linguistics, October. find
  33. Eugene W. Myers. 1986. An O(ND) difference algorithm and its variations. Algorithmica, 1(2):251—-266. find
  34. Vincent Ng. 2007. Shallow semantics for coreference resolution. In Proceedings of the IJCAI. find
  35. Vincent Ng. 2010. Supervised noun phrase coreference research: The first fifteen years. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 1396–1411, Uppsala, Sweden, July. find our card
  36. David S. Pallett. 2002. The role of the National Institute of Standards and Technology in DARPA’s Broadcast News continuous speech recognition research program. Speech Communication, 37(1-2), May. find
  37. Martha Palmer, Daniel Gildea, and Paul Kingsbury. 2005. The Proposition Bank: An annotated corpus of semantic roles. Computational Linguistics, 31(1):71– 106. ↗ our card
  38. Martha Palmer, Hoa Trang Dang, and Christiane Fellbaum. 2007. Making fine-grained and coarse-grained sense distinctions, both manually and automatically. find
  39. Martha Palmer, Olga Babko-Malaya, Ann Bies, Mona Diab, Mohammed Maamouri, Aous Mansouri, and Wajdi Zaghouani. 2008. A pilot arabic propbank. In Proceedings of the International Conference on Language Resources and Evaluation (LREC), Marrakech, Morocco, May 28-30. find
  40. Rebecca Passonneau. 2004. Computing reliability for coreference annotation. In Proceedings of LREC. find
  41. Slav Petrov and Dan Klein. 2007. Improved Inferencing for Unlexicalized Parsing. In Proc of HLT-NAACL. find
  42. Massimo Poesio and Ron Artstein. 2005. The reliability of anaphoric annotation, reconsidered: Taking ambiguity into account. In Proceedings of the Workshop on Frontiers in Corpus Annotations II: Pie in the Sky. find
  43. Massimo Poesio. 2004. The mate/gnome scheme for anaphoric annotation, revisited. In Proceedings of SIGDIAL. find
  44. Simone Paolo Ponzetto and Massimo Poesio. 2009. State-of-the-art nlp approaches to coreference resolution: Theory and practical recipes. In Tutorial Abstracts of ACL-IJCNLP 2009, page 6, Suntec, Singapore, August. find
  45. Simone Paolo Ponzetto and Michael Strube. 2005. Semantic role labeling for coreference resolution. In Companion Volume of the Proceedings of the 11th Meeting of the European Chapter of the Association for Computational Linguistics, pages 143–146, Trento, Italy, April. find
  46. Simone Paolo Ponzetto and Michael Strube. 2006. Exploiting semantic role labeling, WordNet and Wikipedia for coreference resolution. In Proceedings of the HLT/NAACL, pages 192–199, New York City, N.Y., June. find
  47. Sameer Pradhan, Kadri Hacioglu, Valerie Krugler, Wayne Ward, James Martin, and Dan Jurafsky. 2005. Support vector learning for semantic argument classification. Machine Learning Journal, 60(1):11–39. find
  48. Sameer Pradhan, Eduard Hovy, Mitchell Marcus, Martha Palmer, Lance Ramshaw, and Ralph Weischedel. 2007a. OntoNotes: A Unified Relational Semantic Representation. International Journal of Semantic Computing, 1(4):405–419. find
  49. Sameer Pradhan, Lance Ramshaw, Ralph Weischedel, Jessica MacBride, and Linnea Micciulla. 2007b. Unrestricted Coreference: Indentifying Entities and Events in OntoNotes. In in Proceedings of the IEEE International Conference on Semantic Computing (ICSC), September 17-19. find
  50. Karthik Raghunathan, Heeyoung Lee, Sudarshan Rangarajan, Nate Chambers, Mihai Surdeanu, Dan Jurafsky, and Christopher Manning. 2010. A multi-pass sieve for coreference resolution. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 492–501, Cambridge, MA, October. Association for Computational Linguistics. find our card
  51. Altaf Rahman and Vincent Ng. 2009. Supervised models for coreference resolution. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 968–977, Singapore, August. Association for Computational Linguistics. find
  52. William M. Rand. 1971. Objective criteria for the evaluation of clustering methods. Journal of the American Statistical Association, 66(336). find
  53. Marta Recasens and Eduard Hovy. 2011. Blanc: Implementing the rand index for coreference evaluation. Natural Language Engineering. find
  54. Marta Recasens, Lluı́s Màrquez, Emili Sapena, M. Antònia Martı́, Mariona Taulé, Véronique Hoste, Massimo Poesio, and Yannick Versley. 2010. Semeval-2010 task 1: Coreference resolution in multiple languages. In Proceedings of the 5th International Workshop on Semantic Evaluation, pages 1–8, Uppsala, Sweden, July. find
  55. Wee Meng Soon, Hwee Tou Ng, and Daniel Chung Yong Lim. 2001. A machine learning approach to coreference resolution of noun phrase. Computational Linguistics, 27(4):521–544. find
  56. Veselin Stoyanov, Nathan Gilbert, Claire Cardie, and Ellen Riloff. 2009. Conundrums in noun phrase coreference resolution: Making sense of the state-of-the-art. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, pages 656–664, Suntec, Singapore, August. Association for Computational Linguistics. find our card
  57. Mihai Surdeanu, Richard Johansson, Adam Meyers, Lluı́s Màrquez, and Joakim Nivre. 2008. find
  58. The CoNLL 2008 shared task on joint parsing of syntactic and semantic dependencies. In CoNLL 2008: Proceedings of the Twelfth Conference on Computational Natural Language Learning, pages 159–177, Manchester, England, August. find our card
  59. Ozlem Uzuner, Andreea Bodnari, Shuying Shen, Tyler Forbush, John Pestian, and Brett R South. 2012. Evaluating the state of the art in coreference resolution for electronic medical records. Journal of American Medical Informatics Association, 19(5), September. find
  60. Yannick Versley, Simone Paolo Ponzetto, Massimo Poesio, Vladimir Eidelman, Alan Jern, Jason Smith, Xiaofeng Yang, and Alessandro Moschitti. 2008. BART: A modular toolkit for coreference resolution. In Proceedings of the 6th International Conference on Language Resources and Evaluation, Marrakech, Morocco, May. find
  61. Yannick Versley. 2007. Antecedent selection techniques for high-recall coreference resolution. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL). find
  62. Marc Vilain, John Burger, John Aberdeen, Dennis Connolly, and Lynette Hirschman. 1995. A model theoretic coreference scoring scheme. In Proceedings of the Sixth Message Undersatnding Conference (MUC-6), pages 45–52. find our card
  63. Ralph Weischedel and Ada Brunstein. 2005. BBN pronoun coreference and entity type corpus LDC catalog no.: LDC2005T33. BBN Technologies. find
  64. Ralph Weischedel, Eduard Hovy, Mitchell Marcus, Martha Palmer, Robert Belvin, Sameer Pradhan, Lance Ramshaw, and Nianwen Xue. 2011. OntoNotes: A large training corpus for enhanced processing. In Joseph Olive, Caitlin Christianson, and John McCary, editors, Handbook of Natural Language Processing and Machine Translation: DARPA Global Autonomous Language Exploitation. Springer. find
  65. Nianwen Xue and Martha Palmer. 2009. Adding semantic roles to the Chinese Treebank. Natural Language Engineering, 15(1):143–172. find
  66. Nianwen Xue, Fei Xia, Fu dong Chiou, and Martha Palmer. 2005. The Penn Chinese TreeBank: Phrase Structure Annotation of a Large Corpus. Natural Language Engineering, 11(2):207–238. find
  67. Nianwen Xue. 2008. Labeling Chinese Predicates with Semantic Roles. Computational Linguistics, 34(2):225–255. find
  68. Yaqin Yang and Nianwen Xue. 2010. Chasing the ghost: recovering empty categories in the chinese treebank. In Proceedings of Proceedings of the 23rd International Conference on Computational Linguistics (COLING), Beijing, China. find
  69. Wajdi Zaghouani, Mona Diab, Aous Mansouri, Sameer Pradhan, and Martha Palmer. 2010. The revised arabic propbank. In Proceedings of the Fourth Linguistic Annotation Workshop, pages 222–226, Uppsala, Sweden, July. find
  70. Zhi Zhong and Hwee Tou Ng. 2010. It makes sense: A wide-coverage word sense disambiguation system for free text. In Proceedings of the ACL 2010 System Demonstrations, pages 78–83, Uppsala, Sweden. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/pradhan-2012-conll-2012-shared-task-modeling/