WikiCoref: An English Coreference-annotated Corpus of Wikipedia Articles

What it is about

WikiCoref is an English coreference corpus of Wikipedia articles, a genre that earlier corpora covered little, since most of them are newswire. The annotation follows OntoNotes with some changes. Each markable has a coreference type, a mention type and, for each chain, the equivalent Freebase topic. The first release has 30 documents, 59,652 tokens and 2,229 sentences, from 209 to 9,869 tokens each. The mention extent is the maximal one, so it includes pre-modifiers, post-modifiers and relative clauses. Agreement between two annotators is a Cohen's kappa of 0.78 on mention identification, and a MUC F1 of 83.3%. The authors call both values fair agreement.

Ideas

• I1. Use the maximal-extent rule as a test for coref/src/mention.rs: a mention should span the full UD subtree of its head, minus the determiner where the paper excludes it. Report the share of mentions whose span differs and why.
• I2. Run the coreference sieve on the 30 documents, with the scores split by document length, to see whether the errors come from the long documents. Keep the 30 documents as an evaluation set, not for development.
• I3. Link each chain to a world entity, as the paper links chains to Freebase topics, using Mova's knowledge base in world/src/knowledge.rs. Count how many chains find an entity and how many stay unlinked.
• I4. Repeat the paper's double annotation with Mova's own output against a human check on 20 documents, and report kappa and MUC F1 together.

For Mova

The chapter is 35. Reference and coreference. The code is coref/src/mention.rs and coref/src/sieve.rs, with the entity links in world. The measure is MUC F1 and CoNLL coreference F1 on the held-out documents, run twice. The corpus is NC-licensed, so it is used for evaluation only.

Topics: Coreference resolution · Treebanks

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.63317/3yjtmzi2qxqk

@inproceedings{ghaddar2016wikicoref,
  title = {{WikiCoref: An English Coreference-annotated Corpus of Wikipedia Articles}},
  author = {Ghaddar, Abbas and Langlais, Phillippe},
  booktitle = {Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)},
  year = {2016},
  doi = {10.63317/3yjtmzi2qxqk},
  url = {https://doi.org/10.63317/3yjtmzi2qxqk}
}

References

25 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Bollacker, K., Evans, C., Paritosh, P., Sturge, T., and Taylor, J. (2008). Freebase: a collaboratively created graph database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD international conference on Management of data, pages 1247–1250. find
2. Carletta, J. (1996). Assessing agreement on classification tasks: the kappa statistic. Computational linguistics, 22(2):249–254. find
3. Castano, J., Zhang, J., and Pustejovsky, J. (2002). Anaphora resolution in biomedical literature. In International Symposium on Reference Resolution. find
4. Cohen, K. B., Lanfranchi, A., Corvey, W., Baumgartner Jr, W. A., Roeder, C., Ogren, P. V., Palmer, M., and Hunter, L. (2010). Annotation of all coreference in biomedical text: Guideline selection and adaptation. In Proceedings of BioTxtM 2010, pages 37–41. find
5. Doddington, G. R., Mitchell, A., Przybocki, M. A., Ramshaw, L. A., Strassel, S., and Weischedel, R. M. (2004). The automatic content extraction (ACE) program-tasks, data, and evaluation. In LREC, volume 2, page 1. find
6. Haghighi, A. and Klein, D. (2009). Simple coreference resolution with rich syntactic and semantic features. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 1152–1161. find our card
7. Hendrickx, I. and Hoste, V. (2009). Coreference resolution on blogs and commented news. In Anaphora Processing and Applications, pages 43–53. find
8. Hirshman, L. and Chinchor, N. (1998). MUC-7 coreference task definition. version 3.0. In Proceedings of the Seventh Message Understanding Conference (MUC-7). find
9. Klaus, K. (1980). Content analysis: An introduction to its methodology. find
10. Lee, H., Chang, A., Peirsman, Y., Chambers, N., Surdeanu, M., and Jurafsky, D. (2013). Deterministic coreference resolution based on entity-centric, precision-ranked rules. Computational Linguistics, 39(4):885–916. find our card
11. Manning, C. D., Surdeanu, M., Bauer, J., Finkel, J. R., Bethard, S., and McClosky, D. (2014). The stanford CoreNLP natural language processing toolkit. In ACL (System Demonstrations), pages 55–60. find
12. Milne, D. and Witten, I. H. (2008). Learning to link with wikipedia. In Proceedings of the 17th ACM conference on Information and knowledge management, pages 509– 518. find
13. Müller, C. and Strube, M. (2006). Multi-level annotation of linguistic data with MMAX2. Corpus technology and language pedagogy: New resources, new tools, new methods, pages 197–214. find
14. Nguyen, N., Kim, J. D., and Tsujii, J. (2011). Overview of bionlp 2011 protein coreference shared task. In Proceedings of BioNLP Shared Task 2011 Workshop, pages 74–82. find
15. Nicolov, N., Salvetti, F., and Ivanova, S. (2008). Sentiment analysis: Does coreference matter. In AISB 2008 Convention Communication, Interaction and Social Intelligence, volume 1, page 37. find
16. Poesio, M., Di Eugenio, B., and Keohane, G. (2002). Discourse structure and anaphora: An empirical study. Rapport Technique NLE Technical Note TN-02-02. University of Essex. find
17. Poesio, M. (2004). Discourse annotation and semantic annotation in the GNOME corpus. In Proceedings of the 2004 ACL Workshop on Discourse Annotation, pages 72–79. find
18. Pradhan, S. S., Ramshaw, L., Weischedel, R., MacBride, J., and Micciulla, L. (2007). Unrestricted coreference: Identifying entities and events in OntoNotes. In null, pages 446–453. find
19. Pradhan, S., Ramshaw, L., Marcus, M., Palmer, M., Weischedel, R., and Xue, N. (2011). Conll-2011 shared task: Modeling unrestricted coreference in ontonotes. In Proceedings of the Fifteenth Conference on Computational Natural Language Learning: Shared Task, pages 1–27. find our card
20. Raghunathan, K., Lee, H., Rangarajan, S., Chambers, N., Surdeanu, M., Jurafsky, D., and Manning, C. (2010). A multi-pass sieve for coreference resolution. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 492–501. find our card
21. Ratinov, L. and Roth, D. (2009). Design challenges and misconceptions in named entity recognition. In Proceedings of the Thirteenth Conference on Computational Natural Language Learning, pages 147–155. find
22. Rodriguez, K. J., Delogu, F., Versley, Y., Stemle, E. W., and Poesio, M. (2010). Anaphoric annotation of wikipedia and blogs in the live memories corpus. In Proceedings of LREC, pages 157–163. find
23. Schäfer, U., Spurk, C., and Steffen, J. (2012). A fully coreference-annotated corpus of scholarly papers from the acl anthology. In Proceedings of COLING 2012, pages 1059–1070. find
24. Segura-Bedmar, I., Crespo, M., de Pablo-Sánchez, C., and Martı́nez, P. (2010). Resolving anaphoras for the extraction of drug-drug interactions in pharmacological documents. BMC bioinformatics, 11:1–11. find
25. Vilain, M., Burger, J., Aberdeen, J., Connolly, D., and Hirschman, L. (1995). A model-theoretic coreference scoring scheme. In Proceedings of the 6th conference on Message understanding, pages 45–52. find our card

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/ghaddar-2016-wikicoref-english-coreference-annotated-corpus/