Mova0.5 pre-release⌕
☰
home / docs / papers / Findings of the Fifth Shared Task on Multilingual Coreference Resolution: Expanding Datasets for Long-Range Entities

Findings of the Fifth Shared Task on Multilingual Coreference Resolution: Expanding Datasets for Long-Range Entities

Summary

The fifth edition of the multilingual coreference shared task, held with CODI-CRAC 2026, again asked for mention detection and identity coreference clustering. This year the focus was on long-range entities whose mentions are spread over long documents, which benchmarks dominated by short documents underrepresent. CorefUD 1.4 added five datasets and two languages (Dutch and Latin), including literary corpora with long chains and a spoken Czech corpus; 27 datasets in 19 languages were used. A dataset counts as long-range when the 95th percentile of first-to-last mention distance of non-singleton entities exceeds 1500 words. Zero anaphora is again represented by UD empty nodes. Ten systems participated in an unconstrained and an LLM track; encoder-based CorPipe systems kept first place, with the best LLM systems not far behind.

How Mova uses it

Effectiveness in Mova

Not measured separately: the paper supplies the evaluation convention and reference numbers. Mova's untrained deterministic resolver reaches CoNLL F1 60.3 on GUM test with gold trees and 54.0 with its own trees (head match, without singletons), below the shared-task baselines; settings differ (CorefUD 1.4 there, full UD 2.18 test here), so the comparison is only approximate.

👨‍🔬💥

Topics: Coreference resolution

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.codi-1.21

@inproceedings{novak2026findings,
  title = {{Findings of the Fifth Shared Task on Multilingual Coreference Resolution: Expanding Datasets for Long-Range Entities}},
  author = {Novák, Michal and Konopík, Miloslav and Nedoluzhko, Anna and Popel, Martin and Prazak, Ondrej and Sido, Jakub and Straka, Milan and Žabokrtský, Zdeněk and Zeman, Daniel},
  booktitle = {Proceedings of the 2nd Joint Workshop on Computational Approaches to Discourse, Context and Document-Level Inferences and Computational Models of Reference, Anaphora and Coreference (CODI-CRAC 2026)},
  year = {2026},
  doi = {10.18653/v1/2026.codi-1.21},
  url = {https://doi.org/10.18653/v1/2026.codi-1.21}
}

References

61 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Amit Bagga and Breck Baldwin. 1998. Algorithms for Scoring Coreference Chains. In Proceedings of The First International Conference on Language Resources and Evaluation Workshop on Linguistics Coreference, pages 563–566. find our card
  2. David Bamman, Olivia Lewke, and Anya Mansoor. 2020. An Annotated Dataset of Coreference in English Literature. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 44–54, Marseille, France. European Language Resources Association. find our card
  3. David Bamman, Ted Underwood, and Noah A. Smith. 2014. A Bayesian Mixed Effects Model of Literary Character. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 370–379, Baltimore, Maryland. Association for Computational Linguistics. ↗
  4. Antoine Bourgois and Thierry Poibeau. 2025. The Elephant in the Coreference Room: Resolving Coreference in Full-Length French Fiction Works. In Proceedings of the Eighth Workshop on Computational Models of Reference, Anaphora and Coreference, pages 55–69, Suzhou, China. Association for Computational Linguistics. ↗
  5. Peter Bourgonje and Manfred Stede. 2020. The Potsdam Commentary Corpus 2.2: Extending Annotations for Shallow Discourse Parsing. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 1061–1066, Marseille, France. European Language Resources Association. find
  6. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised Cross-lingual Representation Learning at Scale. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8440– 8451, Online. Association for Computational Linguistics. ↗
  7. Eleonora Delfino, Roberta Leotta, Marco Passarotti, and Giovanni Moretti. 2024. Building CorefLat. A Linguistic Resource for Coreference and Anaphora Resolution in Latin. In Proceedings of the 10th Italian Conference on Computational Linguistics (CLiC-it 2024), pages 273–279. find
  8. Pascal Denis and Jason Baldridge. 2009. Global joint models for coreference resolution and named entity classification. Procesamiento del lenguaje natural, 42. find
  9. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. CoRR, abs/1810.04805. *Figure 4: Performance of the systems on documents as their p95 entity range increases. Each data point represents the average performance across multiple documents in a sliding window of the fixed size of 50k tokens.* Qipeng Guo, Xiangkun Hu, Yue Zhang, Xipeng Qiu, and Zheng Zhang. 2023. Dual Cache for Long Document Neural Coreference Resolution. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15272–15285, Toronto, Canada. Association for Computational Linguistics. ↗
  10. Talika Gupta, Hans Ole Hatzel, and Chris Biemann. 2024. Coreference in Long Documents using Hierarchical Entity Merging. In Proceedings of the 8th Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature (LaTeCH-CLfL 2024), pages 11–17, St. Julians, Malta. Association for Computational Linguistics. ↗
  11. Sooyoun Han, Sumin Seo, Minji Kang, Jongin Kim, Nayoung Choi, Min Song, and Jinho D. Choi. 2021. FantasyCoref: Coreference Resolution on Fantasy Literature Through Omniscient Writer’s Point of View. In Proceedings of the Fourth Workshop on Computational Models of Reference, Anaphora and Coreference, pages 24–35, Punta Cana, Dominican Republic. Association for Computational Linguistics. ↗
  12. Dag Trygve Truslew Haug and Marius L. Jøhndal. 2008. Creating a Parallel Treebank of the Old Indo-European Bible Translations. In Proceedings of the second workshop on language technology for cultural heritage data (LaTeCH 2008). find
  13. Kyungeun Kim, Seungjun Lee, Yohan Jo, Sungchul Kim, Chanhee Lee, Junyeong Park, Heegeun Yoon, Seok-min Shin, Ilhong Yun, and Kyomin Jung. 2024. Ko-CoNovel: Annotated Dataset of Character Coreference in Korean Novels. find
  14. Frédéric Landragin. 2021. Le corpus Democrat et son exploitation. Présentation. Langages, 224:11–24. ↗
  15. Ekaterina Lapshinova-Koltunski, Christian Hardmeier, and Pauline Krielke. 2018. ParCorFull: a Parallel Corpus Annotated with Full Coreference. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Association. ↗
  16. Kenton Lee, Luheng He, Mike Lewis, and Luke Zettlemoyer. 2017. End-to-end Neural Coreference Resolution. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 188–197, Copenhagen, Denmark. Association for Computational Linguistics. ↗ our card
  17. Xiaoqiang Luo. 2005. On Coreference Resolution Performance Metrics. In Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing, HLT 2005, pages 25–32. Association for Computational Linguistics. ↗ our card
  18. Petter Mæhlum, Dag Haug, Tollef Jørgensen, Andre Kåsen, Anders Nøklestad, Egil Rønningstad, Per Erik Solberg, Erik Velldal, and Lilja Øvrelid. 2022. NARC–Norwegian Anaphora Resolution Corpus. In Proceedings of the Fifth Workshop on Computational Models of Reference, Anaphora and Coreference, pages 48–60, Gyeongju, Korea. Association for Computational Linguistics. find
  19. Giuliano Martinelli, Edoardo Barba, and Roberto Navigli. 2024. Maverick: Efficient and Accurate Coreference Resolution Defying Recent Trends. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 13380–13394, Bangkok, Thailand. Association for Computational Linguistics. ↗
  20. Giuliano Martinelli, Tommaso Bonomo, Pere-Lluís Huguet Cabot, and Roberto Navigli. 2025a. BOOK-COREF: Coreference Resolution at Book Scale. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 24526–24544, Vienna, Austria. Association for Computational Linguistics. find
  21. Giuliano Martinelli, Bruno Gatti, and Roberto Navigli. 2025b. xCoRe: Cross-context Coreference Resolution. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 34264–34278, Suzhou, China. Association for Computational Linguistics. ↗
  22. Frédérique Mélanié-Becquet, Jean Barré, Olga Seminck, Clément Plancq, Marco Naguib, Martial Pastor, and Thierry Poibeau. 2024. BookNLP-fr, the French Versant of BookNLP: A Tailored Pipeline for 19th and 20th Century French Literature. Journal of Computational Literary Studies, 3(1):1–34. find
  23. Marie Mikulová, Jiří Mírovský, Anna Nedoluzhko, Petr Pajas, Jan Štěpánek, and Jan Hajič. 2017a. PDTSC 2.0 – Spoken Corpus with Rich Multi-layer Structural Annotation. In Lecture Notes in Computer Science, 20th International Conference, TSD 2017 Prague, Czech Republic, No. 10415, pages 129–137, Cham / Heidelberg / New York / Dordrecht / London. Springer International Publishing. ↗
  24. Marie Mikulová, Jiří Mírovský, Anna Nedoluzhko, Petr Pajas, Jan Štěpánek, and Jan Hajič. 2017b. PDTSC 2.0 - Spoken Corpus with Rich Multi-layer Structural Annotation. In Text, Speech, and Dialogue. TSD 2017, volume 10415 of Lecture Notes in Computer Science, Cham, Switzerland. Springer. ↗
  25. Marie Mikulová, Barbora Štěpánková, Daniel Zeman, Jan Štěpánek, Milan Straka, and Jan Hajič. 2026. Meet UD_Czech-PDTC: A Large and Genre-Rich Treebank in Universal Dependencies. In Proceedings of the 15th International Conference on Language Resources and Evaluation (LREC 2026), Palma, Spain. European Language Resources Association. ↗
  26. Nafise Sadat Moosavi and Michael Strube. 2016. Which Coreference Evaluation Metric Do You Trust? A Proposal for a Link-based Entity Aware Metric. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 632–642, Berlin, Germany. Association for Computational Linguistics. ↗ our card
  27. Vandan Mujadia, Palash Gupta, and Dipti Misra Sharma. 2016. Coreference Annotation Scheme and Relation Types for Hindi. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 161–168, Portorož, Slovenia. European Language Resources Association (ELRA). ↗
  28. Judith Muzerelle, Anaïs Lefeuvre, Emmanuel Schang, Jean-Yves Antoine, Aurore Pelletier, Denis Maurel, Iris Eshkol, and Jeanne Villaneau. 2014. AN-COR_Centre, a large free spoken French coreference corpus: description of the resource and reliability measures. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 843–847, Reykjavik, Iceland. European Language Resources Association (ELRA). ↗
  29. Frédérique Mélanie-Becquet, Jean Barré, Olga Seminck, Clément Plancq, Marco Naguib, Martial Pastor, and Thierry Poibeau. 2024. BookNLP-fr, the French Versant of BookNLP. A Tailored Pipeline for 19th and 20th Century French Literature. Journal of Computational Literary Studies, 3:1–34. find
  30. Sangha Nam, Minho Lee, Donghwan Kim, Kijong Han, Kuntae Kim, Sooji Yoon, Eun-kyung Kim, and Key-Sun Choi. 2020. Effective Crowdsourcing of Multiple Tasks for Comprehensive Knowledge Extraction. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 212–219, Marseille, France. European Language Resources Association. find
  31. Anna Nedoluzhko, Michal Novák, Silvie Cinková, Marie Mikulová, and Jiří Mírovský. 2016. Coreference in Prague Czech-English Dependency Treebank. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016), pages 169–176, Portorož, Slovenia. European Language Resources Association. ↗
  32. Anna Nedoluzhko, Michal Novák, Martin Popel, Zdeněk Žabokrtský, Amir Zeldes, and Daniel Zeman. 2022. CorefUD 1.0: Coreference Meets Universal Dependencies. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 4859– 4872, Marseille, France. European Language Resources Association. ↗ our card
  33. Michal Novák, Barbora Dohnalová, Miloslav Konopik, Anna Nedoluzhko, Martin Popel, Ondrej Prazak, Jakub Sido, Milan Straka, Zdeněk Žabokrtský, and Daniel Zeman. 2024. Findings of the Third Shared Task on Multilingual Coreference Resolution. In Proceedings of the Seventh Workshop on Computational Models of Reference, Anaphora and Coreference, pages 78–96, Miami. Association for Computational Linguistics. ↗
  34. Michal Novák, Miloslav Konopik, Anna Nedoluzhko, Martin Popel, Ondrej Prazak, Jakub Sido, Milan Straka, Zdeněk Žabokrtský, and Daniel Zeman. 2025. Findings of the Fourth Shared Task on Multilingual Coreference Resolution: Can LLMs Dethrone Traditional Approaches? In Proceedings of the Eighth Workshop on Computational Models of Reference, Anaphora and Coreference, pages 95–118, Suzhou, China. Association for Computational Linguistics. ↗ our card
  35. Maciej Ogrodniczuk, Katarzyna Glowińska, Mateusz Kopeć, Agata Savary, and Magdalena Zawisławska. 2013. Polish Coreference Corpus. In Human Language Technology. Challenges for Computer Science and Linguistics — 6th Language and Technology Conference (LTC 2013), Revised Selected Papers, volume 9561 of Lecture Notes in Computer Science, pages 215–226. Springer. find
  36. Maciej Ogrodniczuk, Katarzyna Głowińska, Mateusz Kopeć, Agata Savary, and Magdalena Zawisławska. 2015. Coreference in Polish: Annotation, Resolution and Evaluation. Walter De Gruyter. find
  37. Janis Pagel and Nils Reiter. 2020. GerDraCor-Coref: A Coreference Corpus for Dramatic Texts in German. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 55–64, Marseille, France. European Language Resources Association. find
  38. Tuğba Pamay and Gülşen Eryiğit. 2018. Turkish Coreference Resolution. In 2018 Innovations in Intelligent Systems and Applications (INISTA), pages 1–7. ↗
  39. Sameer Pradhan, Xiaoqiang Luo, Marta Recasens, Eduard Hovy, Vincent Ng, and Michael Strube. 2014. Scoring Coreference Partitions of Predicted Mentions: A Reference Implementation. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 30–35, Baltimore, Maryland. Association for Computational Linguistics. ↗ our card
  40. Ondřej Pražák, Miloslav Konopík, and Jakub Sido. 2021. Multilingual Coreference Resolution with Harmonized Annotations. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021), pages 1119– 1123, Held Online. INCOMA Ltd. find
  41. Marta Recasens, Eduard Hovy, and M. Antònia Martí. 2010. A Typology of Near-Identity Relations for Coreference (NIDENT). In Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC 2010), Valletta, Malta. European Language Resources Association. ↗
  42. Marta Recasens and Eduard H. Hovy. 2011. BLANC: Implementing the Rand index for coreference evaluation. Natural Language Engineering, 17(4):485–510. ↗
  43. Marta Recasens and M. Antònia Martí. 2010. AnCora-CO: Coreferentially Annotated Corpora for Spanish and Catalan. Language Resources and Evaluation, 44(4):315–345. ↗
  44. Kumar Shridhar, Nicholas Monath, Raghuveer Thirukovalluru, Alessandro Stolfo, Manzil Zaheer, Andrew McCallum, and Mrinmaya Sachan. 2023. Longtonotes: OntoNotes with Longer Coreference Chains. In Findings of the Association for Computational Linguistics: EACL 2023, pages 1428–1442, Dubrovnik, Croatia. Association for Computational Linguistics. ↗
  45. Milan Straka. 2026. CorPipe at CRAC 2026: Empty Nodes and Cross-Lingual Transfer in Multilingual Coreference Resolution. In Proceedings of the 2nd Joint Workshop on Computational Approaches to Discourse, Context and Document-Level Inferences and Computational Models of Reference, Anaphora and Coreference (CODI-CRAC 2026), San Diego, California, USA. Association for Computational Linguistics. ↗
  46. Daniel G. Swanson, Bryce D. Bussert, and Francis Tyers. 2024. Towards Named-Entity and Coreference Annotation of the Hebrew Bible. In Proceedings of the Third Workshop on Language Technologies for Historical and Ancient Languages (LT4HALA) @ LREC-COLING-2024, pages 36–40, Torino, Italia. ELRA and ICCL. ↗ our card
  47. Mariona Taulé, M. Antònia Martí, and Marta Recasens. 2008. AnCora: Multilevel Annotated Corpora for Catalan and Spanish. In Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC 2008), Marrakech, Morocco. European Language Resources Association. ↗
  48. Svetlana Toldova, Anna Roytberg, Alina Ladygina, Maria Vasilyeva, Ilya Azerkovich, Matvei Kurzukov, G. Sim, D.V. Gorshkov, A. Ivanova, Anna Nedoluzhko, and Yulia Grishina. 2014. Evaluating Anaphora and Coreference Resolution for Russian. In Komp’juternaja lingvistika i intellektual’nye tehnologii. Po materialam ezhegodnoj Mezhdunarodnoj konferencii Dialog, pages 681–695. find
  49. Shubham Toshniwal, Sam Wiseman, Allyson Ettinger, Karen Livescu, and Kevin Gimpel. 2020. Learning to Ignore: Long Document Coreference with Bounded Memory Neural Networks. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8519–8526, Online. Association for Computational Linguistics. ↗
  50. Shubham Toshniwal, Patrick Xia, Sam Wiseman, Karen Livescu, and Kevin Gimpel. 2021. On Generalization in Coreference Resolution. In Proceedings of the Fourth Workshop on Computational Models of Reference, Anaphora and Coreference, pages 111– 120, Punta Cana, Dominican Republic. Association for Computational Linguistics. ↗ our card
  51. Noémi Vadász. 2022. Building a Manually Annotated Hungarian Coreference Corpus: Workflow and Tools. In Proceedings of the Fifth Workshop on Computational Models of Reference, Anaphora and Coreference, pages 38–47, Gyeongju, Korea. Association for Computational Linguistics. find
  52. Andreas van Cranenburgh. 2019. A Dutch coreference resolution system with an evaluation on literary fiction. Computational Linguistics in the Netherlands Journal. find
  53. Andreas van Cranenburgh and Gertjan van Noord. 2022. OpenBoek: A Corpus of Literary Coreference and Entities with an Exploration of Historical Spelling Normalization. Computational Linguistics in the Netherlands Journal, 12:235–251. find
  54. Marc Vilain, John Burger, John Aberdeen, Dennis Connolly, and Lynette Hirschman. 1995. A Model-Theoretic Coreference Scoring Scheme. In Sixth Message Understanding Conference (MUC-6): Proceedings of a Conference Held in Columbia, Maryland, November 6-8, 1995. ↗ our card
  55. Veronika Vincze, Klára Hegedűs, Alex Sliz-Nagy, and Richárd Farkas. 2018. SzegedKoref: A Hungarian Coreference Corpus. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Association. ↗
  56. Ralph Weischedel, Eduard Hovy, Mitchell Marcus, Martha Palmer, Robert Belvin, Sameer Pradhan, Lance Ramshaw, and Nianwen Xue. 2011. OntoNotes: A Large Training Corpus for Enhanced Processing. In Handbook of Natural Language Processing and Machine Translation: DARPA Global Autonomous Language Exploitation, pages 54–63, New York. Springer-Verlag. find
  57. Juntao Yu, Michal Novák, Abdulrahman Aloraini, Nafise Sadat Moosavi, Silviu Paun, Sameer Pradhan, and Massimo Poesio. 2023. The Universal Anaphora Scorer 2.0. In Proceedings of the 15th International Conference on Computational Semantics, pages 183– 194, Nancy, France. Association for Computational Linguistics. find
  58. Zdeněk Žabokrtský, Miloslav Konopík, Anna Nedoluzhko, Michal Novák, Maciej Ogrodniczuk, Martin Popel, Ondřej Pražák, Jakub Sido, and Daniel Zeman. 2023. Findings of the Second Shared Task on Multilingual Coreference Resolution. In Proceedings of the CRAC 2023 Shared Task on Multilingual Coreference Resolution, pages 1–18, Singapore. Association for Computational Linguistics. ↗
  59. Zdeněk Žabokrtský, Miloslav Konopík, Anna Nedoluzhko, Michal Novák, Maciej Ogrodniczuk, Martin Popel, Ondřej Pražák, Jakub Sido, Daniel Zeman, and Yilun Zhu. 2022. Findings of the Shared Task on Multilingual Coreference Resolution. In Proceedings of the CRAC 2022 Shared Task on Multilingual Coreference Resolution, pages 1–17, Gyeongju, Korea. Association for Computational Linguistics. ↗
  60. Amir Zeldes. 2017. The GUM Corpus: Creating Multilayer Resources in the Classroom. Language Resources and Evaluation, 51(3):581–612. ↗
  61. Voldemaras Žitkus and Rita Butkienė. 2018. Coreference Annotation Scheme and Corpus for Lithuanian Language. In Fifth International Conference on Social Networks Analysis, Management and Security, SNAMS 2018, Valencia, Spain, October 15-18, 2018, pages 243–250. IEEE. ↗

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/novak-2026-findings-fifth-shared-task-multilingual/