Mova0.5 pre-release⌕
☰
home / docs / papers / English / Challenges to Evaluating the Generalization of Coreference Resolution Models: A Measurement Modeling Perspective

Challenges to Evaluating the Generalization of Coreference Resolution Models: A Measurement Modeling Perspective

What it is about

The paper asks whether multi-dataset evaluations of coreference models really measure generalization. It borrows the measurement-modelling framework of the social sciences: a construct (what is measured, here coreference) is separated from its operationalization (how the annotation guidelines implement it). Seven English datasets are compared (OntoNotes, PreCo, Phrase Detectives, OntoGUM, LitBank, ARRAU, MMC). The three largest differ in definitions: whether generic mentions, verb-phrase heads, appositions and copular predicates count as coreference, and in nesting and compound modifiers. The authors define an aggregate generalization gap and, per coreference type (nested, OntoNotes-generic, compound, copular), a type generalization gap, computed with a B3-based score restricted to mentions of that type. Their claim, from the abstract, is that measured generalization is correlated with the differences in definition and operationalization, so a single generalization number conflates them. The abstract does not give a single headline score.

Ideas

For Mova

Code: coref/src/eval.rs (per-type scores), coref/src/mention.rs (mention types from UD relations), coref/src/corefud.rs. Chapter: 35. Reference and coreference. Measure: per-type scores on two corpora with different definitions, run twice, and a negative control where the type labels are shuffled, which should give no gap pattern.

Topics: Coreference resolution

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/2024.findings-acl.909 · arXiv 2303.09092

@inproceedings{porada2024challenges,
  title = {{Challenges to Evaluating the Generalization of Coreference Resolution Models: A Measurement Modeling Perspective}},
  author = {Porada, Ian and Olteanu, Alexandra and Suleman, Kaheer and Trischler, Adam and Cheung, Jackie},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2024},
  year = {2024},
  doi = {10.18653/v1/2024.findings-acl.909},
  eprint = {2303.09092},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2024.findings-acl.909}
}

References

84 references, 20 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Robert Adcock and David Collier. 2001. Measurement validity: A shared standard for qualitative and quantitative research. American Political Science Review, 95(3):529–546. find
  2. Amit Bagga and Breck Baldwin. 1998. Entity-based cross-document coreferencing using the vector space model. In 36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics, Volume 1, pages 79–85, Montreal, Quebec, Canada. Association for Computational Linguistics. find
  3. David Bamman, Olivia Lewke, and Anya Mansoor. 2020. An annotated dataset of coreference in English literature. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 44–54, Marseille, France. European Language Resources Association. find our card
  4. BBN Technologies. 2007. Co-reference guidelines for english ontonotes. Version 7.0. Guidelines accompanying OntoNotes 5.0 data release. find
  5. Iz Beltagy, Matthew E. Peters, and Arman Cohan. 2020. Longformer: The long-document transformer. arXiv:2004.05150. ↗
  6. Anol Bhattacherjee. 2012. Social science research: Principles, methods, and practices. USA. find
  7. Thomas R. Black. 1999. Doing quantitative research in the social sciences: An integrated approach to research design, measurement and statistics. Doing quantitative research in the social sciences: An integrated approach to research design, measurement and statistics., pages xiv, 751–xiv, 751. Place: Thousand Oaks, CA Publisher: Sage Publications Ltd. find
  8. Su Lin Blodgett, Gilsinia Lopez, Alexandra Olteanu, Robert Sim, and Hanna Wallach. 2021. Stereotyping Norwegian salmon: An inventory of pitfalls in fairness benchmark datasets. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 1004–1015, Online. Association for Computational Linguistics. find
  9. Bernd Bohnet, Chris Alberti, and Michael Collins. 2023. Coreference resolution through a seq2seq transition-based system. Transactions of the Association for Computational Linguistics, 11:212–226. find
  10. K. A. Bollen. 2001. Measurement models. In Neil J. Smelser and Paul B. Baltes, editors, Indicator: Methodology, International Encyclopedia of the Social & Behavioral Sciences, chapter 2, pages 7282– 7287. Pergamon, Oxford. find
  11. Yang Trista Cao and Hal Daumé III. 2021. Toward gender-inclusive coreference resolution: An analysis of gender and bias throughout the machine learning lifecycle. Computational Linguistics, 47(3):615– 661. find
  12. Haixia Chai, Nafise Sadat Moosavi, Iryna Gurevych, and Michael Strube. 2022. Evaluating coreference resolvers on community-based question answering: From rule-based to state of the art. In Proceedings of the Fifth Workshop on Computational Models of Reference, Anaphora and Coreference, pages 61–73, Gyeongju, Republic of Korea. Association for Computational Linguistics. find
  13. Haixia Chai and Michael Strube. 2023. Investigating multilingual coreference resolution by universal annotations. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 10010– 10024, Singapore. Association for Computational Linguistics. find
  14. Hong Chen, Zhenhua Fan, Hao Lu, Alan Yuille, and Shu Rong. 2018. PreCo: A large-scale dataset in preschool vocabulary for coreference resolution. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 172– 181, Brussels, Belgium. Association for Computational Linguistics. find our card
  15. Nancy Chinchor. 1992. The statistical significance of the MUC-4 results. In Fourth Message Understanding Conference (MUC-4): Proceedings of a Conference Held in McLean, Virginia, June 16-18, 1992. find
  16. H. H. Clark. 1977. Bridging, pages 411–420. Cambridge University Press. find
  17. Agata Cybulska and Piek Vossen. 2014. Using a sledgehammer to crack a nut? lexical diversity and event coreference resolution. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 4545–4552, Reykjavik, Iceland. European Language Resources Association (ELRA). find
  18. Pradeep Dasigi, Nelson F. Liu, Ana Marasović, Noah A. Smith, and Matt Gardner. 2019. Quoref: A reading comprehension dataset with questions requiring coreferential reasoning. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 5925–5932, Hong Kong, China. Association for Computational Linguistics. find our card
  19. Pascal Denis and Jason Baldridge. 2009. Global joint models for coreference resolution and named entity classification. Procesamiento del lenguaje natural, 42. find
  20. George Doddington, Alexis Mitchell, Mark Przybocki, Lance Ramshaw, Stephanie Strassel, and Ralph Weischedel. 2004. The automatic content extraction (ACE) program – tasks, data, and evaluation. In Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC’04), Lisbon, Portugal. European Language Resources Association (ELRA). find
  21. Greg Durrett and Dan Klein. 2013. Easy victories and uphill battles in coreference resolution. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1971–1982, Seattle, Washington, USA. Association for Computational Linguistics. find our card
  22. Ellen Edes. 1968. Output conditions on anaphoric expressions with split antecedents. unpublished paper, Harvard University. find
  23. Ali Emami, Paul Trichelair, Adam Trischler, Kaheer Suleman, Hannes Schulz, and Jackie Chi Kit Cheung. 2019. The KnowRef coreference corpus: Removing gender and number cues for difficult pronominal anaphora resolution. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3952–3961, Florence, Italy. Association for Computational Linguistics. find our card
  24. Nupoor Gandhi, Anjalie Field, and Emma Strubell. 2022. Mention annotations alone enable efficient domain adaptation for coreference resolution. find
  25. Ralph Grishman and Beth Sundheim. 1996. Design of the MUC-6 evaluation. In TIPSTER TEXT PRO-GRAM PHASE II: Proceedings of a Workshop held at Vienna, Virginia, May 6-8, 1996, pages 413–422, Vienna, Virginia, USA. Association for Computational Linguistics. find
  26. Graeme Hirst. 1981. Anaphora, pages 4–32. Springer Berlin Heidelberg, Berlin, Heidelberg. find
  27. Jerry R Hobbs. 1978. Resolving pronoun references. Lingua, 44(4):311–338. find our card
  28. Tamanna Hossain, Sunipa Dev, and Sameer Singh. 2023. MISGENDERED: Limits of large language models in understanding pronouns. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5352–5367, Toronto, Canada. Association for Computational Linguistics. find
  29. Ben Hutchinson, Negar Rostamzadeh, Christina Greer, Katherine Heller, and Vinodkumar Prabhakaran. 2022. Evaluation gaps in machine learning practice. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’22, page 1859–1876, New York, NY, USA. Association for Computing Machinery. find
  30. Abigail Z. Jacobs and Hanna Wallach. 2021. Measurement and fairness. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’21, page 375–385, New York, NY, USA. Association for Computing Machinery. find
  31. Robert Neal Kantor. 1977. The management and comprehension of discourse connection by pronouns in English. The Ohio State University. find
  32. R.B. Kline. 2011. Principles and Practice of Structural Equation Modeling. Methodology in the social sciences. Guilford Publications. find
  33. Vid Kocijan, Oana-Maria Camburu, and Thomas Lukasiewicz. 2021. The gap on gap: Tackling the problem of differing data distributions in bias-measuring datasets. Proceedings of the AAAI Conference on Artificial Intelligence, 35(14):13180–13188. find
  34. Sandra Kübler and Desislava Zhekova. 2011. Singletons and coreference resolution evaluation. In Proceedings of the International Conference Recent Advances in Natural Language Processing 2011, pages 261–267, Hissar, Bulgaria. Association for Computational Linguistics. find our card
  35. Ekaterina Lapshinova-Koltunski, Pedro Augusto Ferreira, Elina Lartaud, and Christian Hardmeier. 2022. ParCorFull2.0: a parallel corpus annotated with full coreference. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 805–813, Marseille, France. European Language Resources Association. find
  36. Jing Lu and Vincent Ng. 2018. Event coreference resolution: A survey of two decades of research. In IJCAI, pages 5479–5486. find
  37. Jing Lu and Vincent Ng. 2020. Conundrums in entity coreference resolution: Making sense of the state of the art. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6620–6631, Online. Association for Computational Linguistics. find
  38. Christopher Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven Bethard, and David McClosky. 2014. The Stanford CoreNLP natural language processing toolkit. In Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 55–60, Baltimore, Maryland. Association for Computational Linguistics. find
  39. Nafise Sadat Moosavi, Leo Born, Massimo Poesio, and Michael Strube. 2019. Using automatically extracted minimum spans to disentangle coreference evaluation from boundary detection. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4168–4178, Florence, Italy. Association for Computational Linguistics. find our card
  40. Nafise Sadat Moosavi and Michael Strube. 2017. Lexical features in coreference resolution: To be used with caution. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 14–19, Vancouver, Canada. Association for Computational Linguistics. find our card
  41. Nafise Sadat Moosavi and Michael Strube. 2018. Using linguistic features to improve the generalization capability of neural coreference resolvers. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 193–203, Brussels, Belgium. Association for Computational Linguistics. find our card
  42. Anna Nedoluzhko, Michal Novák, Martin Popel, Zdenek Žabokrtskỳ, and Daniel Zeman. 2021. Coreference meets universal dependencies–a pilot experiment on harmonizing coreference datasets for 11 languages. ÚFAL MFF UK, Praha, Czechia. find our card
  43. Joakim Nivre, Marie-Catherine de Marneffe, Filip Ginter, Jan Hajič, Christopher D. Manning, Sampo Pyysalo, Sebastian Schuster, Francis Tyers, and Daniel Zeman. 2020. Universal Dependencies v2: An evergrowing multilingual treebank collection. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 4034–4043, Marseille, France. European Language Resources Association. find our card
  44. Haoruo Peng, Daniel Khashabi, and Dan Roth. 2015. Solving hard coreference problems. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 809–819, Denver, Colorado. Association for Computational Linguistics. find
  45. M. Poesio, F. Bruneseaux, and L. Romary. 1999. The MATE meta-scheme for coreference in dialogues in multiple languages. In Towards Standards and Tools for Discourse Tagging. find
  46. Massimo Poesio, Juntao Yu, Silviu Paun, Abdulrahman Aloraini, Pengcheng Lu, Janosch Haber, and Derya Cokal. 2023. Computational models of anaphora. Annual Review of Linguistics, 9:561–587. find
  47. Corbèn Poot and Andreas van Cranenburgh. 2020. A benchmark of rule-based and neural coreference resolution in Dutch novels and news. In Proceedings of the Third Workshop on Computational Models of Reference, Anaphora and Coreference, pages 79–90, Barcelona, Spain (online). Association for Computational Linguistics. find
  48. Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Olga Uryupina, and Yuchen Zhang. 2012. CoNLL-2012 shared task: Modeling multilingual unrestricted coreference in OntoNotes. In Joint Conference on EMNLP and CoNLL - Shared Task, pages 1–40, Jeju Island, Korea. Association for Computational Linguistics. find our card
  49. Peng Qi, Yuhao Zhang, Yuhui Zhang, Jason Bolton, and Christopher D. Manning. 2020. Stanza: A Python natural language processing toolkit for many human languages. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations. ↗ our card
  50. Altaf Rahman and Vincent Ng. 2012. Resolving complex cases of definite pronouns: The Winograd schema challenge. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 777–789, Jeju Island, Korea. Association for Computational Linguistics. find
  51. Shon Otmazgin, Arie Cattan, and Yoav Goldberg. 2022. Lingmess: Linguistically informed multi expert scorers for coreference resolution. find our card
  52. Ronald E Robertson, Alexandra Olteanu, Fernando Diaz, Milad Shokouhi, and Peter Bailey. 2021. “i can’t reply with that”: Characterizing problematic email reply suggestions. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, CHI ’21, New York, NY, USA. Association for Computing Machinery. find
  53. Shon Otmazgin, Arie Cattan, and Yoav Goldberg. 2023. LingMess: Linguistically informed multi expert scorers for coreference resolution. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2752– 2760, Dubrovnik, Croatia. Association for Computational Linguistics. find our card
  54. Ina Roesiger, Arndt Riester, and Jonas Kuhn. 2018. Bridging resolution: Task definition, corpus resources and rule-based experiments. In Proceedings of the 27th International Conference on Computational Linguistics, pages 3516–3528, Santa Fe, New Mexico, USA. Association for Computational Linguistics. find our card
  55. Nikil Roashan Selvam, Sunipa Dev, Daniel Khashabi, Tushar Khot, and Kai-Wei Chang. 2022. The tail wagging the dog: Dataset construction biases of social bias benchmarks. arXiv preprint arXiv:2210.10040. ↗
  56. Kellie Webster, Marta Recasens, Vera Axelrod, and Jason Baldridge. 2018. Mind the GAP: A balanced corpus of gendered ambiguous pronouns. Transactions of the Association for Computational Linguistics, 6:605–617. find our card
  57. Milan Straka. 2023. ÚFAL CorPipe at CRAC 2023: Larger context improves multilingual coreference resolution. In Proceedings of the CRAC 2023 Shared Task on Multilingual Coreference Resolution, pages 41–51, Singapore. Association for Computational Linguistics. find
  58. Ralph Weischedel et al. 2013. Ontonotes release 5.0. find
  59. Anselm Strauss and Juliet M Corbin. 1997. Grounded theory in practice. Sage. find
  60. Sanjay Subramanian and Dan Roth. 2019. Improving generalization in coreference resolution via adversarial training. In Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics (*SEM 2019), pages 192–197, Minneapolis, Minnesota. Association for Computational Linguistics. find
  61. Rhea Sukthanker, Soujanya Poria, Erik Cambria, and Ramkumar Thirunavukarasu. 2020. Anaphora and coreference resolution: A review. Information Fusion, 59:139–162. find
  62. Shubham Toshniwal, Patrick Xia, Sam Wiseman, Karen Livescu, and Kevin Gimpel. 2021. On generalization in coreference resolution. In Proceedings of the Fourth Workshop on Computational Models of Reference, Anaphora and Coreference, pages 111–120, Punta Cana, Dominican Republic. Association for Computational Linguistics. ↗ our card
  63. Gorka Urbizu, Ander Soraluze, and Olatz Arregi. 2019. Deep cross-lingual coreference resolution for less-resourced languages: The case of Basque. In Proceedings of the Second Workshop on Computational Models of Reference, Anaphora and Coreference, pages 35–41, Minneapolis, USA. Association for Computational Linguistics. find
  64. Olga Uryupina. 2008. Error analysis for learning-based coreference resolution. In Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC’08), Marrakech, Morocco. European Language Resources Association (ELRA). find
  65. Olga Uryupina, Ron Artstein, Antonella Bristot, Federica Cavicchio, Francesca Delogu, Kepa J. Rodriguez, and Massimo Poesio. 2020. Annotating a broad range of anaphoric phenomena, in a variety of genres: the arrau corpus. Natural Language Engineering, 26(1):95–128. find
  66. Yannick Versley. 2008. Vagueness and referential ambiguity in a large-scale annotated corpus. Research on Language and Computation, 6(3):333–353. find
  67. Bonnie Lynn Webber. 1991. Structure and ostension in the interpretation of discourse deixis. Language and Cognitive processes, 6(2):107–135. find
  68. Terry Winograd. 1972. Understanding natural language. Cognitive psychology, 3(1):1–191. find
  69. Patrick Xia and Benjamin Van Durme. 2021. Moving on from OntoNotes: Coreference resolution model transfer. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5241–5256, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics. find
  70. Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2021. mT5: A massively multilingual pre-trained text-to-text transformer. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 483–498, Online. Association for Computational Linguistics. find
  71. Jian Bo Yang, Qi Mao, Qiao Liang Xiang, Ivor Wai-Hung Tsang, Kian Ming Adam Chai, and Hai Leong Chieu. 2012. Domain adaptation for coreference resolution: An adaptive ensemble approach. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 744–753, Jeju Island, Korea. Association for Computational Linguistics. find
  72. Juntao Yu, Michal Novák, Abdulrahman Aloraini, Nafise Sadat Moosavi, Silviu Paun, Sameer Pradhan, and Massimo Poesio. 2023a. The universal anaphora scorer 2.0. In Proceedings of the 15th International Conference on Computational Semantics, pages 183– 194, Nancy, France. Association for Computational Linguistics. find
  73. Juntao Yu, Silviu Paun, Maris Camilleri, Paloma Garcia, Jon Chamberlain, Udo Kruschwitz, and Massimo Poesio. 2023b. Aggregating crowdsourced and automatic judgments to scale up a corpus of anaphoric reference for fiction and Wikipedia texts. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 767–781, Dubrovnik, Croatia. Association for Computational Linguistics. find
  74. Michelle Yuan, Patrick Xia, Chandler May, Benjamin Van Durme, and Jordan Boyd-Graber. 2022. Adapting coreference resolution models through active learning. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7533–7549, Dublin, Ireland. Association for Computational Linguistics. find
  75. Zdeněk Žabokrtský, Miloslav Konopik, Anna Nedoluzhko, Michal Novák, Maciej Ogrodniczuk, Martin Popel, Ondrej Prazak, Jakub Sido, and Daniel Zeman. 2023. Findings of the second shared task on multilingual coreference resolution. In Proceedings of the CRAC 2023 Shared Task on Multilingual Coreference Resolution, pages 1–18, Singapore. Association for Computational Linguistics. ↗
  76. Zdeněk Žabokrtský, Miloslav Konopík, Anna Nedoluzhko, Michal Novák, Maciej Ogrodniczuk, Martin Popel, Ondřej Pražák, Jakub Sido, Daniel Zeman, and Yilun Zhu. 2022. Findings of the shared task on multilingual coreference resolution. In Proceedings of the CRAC 2022 Shared Task on Multilingual Coreference Resolution, pages 1–17, Gyeongju, Republic of Korea. Association for Computational Linguistics. find
  77. Amir Zeldes. 2017. The gum corpus: Creating multilayer resources in the classroom. Lang. Resour. Eval., 51(3):581–612. find
  78. Amir Zeldes. 2022. Opinion piece: Can we fix the scope for coreference? Dialogue & Discourse, 13(1):41–62. find our card
  79. Amir Zeldes and Shuo Zhang. 2016. When annotation schemes change rules help: A configurable approach to coreference resolution beyond OntoNotes. In Proceedings of the Workshop on Coreference Resolution Beyond OntoNotes (CORBON 2016), pages 92–101, San Diego, California. Association for Computational Linguistics. find
  80. Shanheng Zhao and Hwee Tou Ng. 2014. Domain adaptation with active learning for coreference resolution. In Proceedings of the 5th International Workshop on Health Text Mining and Information Analysis (Louhi), pages 21–29, Gothenburg, Sweden. Association for Computational Linguistics. find
  81. Boyuan Zheng, Patrick Xia, Mahsa Yarmohammadi, and Benjamin Van Durme. 2023. Multilingual coreference resolution in multiparty dialogue. Transactions of the Association for Computational Linguistics, 11:922–940. find
  82. Ethan Zhou and Jinho D. Choi. 2018. They exist! introducing plural mentions to coreference resolution and entity linking. In Proceedings of the 27th International Conference on Computational Linguistics, pages 24–34, Santa Fe, New Mexico, USA. Association for Computational Linguistics. find
  83. Yilun Zhu, Sameer Pradhan, and Amir Zeldes. 2021. OntoGUM: Evaluating contextualized SOTA coreference resolution on 12 more genres. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 461–467, Online. Association for Computational Linguistics. ↗
  84. Heike Zinsmeister and Stefanie Dipper. 2010. Towards a standard for annotating abstract anaphora. In Workshop on Language Resource and Language Technology Standards at the International Conference on Language Resources and Evaluation. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/porada-2024-challenges-evaluating-generalization-coreference-resolution/