Beyond Word Boundaries: A Hebrew Coreference Benchmark and an Evaluation Protocol for Morphologically Complex Text

What it is about

The paper builds KibutzR, a Modern Hebrew coreference dataset on the UD Hebrew treebank. The 6,151 sentences are regrouped into 351 documents, and any document that appears in more than one partition is moved wholly into training. Mentions are marked at word, morpheme and multiword level: a possessive suffix or clitic is its own mention, and a construct-state chain is a nested mention with its embedded head. A rule-based detector that reads UD parses proposes candidates, and three annotators work in five stages, with final agreement of 0.87 mention F1 and 0.81 CoNLL F1. Models are scored in three input regimes (raw text, automatic segmentation, gold segmentation): neural encoders reach 50.4 CoNLL F1 with gold segmentation, and automatic segmentation costs them 1.4 to 2.6 points. The best LLM (GPT-4o) reaches 45.4 with gold mentions, 5.0 points under the encoder.

Ideas

• I1. Three-regime report for Mova's Hebrew stack: run any mention or clause-level output on gold segmentation and on the segmenter's output, and report the difference as its own number. The paper's point is that segmentation alone costs points and is hidden if only gold numbers are shown.
• I2. A recall-oriented candidate detector from UD trees: a clitic rule that splits a token into its own pronominal mention, and a construct-state rule that proposes the full chain and its embedded head. Write it in prag or world from the parse, and measure its recall on any gold mention set whose licence allows the use.
• I3. Document-grouped splits for every Hebrew test: group sentences by document before splitting a treebank, so that no document crosses train and test. This is the paper's partition rule, and it should hold for any treebanks/he/ evaluation.

For Mova

I1 is a reporting rule for tools/hbo and for 11. Modern Hebrew in Universal Dependencies. I2 goes into prag (mention candidates) with a test on treebanks/he/htb. I3 is a measurement rule for the split of every Hebrew treebank test. The finding that small encoders beat large decoders on Hebrew coreference supports the rule and small-model approach that Mova takes; it is a note for 7. Syntax, not a new feature.

Topics: Coreference resolution · Treebanks

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.488 · arXiv 2604.17108

@inproceedings{greenfeld2026beyond,
  title = {{Beyond Word Boundaries: A Hebrew Coreference Benchmark and an Evaluation Protocol for Morphologically Complex Text}},
  author = {Greenfeld, Refael Shaked and Tsarfaty, Reut},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.488},
  eprint = {2604.17108},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2026.acl-long.488}
}

References

29 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Aseel Al-Thubaity and Sarah Al-Dossari. 2017. A coreference resolution approach using morphological features in Arabic. In 2017 2nd International Conference on Anti-Cybercrime (ICACC), pages 123–128. IEEE. find
2. Amit Bagga and Breck Baldwin. 1998. Entity-based cross-document coreferencing using the vector space model. In 36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics, Volume 1, pages 79–85, Montreal, Quebec, Canada. Association for Computational Linguistics. find
3. David Bamman, Ted Underwood, and Noah A. Smith. 2019. A large-scale corpus of coreference in literary text. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 5746–5756, Hong Kong, China. Association for Computational Linguistics. find
4. Dan Bareket and Reut Tsarfaty. 2021. Neural modeling for named entities and morphology (NEMO2). Transactions of the Association for Computational Linguistics, 9:909–928. find our card
5. Ari Bornstein, Arie Cattan, and Ido Dagan. 2020. CoRefi: A crowd sourcing suite for coreference annotation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 205–215, Online. Association for Computational Linguistics. find
6. Arie Cattan, Alon Eirew, Gabriel Stanovsky, Mandar Joshi, and Ido Dagan. 2021. Realistic evaluation principles for cross-document coreference resolution. In Proceedings of *SEM 2021: The Tenth Joint Conference on Lexical and Computational Semantics, pages 143–151, Online. Association for Computational Linguistics. find
7. Hong Chen, Zhenhua Fan, Hao Lu, Alan Yuille, and Shu Rong. 2018. PreCo: A large-scale dataset in preschool vocabulary for coreference resolution. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 172–181, Brussels, Belgium. Association for Computational Linguistics. find our card
8. Amir DN Cohen, Hilla Merhav, Yoav Goldberg, and Reut Tsarfaty. 2025. Heq: a large and diverse hebrew reading comprehension benchmark. Preprint, arXiv:2508.01812. ↗ our card
9. Şeniz Demir and Hanifi İbrahim Akdağ. 2024. Mention detection in Turkish coreference resolution. Turkish Journal of Electrical Engineering and Computer Sciences, 32(5):682–697. find
10. Vladimir Dobrovolskii. 2021. Word-level coreference resolution. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7670–7675, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics. find
11. Yanai Elazar, Victoria Basmov, Yoav Goldberg, and Reut Tsarfaty. 2022. Text-based NP enrichment. Transactions of the Association for Computational Linguistics, 10:764–784. find
12. Mandar Joshi, Omer Levy, Luke Zettlemoyer, and Daniel Weld. 2019. BERT for coreference resolution: Baselines and analysis. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 5803–5808, Hong Kong, China. Association for Computational Linguistics. find
13. Amir More and Reut Tsarfaty. 2016. Data-driven morphological analysis and disambiguation for morphologically rich languages in the universal dependencies framework. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1481–1491. find
14. Shon Otmazgin, Arie Cattan, and Yoav Goldberg. 2023. LingMess: Linguistically informed multi expert scorers for coreference resolution. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2752– 2760, Dubrovnik, Croatia. Association for Computational Linguistics. find our card
15. Nghia T. Le and Alan Ritter. 2023. Are large language models robust coreference resolvers? Preprint, arXiv:2305.14489. ↗
16. Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Olga Uryupina, and Yuchen Zhang. 2012. CoNLL-2012 shared task: Modeling multilingual unrestricted coreference in OntoNotes. In Joint Conference on EMNLP and CoNLL -Shared Task, pages 1–40, Jeju Island, Korea. Association for Computational Linguistics. find our card
17. Kenton Lee, Luheng He, Mike Lewis, and Luke Zettlemoyer. 2017. End-to-end neural coreference resolution. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 188–197, Copenhagen, Denmark. Association for Computational Linguistics. ↗ our card
18. Shoval Sade, Amit Seker, and Reut Tsarfaty. 2018. The Hebrew Universal Dependency treebank: Past present and future. In Proceedings of the Second Workshop on Universal Dependencies (UDW 2018), pages 133–143, Brussels, Belgium. Association for Computational Linguistics. find our card
19. Xiaoqiang Luo. 2005. On coreference resolution performance metrics. In Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing, pages 25–32, Vancouver, British Columbia, Canada. Association for Computational Linguistics. find our card
20. Peter Schüller, Kübra Cıngıllı, Ferit Tunçer, Barış Gün Sürmeli, Ayşegül Pekel, Ayşe Hande Karatay, and Hacer Ezgi Karakaş. 2017. Marmara Turkish Coreference Corpus and Coreference Resolution Baseline. arXiv preprint arXiv:1706.01863. ↗
21. Mohamed Maamouri, Ann Bies, Tim Buckwalter, and Wigdan Mekki. 2004. The Penn Arabic Treebank: Building a large-scale annotated Arabic corpus. In NEMLAR conference on Arabic language resources and tools. find
22. Mohamed Maamouri, Ann Bies, Seth Kulick, Nizar Habash, and Owen Obeid. 2009. A conventional orthography for dialectal Arabic. In Proceedings of the LREC 2009 Workshop on Semitic Language Processing. find
23. Amit Seker, Elron Bandel, Dan Bareket, Idan Brusilovsky, Refael Greenfeld, and Reut Tsarfaty. 2022. AlephBERT: Language model pre-training and evaluation from sub-word to sentence level. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 46– 56, Dublin, Ireland. Association for Computational Linguistics. find our card
24. Avi Shmidman, Shaltiel Tzion, and Moshe Shmidman. 2023. DictaBERT: A state-of-the-art BERT suite for modern Hebrew. arXiv preprint arXiv:2308.16687. ↗ our card
25. Shaltiel Shmidman, Avi Shmidman, Amir DN Cohen, and Moshe Koppel. 2024. Adapting llms to hebrew: Unveiling dictalm 2.0 with enhanced vocabulary and instruction capabilities. Preprint, arXiv:2407.07080. ↗ our card
26. Ander Soraluze, Xabier Arregi, Olatz Arregi, and Xabier Artola. 2015. Adapting the Stanford Coreference Resolution System to Basque. In Proceedings of the 20th International Conference on Applications of Natural Language to Information Systems (NLDB 2015), pages 289– 296. find
27. Ander Soraluze, Xabier Arregi, and Xabier Artola. 2019. EUSKOR: An end-to-end coreference resolution system for Basque. PloS one, 14(9):e0221801. find
28. Marc Vilain, John Burger, John Aberdeen, Dennis Connolly, and Lynette Hirschman. 1995. A model-theoretic coreference scoring scheme. In Sixth Message Understanding Conference (MUC-6): Proceedings of a Conference Held in Columbia, Maryland, November 6-8, 1995. ↗ our card
29. Danit Yshaayahu Levi and Reut Tsarfaty. 2024. A truly joint neural architecture for segmentation and parsing. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), St. Julian’s, Malta. Association for Computational Linguistics. find our card

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/coref-2026-greenfeld-beyond-word-boundaries-hebrew-coreference/