ARRAU: Linguistically-Motivated Annotation of Anaphoric Descriptions

What it is about

The paper presents the second release of ARRAU, a multi-domain corpus for anaphora. Its domains are news (from the RST Discourse Treebank), task-oriented dialogue (TRAINS), fiction (PEAR) and medical leaflets (GNOME). Release 2 has 552 documents, 348,072 tokens and 99,582 markables. Release 1 had 264 documents, 184,748 tokens and 60,884 markables. Each markable has a minimal span (the head or bare name) and a maximal span (the full NP). The annotation also marks referentiality and genericity, with sub-types such as generic, quantifier, expletive, idiom, coordination and predicate. A consistency pass was applied: non-referential markables may not be in a coreference chain; chains are checked for number and gender clashes and for semantic type; brackets may not cross or cross sentence boundaries. Non-referential and singleton markables make up up to a third of all markables. The paper gives no inter-annotator or coreference scores.

Ideas

• I1. Give each Mova mention a minimal span (the head) and a maximal span (the full NP) in coref/src/mention.rs. Score lenient matches inside the maximal span, and measure how many dev errors become lenient matches.
• I2. Turn the consistency rules into code: a chain may not mix non-referential markables, and may not mix genders. Count violations in the coref output before and after each sieve change. A random chain assignment is the negative control, and it must show many violations.
• I3. Use referentiality as a first filter: a non-referential markable never becomes an anaphor. Keep the sub-types (expletive, idiom, quantifier, generic) in a table, so that each filtered mention has a reason.
• I4. Report coreference per domain (news, dialogue, fiction, medical), so that a drop in one domain does not disappear in the average.

For Mova

The code is coref/src/mention.rs, coref/src/sieve.rs and coref/src/eval.rs. The chapters are 35. Reference and coreference and 37. Register and variation. The corpus is non-commercial, so it is for evaluation only and is never built into a release. The measure is per-domain coreference F1 on an evaluation split, run twice.

Topics: Coreference resolution

Cite

The paper itself, not this card. BibTeX from the library's record of the work.

@inproceedings{uryupina2016arrau,
  title = {{ARRAU: Linguistically-Motivated Annotation of Anaphoric Descriptions}},
  author = {Uryupina, Olga and Artstein, Ron and Bristot, Antonella and Cavicchio, Federica and Rodriguez, Kepa and Poesio, Massimo},
  booktitle = {Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)},
  year = {2016},
  url = {https://aclanthology.org/L16-1326/}
}

References

10 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Lynn Carlson, Daniel Marcu, and Mary Ellen Okurowski. 2002. Rst discourse treebank ldc2002t07. find
2. George Doddington, Alexis Mitchell, Mark Przybocki, Lance Ramshaw, Stephanie Strassell, and Ralph Weischedel. 2004. The automatic content extraction (ACE) program–tasks, data, and evaluation. In Proceedings of the Language Resources and Evaluation Conference. find
3. Jonathan K Kummerfeld, Mohit Bansal, David Burkett, and Dan Klein. 2011. Mention detection: Heuristics for the ontonotes annotations. In Conference on Natural Language Learning, pages 102–106. Association for Computational Linguistics. find
4. Cristoph Müller and Michael Strube. 2006. Multi-level annotation of linguistic data with MMAX2. In Corpus Technology and Language Pedagogy, pages 197–214. S. Braun and K. Kohn and J. Mukherjee. find
5. Haoruo Peng, Kai-Wei Chang, and Dan Roth. 2015. A joint framework for coreference resolution and mention head detection. CoNLL 2015, 51:12. find
6. Massimo Poesio and Ron Artstein. 2008. Anaphoric annotation in the arrau corpus. In Proceedings of the Language Resources and Evaluation Conference. find
7. Sameer Pradhan, Lance Ramshaw, Mitchell Marcus, Martha Palmer, Ralph Weischedel, and Nianwen Xue. 2011. Conll-2011 shared task: Modeling unrestricted coreference in ontonotes. In Proceedings of the Fifteenth Conference on Computational Natural Language Learning (CoNLL 2011), Portland, Oregon, June. find our card
8. Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Olga Uryupina, and Yuchen Zhang. 2012. CoNLL-2012 shared task: Modeling multilingual unrestricted coreference in OntoNotes. In Proceedings of the Sixteenth Conference on Computational Natural Language Learning (CoNLL 2012), Jeju, Korea. find our card
9. Marta Recasens, Lluı́s Màrquez, Emili Sapena, M.Antònia Martı́, Mariona Taulé, Véronique Hoste, Massimo Poesio, and Yannick Versley. 2010. SemEval-2010 Task 1: Coreference resolution in multiple languages. In Proceedings of the 5th International Workshop on Semantic Evaluations (SemEval-2010), Uppsala, Sweden. find
10. Desislava Zhekova and Sandra Kübler. 2013. Machine learning for mention head detection in multilingual coreference resolution. In RANLP, pages 747–754. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/uryupina-2016-arrau-linguistically-motivated-annotation-anaphoric/