PreCo: A Large-scale Dataset in Preschool Vocabulary for Coreference Resolution

What it is about

PreCo is 38K documents and 12.5M words of English reading-comprehension passages from Chinese middle- and high-school tests, chosen because their vocabulary (about 3,000 common words) resembles a preschooler's and because a small vocabulary raises the overlap between training and test, so that errors reflect coreference itself rather than unseen words. Annotation follows OntoNotes with changes: singletons are marked, generic mentions may corefer directly, non-proper noun modifiers are markable, and an event verb can corefer with a later noun («Sales [grew]… [The growth]…»). The development and test sets were annotated by two annotators and merged by a third; merging mainly raises recall, because single annotators miss distant links out of inattention. Inter-annotator CoNLL agreement is 85.3. A strong neural model reaches 81.5 F1 against an estimated human 87.9; with oracle mentions a model trained on 2.5K documents goes from 77.3 to 81.6, so most of the remaining gap is clustering.

Ideas

• I1. School-register evaluation for Mova's coreference: PreCo's dev/test passages are exactly the register of Mova's courses (simple vocabulary, reading-test stories). If the licence allows evaluation, run coref eval (SLM trees) and the frame reader's resolver on them and report per-pronoun precision; if not, use the paper's finding (clustering, not vocabulary, is the hard part in simple texts) to prioritise pronoun linking over mention detection for lessons.
• I2. Event-to-noun coreference, one of the «What next» items of coref (event mentions): link a definite deverbal noun to an earlier verb of the same root («grew… the growth», «decided… the decision», «arrived… her arrival»). The root pairs come from Mova's own derivation rules (en morphology, the suffix chapter), not from NOMLEX (research-only). Count candidate pairs on GUM dev and check them against GUM's verbal mentions before turning the rule on.
• I3. Copy the two-pass-and-merge procedure for teacher silver on coreference: two independent teacher passes over a text, merged by rule (union of links, conflicts to a third look), since single passes lose recall on distant links. Measure the recall of each pass against the merge on a few documents.

For Mova

35. Reference and coreference §3.4 (abstract and event anaphora), §9; 17-derivation-suffixes.md (deverbal nouns). Code: coref/src/mention.rs and coref/src/sieve.rs (event-noun link), the en derivation tables. Measures: per-pronoun precision on school-register texts; precision of event-noun links on GUM dev.

Topics: Coreference resolution

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/d18-1016 · arXiv 1810.09807

@inproceedings{chen2018preco,
  title = {{PreCo: A Large-scale Dataset in Preschool Vocabulary for Coreference Resolution}},
  author = {Chen, Hong and Fan, Zhenhua and Lu, Hao and Yuille, Alan and Rong, Shu},
  booktitle = {Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing},
  year = {2018},
  doi = {10.18653/v1/d18-1016},
  eprint = {1810.09807},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/d18-1016}
}

References

10 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Steven Bird, Edward Loper, and Ewan Klein. 2009. Natural Language Processing with Python. O’Reilly Media Inc. find
2. OntoNotes Guidelines. 2007. Ontonotes english coreference guidelines. find
3. M. E. Peters, M. Neumann, M. Iyyer, M. Gardner, C. Clark, K. Lee, and L. Zettlemoyer. 2018. Deep contextualized word representations. ArXiv e-prints. find
4. Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Olga Uryupina, and Yuchen Zhang. 2012. CoNLL-2012 shared task: Modeling multilingual unrestricted coreference in OntoNotes. In Proceedings of the Sixteenth Conference on Computational Natural Language Learning (CoNLL 2012), Jeju, Korea. find our card
5. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. Squad: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250. ↗
6. Nafise Sadat Moosavi and Michael Strube. 2017. Lexical features in coreference resolution: To be used with caution. find our card
7. Marc Vilain, John Burger, John Aberdeen, Dennis Connolly, and Lynette Hirschman. 1995. A model-theoretic coreference scoring scheme. In Proceedings of the 6th Conference on Message Understanding, MUC6 ’95, pages 45–52, Stroudsburg, PA, USA. Association for Computational Linguistics. find our card
8. Ralph Weischedel, Eduard Hovy, Mitchell Marcus, Martha Palmer, Robert Belvin, Sameer Pradhan, Lance Ramshaw, and Nianwen Xue. 2011. Ontonotes: A large training corpus for enhanced processing. Handbook of Natural Language Processing and Machine Translation. Springer. find
9. Wikipedia. 2018. Vocabulary development. [Online; accessed 18-April-2018]. find
10. Sam Wiseman, Alexander M. Rush, and Stuart M. Shieber. 2016. Learning global features for coreference resolution. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 994–1004. Association for Computational Linguistics. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/chen-2018-preco-large-scale-dataset-preschool/