End-to-end Neural Coreference Resolution

What it is about

The first coreference model trained end to end without a parser or a mention detector. Every span up to 10 words is a candidate; each span gets a mention score and each pair an antecedent score, and a span chooses among all earlier spans plus a dummy antecedent fixed at score 0 (no link, or not a mention). Span vectors combine BiLSTM states at the boundaries with an attention-weighted «soft head» that learns something close to syntactic heads; pair features are same-speaker, genre, binned distance and span width. Spans are pruned greedily by mention score (0.4 per word, up to 250 antecedents) and the model maximises the marginal likelihood of all gold antecedents. On CoNLL-2012 (OntoNotes) it reaches 67.2 average F1 single and 68.8 as a five-model ensemble, +1.5/+3.1 over the previous best; distance and width features are worth 3.8 F1, and gold mentions would add over 17.5 F1, while a rule-based mention detector costs 1 F1.

Ideas

I1. Measure how much of coref's error is mention detection: add an oracle-mention switch to coref eval (gold mention spans, Mova's sieves for linking) and report CoNLL F1 on GUM dev with gold and SLM trees. The paper's +17.5 F1 for gold mentions sets the expectation; the gap tells whether to work on coref/src/mention.rs or on the sieves.
I2. When coref moves from sieves to candidate ranking (README «What next»), take this model's decision shape without the network: for each mention, score every earlier candidate with an averaged perceptron over symbolic features (head match, agreement, Hobbs rank, same voice/speaker, the paper's distance buckets 1, 2, 3, 4, 5–7, 8–15, 16–31, 32–63, 64+ in sentences or mentions, mention width), with a dummy «new entity» candidate fixed at 0 so the model abstains when nothing scores above it. Training data must be licence-clean (GUM is NC: dev/test only). The explanation is the feature contributions of the chosen link over the dummy.
I3. Keep the sieve as a precision layer in front: links the speaker, appos and reflexive sieves make at 73–91% precision stay fixed, and the ranker only decides what the sieves leave open. Compare CoNLL F1 on GUM dev for sieve-only, ranker-only and the combination.

For Mova

35. Reference and coreference §8–9 (algorithms; Mova's coref crate). Code: coref/src/eval.rs (oracle mentions), coref/src/mention.rs, coref/src/sieve.rs, a new ranker next to it. Measures: CoNLL F1 on GUM dev with gold and SLM trees, with and without oracle mentions.

Topics: Coreference resolution

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/d17-1018 · arXiv 1707.07045

@inproceedings{lee2017end,
  title = {{End-to-end Neural Coreference Resolution}},
  author = {Lee, Kenton and He, Luheng and Lewis, Mike and Zettlemoyer, Luke},
  booktitle = {Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing},
  year = {2017},
  doi = {10.18653/v1/d17-1018},
  eprint = {1707.07045},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/d17-1018}
}

References

1 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

Martın Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, et al. 2015. TensorFlow: Large-scale Machine Learning on Heterogeneous Systems. Software available from tensorflow.org. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/lee-2017-end-end-neural-coreference-resolution/