A Machine Learning Approach to Coreference Resolution of Noun Phrases

What it is about

The paper presents a learned, end-to-end coreference resolver for general noun phrases (not only pronouns), trained on a small annotated corpus. A pipeline of HMM-based modules first finds the markables: tokenisation, POS tagging, NP identification, named entities and nested NPs. Each pair of markables is described by 12 shallow features: distance, pronoun flags, string match, definiteness, demonstratives, number, semantic class, gender, both-proper-names, alias, and apposition. A C5 decision tree classifies pairs, and closest-first clustering builds the chains. On MUC-6 the system reaches recall 58.6%, precision 67.3% and F 62.6. On MUC-7 it reaches recall 56.1%, precision 65.5% and F 60.4. Only 6.5% of the 20,910 MUC-6 training pairs are positive. The three features alias, string match and apposition alone come within 2.3 F points (MUC-6) and 1 point (MUC-7) of the full system.

Ideas

• I1. Run a feature ablation in the way the paper's tables do: remove each of the 12 features in turn on the same split, and report the F change. A feature that moves nothing is a candidate for removal before any new feature is added (coref/src/eval.rs).
• I2. Audit the error classes of section 5 (modifier string match, same string but different referents, apposition errors, alias errors) as counts per class on a dev set. Each class has a rule that can be tested, so this works as a negative-control report.
• I3. Replace the learned tree with a deterministic decision list. The tree in the paper's Figure 2 is short enough to print, and each link can then carry the rule that produced it.
• I4. Keep "unknown" as a third value for gender and semantic agreement, as the paper does, so that missing knowledge is never read as disagreement.

For Mova

The code is coref/src/sieve.rs (the pairwise features), coref/src/mention.rs (markables) and coref/src/eval.rs. The chapters are 35. Reference and coreference and, for number, 11. Nouns. The measure is MUC F on the same dev and test splits, with the ablation above. Run every configuration twice and check that the results match exactly.

Topics: Coreference resolution

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.1162/089120101753342653

@article{soon2001machine,
  title = {{A Machine Learning Approach to Coreference Resolution of Noun Phrases}},
  author = {Soon, Wee Meng and Ng, Hwee Tou and Lim, Daniel Chung Yong},
  journal = {Computational Linguistics},
  year = {2001},
  doi = {10.1162/089120101753342653},
  url = {https://doi.org/10.1162/089120101753342653}
}

References

21 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Aone, Chinatsu and Scott William Bennett. 1995. Evaluating automated and manual acquisition of anaphora resolution strategies. In Proceedings of the 33rd Annual Meeting of the Association for Computational Linguistics, 122-129. find
2. Baldwin, Breck. 1997. CogNIAC: High precision coreference with limited knowledge and linguistic resources. In Proceedings of the ACL Workshop on Operational Factors in Practical, Robust Anaphora Resolution for Unrestricted Texts, pages 38-45. find our card
3. Bikel, Daniel M., Richard Schwartz, and Ralph M. Weischedel. 1999. An algorithm that learns what's in a name. Machine Learning, 34(1-3):211-231, February. find
4. Cardie, Claire and Kiri Wagstaff. 1999. Noun phrase coreference as clustering. In Proceedings of the 1999 Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora, pages 82-89. find
5. Chinchor, Nancy A. 1998. Overview of MUC-7/MET-2. In Proceedings of the Seventh Message Understanding Conference (MUC-7). http://www.itl.nist.gov/iad/894.02/related_projects/muc/proceedings/muc_7_toc.html. ↗
6. Church, Kenneth. 1988. A stochastic parts program and noun phrase parser for unrestricted text. In Proceedings of the Second Conference on Applied Natural Language Processing, pages 136-143. find
7. Dietterich, Thomas G. 1998. Approximate statistical tests for comparing supervised classification learning algorithms. Neural Computation, 10(7):1895-1924, October. find
8. Fisher, David, Stephen Soderland, Joseph McCarthy, Fangfang Feng, and Wendy Lehnert. 1995. Description of the UMass system as used for MUC-6. In Proceedings of the Sixth Message Understanding Conference (MUC-6), pages 127-140. find
9. Ge, Niyu, John Hale, and Eugene Charniak. 1998. A statistical approach to anaphora resolution. In Proceedings of the Sixth Workshop on Very Large Corpora, pages 161-170. find
10. Kameyama, Megumi. 1997. Recognizing referential links: An information extraction perspective. In Proceedings of the ACL Workshop on Operational Factors in Practical, Robust Anaphora Resolution for Unrestricted Texts, pages 46-53. find
11. Kehler, Andrew. 1997. Probabilistic coreference in information extraction. In Proceedings of the Second Conference on Empirical Methods in Natural Language Processing, pages 163-173. find
12. Lappin, Shalom and Herbert J. Leass. 1994. An algorithm for pronominal anaphora resolution. Computational Linguistics, 20(4):535-561, December. find our card
13. McCarthy, Joseph F. 1996. A Trainable Approach to Coreference Resolution for Information Extraction. Ph.D. thesis, University of Massachusetts Amherst, Department of Computer Science, September. find
14. McCarthy, Joseph F. and Wendy Lehnert. 1995. Using decision trees for coreference resolution. In Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence, pages 1050-1055. find
15. McEnery, A., I. Tanaka, and S. Botley. 1997. Corpus annotation and reference resolution. In Proceedings of the ACL Workshop on Operational Factors in Practical, Robust Anaphora Resolution for Unrestricted Texts, pages 67-74. find
16. Miller, George A. 1990. WordNet: An on-line lexical database. International Journal of Lexicography, 3(4):235-312. find
17. Mitkov, Ruslan. 1997. Factors in anaphora resolution: They are not the only things that matter. A case study based on two different approaches. In Proceedings of the ACL Workshop on Operational Factors in Practical, Robust Anaphora Resolution for Unrestricted Texts, pages 14-21. find
18. MUC-6. 1995. Coreference task definition (v2.3, 8 Sep 95). In Proceedings of the Sixth Message Understanding Conference (MUC-6), pages 335-344. find
19. MUC-7. 1997. Coreference task definition (v3.0, 13 Jul 97). In Proceedings of the Seventh Message Understanding Conference (MUC-7). find
20. Quinlan, John Ross. 1993. C4.5: Programs for Machine Learning. Morgan Kaufmann, San Francisco, CA. find our card
21. Sundheim, Beth M. 1995. Overview of results of the MUC-6 evaluation. In Proceedings of the Sixth Message Understanding Conference (MUC-6), pages 13-31. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/soon-2001-machine-learning-approach-coreference-resolution/