Using Linguistic Features to Improve the Generalization Capability of Neural Coreference Resolvers

What it is about

Neural coreference resolvers that win on CoNLL-2012 lose their ranking on WikiCoref, which uses the same guidelines. The authors add classic linguistic features to deep-coref (mention type and fine type, named-entity type, gender, number, animacy, dependency relation, POS; pairwise agreement, string match, compatibility of pre- and post-modifiers, salience-based discourse features combining distance and subject/object role). Added wholesale they help little. Selecting only the informative (feature, value) combinations helps a lot: their EPM algorithm stores mention-pair samples in a frequent-pattern tree and keeps patterns that pass a G² likelihood-ratio test (p < 0.01), add information over their sub-patterns, and are frequent enough. The model with these patterns gains in-domain and gives the best out-of-domain results reported at the time on WikiCoref and other sets, on par with a system built specifically for WikiCoref.

Ideas

• I1. Use EPM-style mining to choose feature conjunctions for prag's perceptron, which prag — Mova pragmatics: LLM silver and a transparent SLM lists as a next step (pairs such as aux1×sperson, rform×you): enumerate slot=value pairs and triples over the silver, keep those whose association with a label passes a G² test and adds over their parts, and add only these as conjunction features. Measure act and indirect macro-F1 in prag eval with the shuffled-label control; the mined list itself is an explanation of what the model may use.
• I2. For coreference, mine discriminative patterns over mention-pair features (mention type, agreement, Hobbs rank, sieve that fired, distance bucket) to find conjunctions that predict wrong links of the current sieves, e.g. «it + candidate is a time/number + distance > 1 sentence». Mining must run on licence-clean data or the teacher's silver; on GUM it may only diagnose, not supply rules (GUM is NC and dev-only). Turn each pattern that is linguistically sensible into a hand-written blocker in coref/src/sieve.rs and measure its precision on GUM dev.
• I3. Prefer the paper's four carefully built pairwise features (agreement, modifier compatibility, salience-weighted distance) over raw lexical ones in any future ranker; they carry across domains.

For Mova

35. Reference and coreference §8–9; 31-speech-acts.md §9 (act features). Code: prag/src/feats.rs and prag/src/model.rs (mined conjunctions), coref/src/sieve.rs (blockers). Measures: prag eval macro-F1; sieve precision and CoNLL F1 on GUM dev.

Topics: Coreference resolution

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/d18-1018 · arXiv 1708.00160

@inproceedings{moosavi2018using,
  title = {{Using Linguistic Features to Improve the Generalization Capability of Neural Coreference Resolvers}},
  author = {Moosavi, Nafise Sadat and Strube, Michael},
  booktitle = {Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing},
  year = {2018},
  doi = {10.18653/v1/d18-1018},
  eprint = {1708.00160},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/d18-1018}
}

References

41 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Micha Elsner and Eugene Charniak. 2008. Coreference-inspired coherence modeling. In Proceedings ACL-HLT 2008 Conference Short Papers, Columbus, Ohio, 15–20 June 2008, pages 41–44. find
2. Alan Agresti. 2007. An Introduction to Categorical Data Analysis. John Wiley & Sons. find
3. Amit Bagga and Breck Baldwin. 1998. Algorithms for scoring coreference chains. In Proceedings of the 1st International Conference on Language Resources and Evaluation, Granada, Spain, 28–30 May 1998, pages 563–566. find our card
4. Rong-En Fan, Kai-Wei Chang, Cho-Jui Hsieh, Xiang-Rui Wang, and Chih-Jen Lin. 2008. LIBLINEAR: A library for large linear classification. The Journal of Machine Learning Research, 9:1871–1874. find
5. Iyad Batal and Milos Hauskrecht. 2010. Constructing classification features using minimal predictive patterns. In Proceedings of the 19th ACM International Conference on Information and Knowledge Management, pages 869–878. find
6. Eraldo R Fernandes and Ruy L Milidiú. 2012. Entropy-guided feature generation for structured learning of Portuguese dependency parsing. In Proceedings of the International Conference on Computational Processing of the Portuguese Language, pages 146–156. Springer. find
7. Eric Bengtson and Dan Roth. 2008. Understanding the value of features for coreference resolution. In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, Waikiki, Honolulu, Hawaii, 25–27 October 2008, pages 294–303. find
8. Eraldo Rezende Fernandes, Cı́cero Nogueira dos Santos, and Ruy Luiz Milidiú. 2012. Latent structure perceptron with feature induction for unrestricted coreference resolution. In Proceedings of the Shared Task of the 16th Conference on Computational Natural Language Learning, Jeju Island, Korea, 12–14 July 2012, pages 41–48. find
9. Anders Björkelund and Richárd Farkas. 2012. Data-driven multilingual coreference resolution using resolver stacking. In Proceedings of the Shared Task of the 16th Conference on Computational Natural Language Learning, Jeju Island, Korea, 12–14 July 2012, pages 49–55. find
10. Abbas Ghaddar and Philippe Langlais. 2016a. Coreference in Wikipedia: Main concept resolution. In Proceedings of the 20th Conference on Computational Natural Language Learning, Berlin, Germany, 7–11 August 2016, pages 229–238. find
11. Hong Cheng, Xifeng Yan, Jiawei Han, and Chih-Wei Hsu. 2007. Discriminative frequent pattern analysis for effective classification. In Proceedings of the IEEE 23rd International Conference on Data Engineering (ICDE 2007), pages 716–725. find
12. Abbas Ghaddar and Philippe Langlais. 2016b. Wiki-Coref: An English coreference-annotated corpus of Wikipedia articles. In Proceedings of the 10th International Conference on Language Resources and Evaluation, Portorož, Slovenia, 23–28 May 2016. find our card
13. Hong Cheng, Xifeng Yan, Jiawei Han, and Philip S Yu. 2008. Direct discriminative pattern mining for effective classification. In Proceedings of the IEEE 24th International Conference on Data Engineering (ICDE 2008), pages 169–178. find
14. Jiawei Han, Jian Pei, Yiwen Yin, and Runying Mao. 2004. Mining frequent patterns without candidate generation: A frequent-pattern tree approach. Data Mining and Knowledge Discovery, 8(1-5):53–87. find
15. Kevin Clark and Christopher D. Manning. 2016a. Improving coreference resolution by learning entity-level distributed representations. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Berlin, Germany, 7–12 August 2016. find
16. Mirella Lapata and Regina Barzilay. 2005. Automatic evaluation of text coherence: Models and representations. In Proceedings of the 19th International Joint Conference on Artificial Intelligence, Edinburgh, Scotland, 30 July – 5 August 2005, pages 1085–1090. find
17. Kevin Clark and Christopher D. Manning. 2016b. Deep reinforcement learning for mention-ranking coreference models. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, Tex., 1–5 November 2016, pages 2256–2262. find
18. Kenton Lee, Luheng He, Mike Lewis, and Luke Zettlemoyer. 2017. End-to-end neural coreference resolution. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 188–197, Copenhagen, Denmark. find our card
19. Pascal Denis and Jason Baldridge. 2008. Specialized models and ranking for coreference resolution. In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, Waikiki, Honolulu, Hawaii, 25–27 October 2008, pages 660– 669. find
20. M. Lichman. 2013. UCI machine learning repository. find
21. Xiaoqiang Luo. 2005. On coreference resolution performance metrics. In Proceedings of the Human Language Technology Conference and the 2005 Conference on Empirical Methods in Natural Language Processing, Vancouver, B.C., Canada, 6–8 October 2005, pages 25–32. find our card
22. Ted Dunning. 1993. Accurate methods for the statistics of surprise and coincidence. Computational Linguistics, 19(1):61–74. find
23. Marc Segond and Christian Borgelt. 2011. Item set mining based on cover similarity. Advances in Knowledge Discovery and Data Mining, pages 493– 505. find
24. Christopher D. Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven J. Bethard, and David Mc-Closky. 2014. The Stanford CoreNLP natural language processing toolkit. In Association for Computational Linguistics (ACL) System Demonstrations, pages 55–60. find
25. Richard S. Sutton and Andrew G. Barto. 1998. Reinforcement learning: An introduction, volume 1. MIT press Cambridge. find
26. Nafise Sadat Moosavi and Michael Strube. 2016. Which coreference evaluation metric do you trust? A proposal for a link-based entity aware metric. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Berlin, Germany, 7–12 August 2016, pages 632–642. find our card
27. Olga Uryupina. 2007. Knowledge acquisition for coreference resolution. Ph.D. thesis, Saarland University. find
28. Olga Uryupina and Alessandro Moschitti. 2015. A state-of-the-art mention-pair model for coreference resolution. In Proceedings of STARSEM 2015: The Fourth Joint Conference on Lexical and Computational Semantics, Denver, Col., 4–5 June 2015, pages 289–298. find
29. Nafise Sadat Moosavi and Michael Strube. 2017a. Lexical features in coreference resolution: To be used with caution. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), Vancouver, B.C., Canada, 30 July –4 August 2017. find our card
30. Marc Vilain, John Burger, John Aberdeen, Dennis Connolly, and Lynette Hirschman. 1995. A model-theoretic coreference scoring scheme. In Proceedings of the 6th Message Understanding Conference (MUC-6), pages 45–52, San Mateo, Cal. Morgan Kaufmann. find our card
31. Nafise Sadat Moosavi and Michael Strube. 2017b. Use generalized representations, but do not forget surface features. In Proceedings of the 2nd Workshop on Coreference Resolution Beyond OntoNotes (CORBON 2017), pages 1–7, Valencia, Spain. find
32. Geoffrey I. Webb. 2006. Discovering significant patterns. Machine Learning, 68(1):1–39. find
33. Vincent Ng and Claire Cardie. 2002. Improving machine learning approaches to coreference resolution. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, Philadelphia, Penn., 7–12 July 2002, pages 104–111. find
34. Eric W. Noreen. 1989. Computer Intensive Methods for Hypothesis Testing: An Introduction. Wiley, New York, N.Y. find
35. Sam Wiseman, Alexander M. Rush, Stuart Shieber, and Jason Weston. 2015. Learning anaphoricity and antecedent ranking features for coreference resolution. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Beijing, China, 26–31 July 2015, pages 1416–1426. find
36. Simone Paolo Ponzetto and Michael Strube. 2006. Exploiting semantic role labeling, WordNet and Wikipedia for coreference resolution. In Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics, New York, N.Y., 4–9 June 2006, pages 192–199. find
37. Xiaofeng Yang, Jian Su, Guodung Zhou, and Chew Lim Tan. 2004. Improving pronoun resolution by incorporating coreferential information of candidates. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics, Barcelona, Spain, 21–26 July 2004, pages 128– 135. find
38. Sameer Pradhan, Xiaoqiang Luo, Marta Recasens, Eduard Hovy, Vincent Ng, and Michael Strube. 2014. Scoring coreference partitions of predicted mentions: A reference implementation. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), Baltimore, Md., 22–27 June 2014, pages 30– 35. find our card
39. Dian Yu and Heng Ji. 2016. Unsupervised person slot filling based on graph mining. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Berlin, Germany, 7–12 August 2016, pages 44–53. find
40. Marta Recasens and Eduard Hovy. 2009. A deeper look into features for coreference resolution. In Proceedings of the 7th Discourse Anaphora and Anaphor Resolution, pages 29–42. find
41. Sebastian Schuster and Christopher D. Manning. 2016. Enhanced English universal dependencies: An improved representation for natural language understanding tasks. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016), Paris, France. find our card

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/moosavi-2018-linguistic-features-improve-generalization-capability/