Mova0.5 pre-release⌕
☰
home / docs / papers / English / A Mention-Ranking Model for Abstract Anaphora Resolution

A Mention-Ranking Model for Abstract Anaphora Resolution

What it is about

Abstract anaphors (this trend, this, that, it referring to a proposition, fact or event) have almost no annotated training data. The authors make it artificially: in a parsed corpus they find a verb with an embedded clause (a sentential complement such as doubts (that) S, or an adverbial clause with a conjunction such as because, after, although), cut the clause off as the antecedent, and replace it by a fitting anaphor (this/that/it; because of that, therefore for causal clauses), which yields an anaphoric sentence. A mention-ranking model with a siamese BiLSTM scores every syntactic constituent of the context as a candidate antecedent against the anaphoric sentence. It improves the state of the art on shell-noun resolution and gives the first results on the abstract anaphors of ARRAU, well above a «previous sentence» baseline and a random-clause baseline, though far from solved; candidate constituent type (S, VP, SBAR) is an important signal.

Ideas

For Mova

35. Reference and coreference §3.4 (abstract anaphora). Code: a generator next to coref/src/eval.rs, coref/src/sieve.rs, prag/src/frames.rs. Measures: success@1 on the artificial set against the baselines; recovered discourse-deixis links on GUM dev.

Topics: Coreference resolution

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/d17-1021

@inproceedings{marasovic2017mention,
  title = {{A Mention-Ranking Model for Abstract Anaphora Resolution}},
  author = {Marasović, Ana and Born, Leo and Opitz, Juri and Frank, Anette},
  booktitle = {Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing},
  year = {2017},
  doi = {10.18653/v1/d17-1021},
  url = {https://doi.org/10.18653/v1/d17-1021}
}

References

45 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Kevin Clark and Christopher D. Manning. 2016b. Improving coreference resolution by learning entity-level distributed representations. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL), Berling, Germany. find
  2. Martı́n Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dan Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. 2015. TensorFlow: Large-scale machine learning on heterogeneous systems. Software available from tensorflow.org. find
  3. Djork-Arné Clevert, Thomas Unterthiner, and Sepp Hochreiter. 2016. Fast and accurate deep network learning by exponential linear units (elus). In Proceedings of the 4th International Conference on Learning Representations (ICLR), San Juan, Puerto Rico. find
  4. Arpita Das, Harish Yenala, Manoj Kumar Chinnakotla, and Manish Shrivastava. 2016. Together we stand: Siamese networks for similar question retrieval. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL), Berling, Germany. find
  5. Pranav Anand and Daniel Hardt. 2016. Antecedent selection for sluicing: Structure and content. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1234– 1243, Austin, Texas. find our card
  6. Stefanie Dipper and Heike Zinsmeister. 2012. Annotating Abstract Anaphora. Language Resources and Evaluation, 46(1):37–52. find
  7. Nicholas Asher. 1993. Reference to Abstract Objects in Discourse. Kluwer Academic Publishers. find
  8. Miriam Eckert and Michael Strube. 2000. Dialogue acts, synchronising units and anaphora resolution. find
  9. James Bergstra, Rémi Bardenet, Yoshua Bengio, and Balázs Kégl. 2011. Algorithms for hyper-parameter optimization. In Proceedings of the 35th Annual Conference on Neural Information Processing Systems (NIPS), Granada, Spain. find
  10. Alex Graves and Jürgen Schmidhuber. 2005. Frame-wise Phoneme Classification With Bidirectional LSTM And Other Neural Network Architectures. Neural Networks, 18:602–610. find
  11. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In 2015 IEEE International Conference on Computer Vision (ICCV). find
  12. Samuel R. Bowman, Jon Gauthier, Abhinav Rastogi, Raghav Gupta, Christopher D. Manning, and Christopher Potts. 2016. A fast unified model for parsing and sentence understanding. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1466–1477, Berlin, Germany. find
  13. Mikael Henaff, Arthur Szlam, and Yann LeCun. 2016. Recurrent orthogonal networks and long-memory tasks. In Proceedings of the the 33rd International Conference on Machine Learning (ICML), New York City, USA. find
  14. Donna K. Byron. 2004. Resolving pronominal reference to abstract entities. Ph.D. thesis, University of Rochester, Rochester, New York. find
  15. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735–1780. find
  16. Sumit Chopra, Raia Hadsell, and Yann LeCun. 2005. Learning a similarity metric discriminatively, with application to face verification. In Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on, volume 1, pages 539–546. IEEE. find
  17. Sujay Kumar Jauhar, Raul Guerra, Edgar Gonzàlez Pellicer, and Marta Recasens. 2015. Resolving discourse-deictic pronouns: A two-stage approach to do it. In Proceedings of the Fourth Joint Conference on Lexical and Computational Semantics, pages 299–308, Denver, Colorado. find
  18. Kevin Clark and Christopher D. Manning. 2015. Entity-centric coreference resolution with model stacking. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (ACL), Beijing, China. find
  19. Thorsten Joachims. 2002. Optimizing search engines using clickthrough data. In Proceedings of the 8th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 133–142. find
  20. Rafal Józefowicz, Wojciech Zaremba, and Ilya Sutskever. 2015. An empirical exploration of recurrent network architectures. In Proceedings of the 32nd International Conference on Machine Learning (ICML), Lille, France. find
  21. Kevin Clark and Christopher D. Manning. 2016a. Deep reinforcement learning for mention-ranking coreference models. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP), Austin, Texas. find
  22. Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. Glove: Global vectors for word representation. In Empirical Methods in Natural Language Processing (EMNLP), pages 1532– 1543. find
  23. Diederik Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In Proceedings of the 3rd International Conference on Learning Representations (ICLR), San Diego, USA. find
  24. Dan Klein and Christopher D Manning. 2003. Accurate unlexicalized parsing. In Proceedings of the 41st Annual Meeting on Association for Computational Linguistics-Volume 1, pages 423–430. Association for Computational Linguistics. find our card
  25. Massimo Poesio and Ron Artstein. 2008. Anaphoric Annotation in the ARRAU Corpus. In Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC’08), Marrakech, Morocco. find
  26. Varada Kolhatkar and Graeme Hirst. 2014. Resolving shell nouns. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 499–510, Doha, Qatar. find
  27. Dheeraj Rajagopal, Eduard Hovy, and Teruko Mitamura. 2016. Unsupervised event coreference for abstract words. In Proceedings of EMNLP 2016 Workshop on Uphill Battles in Language Processing: Scaling Early Achievements to Robust Methods, pages 22–26, Austin, Texas. find
  28. Varada Kolhatkar, Heike Zinsmeister, and Graeme Hirst. 2013a. Annotating anaphoric shell nouns with their antecedents. In Proceedings of the 7th Linguistic Annotation Workshop and Interoperability with Discourse, pages 112–121, Sofia, Bulgaria. find
  29. Attapol T. Rutherford, Vera Demberg, and Nianwen Xue. 2017. A Systematic Study of Neural Discourse Models for Implicit Discourse Relation. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics. find
  30. Varada Kolhatkar, Heike Zinsmeister, and Graeme Hirst. 2013b. Interpreting anaphoric shell nouns using antecedents of cataphoric shell nouns as training data. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 300–310, Seattle, Washington, USA. find
  31. Nitish Srivastava, Geoffrey E. Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15:1929–1958. find
  32. Jing Lu and Vincent Ng. 2016. Event Coreference Resolution with Multi-Pass Sieves. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC), pages 3996–4003, Portoroz. find
  33. Manfred Stede and Yulia Grishina. 2016. Anaphoricity in Connectives: A Case Study on German. In Proceedings of the Coreference Resolution Beyond OntoNotes (CORBON) Workshop, San Diego, California. Laurens v.d. find
  34. Maaten and Geoffrey Hinton. 2008. Visualizing data using t-sne. Journal of Machine Learning Research, 9(2579-2605):85. find
  35. Michael Strube and Christoph Müller. 2003. A machine learning approach to pronoun resolution in spoken dialogue. In Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics, pages 168–175, Sapporo, Japan. find
  36. Mitchell P Marcus, Mary Ann Marcinkiewicz, and Beatrice Santorini. 1993. Building a large annotated corpus of english: The penn treebank. Computational linguistics, 19(2):313–330. find our card
  37. Kai Sheng Tai, Richard Socher, and Christopher D. Manning. 2015. Improved semantic representations from tree-structured long short-term memory networks. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (ACL), Beijing, China. find
  38. Jonas Mueller and Aditya Thyagarajan. 2016. Siamese recurrent architectures for learning sentence similarity. In Proceedings of the 13th Conference on Artificial Intelligence (AAAI), pages 2786–2792, Phoenix, Arizona. find
  39. Christoph Müller. 2008. Fully Automatic Resolution of It, This and That in Unrestricted Multi-Party Dialog. Ph.D. thesis, Universität Tübingen, Tübingen. find
  40. Olga Uryupina, Ron Artstein, Antonella Bristot, Federica Cavicchio, Kepa J Rodriguez, and Massimo Poesio. 2016. ARRAU: Linguistically-Motivated Annotation of Anaphoric Descriptions. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC), pages 2058–2062, Portoroz. find our card
  41. Paul Neculoiu, Maarten Versteegh, and Mihai Rotaru. 2016. Learning Text Similarity with Siamese Recurrent Networks. In Proceedings of the 1st Workshop on Representation Learning for NLP, pages 148– 157, Berlin, Germany. find
  42. Bonnie Lynn Webber. 1991. Structure and ostension in the interpretation of discourse deixis. Language and Cognitive processes, 6(2):107–135. find
  43. Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio. 2013. On the difficulty of training recurrent neural networks. In Proceedings of the 30th International Conference on Machine Learning (ICML), Atlanta, USA. find
  44. Sam Joshua Wiseman, Alexander Matthew Rush, Stuart Merrill Shieber, and Jason Weston. 2015. Learning anaphoricity and antecedent ranking features for coreference resolution. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (ACL), Beijing, China. find
  45. Jie Zhou and Wei Xu. 2015. End-to-end learning of semantic role labeling using recurrent neural networks. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 1127–1137, Beijing, China. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/marasovic-2017-mention-ranking-model-abstract-anaphora/