Mova0.5 pre-release⌕
☰
home / docs / papers / English / Abductive Commonsense Reasoning

Abductive Commonsense Reasoning

What it is about

The paper introduces abductive reasoning over narratives. Given two observations O1 and O2, a system must pick the more plausible explanation from two hypotheses (αNLI), or generate one (αNLG). The ART dataset has about 20k contexts taken from ROCStories and over 200k hypotheses, written by crowdworkers and then filtered by adversarial filtering with BERT, which keeps pairs that are hard to tell apart. The best αNLI model, a fully connected BERT, reaches 68.9%, against 91.4% for humans; an entailment model reaches 58.8%. Filtering drops BERT from over 88% to about 69%. For generation, the best GPT-2 variant is judged to explain the observations in 45% of cases, against 96% for human text. Numerical items are the weakest category (BERT 56.8%, human 91.5%). Pre-training on ART helps WinoGrande, WSC, DPR and HellaSwag when their training data are small.

Ideas

For Mova

Chapters: 36. Discourse relations and coherence (causal and temporal links between sentences) and 30. Pragmatics and discourse: overview. Code: prag/src/infer.rs, world/src/narr.rs, world/src/events.rs. Measure: accuracy on the ART development items, per category and per broken link, run twice for identical output.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. arXiv 1908.05739

@misc{bhagavatula2019abductive,
  title = {{Abductive Commonsense Reasoning}},
  author = {Bhagavatula, Chandra and Bras, Ronan Le and Malaviya, Chaitanya and Sakaguchi, Keisuke and Holtzman, Ari and Rashkin, Hannah and Downey, Doug and Yih, Scott Wen-tau and Choi, Yejin},
  year = {2019},
  eprint = {1908.05739},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1908.05739}
}

References

52 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Henning Andersen. Abductive and deductive change. Language, pp. 765–793, 1973. URL https://www.jstor.org/stable/pdf/412063.pdf. ↗
  2. Satanjeev Banerjee and Alon Lavie. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, pp. 65–72, 2005. find our card
  3. Antoine Bosselut, Hannah Rashkin, Maarten Sap, Chaitanya Malaviya, Asli Celikyilmaz, and Yejin Choi. Comet: Commonsense transformers for automatic knowledge graph construction. arXiv preprint arXiv:1906.05317, 2019. ↗
  4. Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, 2015. URL https://nlp.stanford.edu/pubs/snli_paper.pdf. ↗
  5. Oana-Maria Camburu, Tim Rocktäschel, Thomas Lukasiewicz, and Phil Blunsom. e-snli: Natural language inference with natural language explanations. In Advances in Neural Information Processing Systems, pp. 9560–9572, 2018. URL https://papers.nips.cc/paper/8163-e-snli-natural-language-inference-with-natural-language-explanations. pdf. ↗
  6. Nathanael Chambers and Dan Jurafsky. Unsupervised learning of narrative schemas and their participants. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, pp. 602–610, Suntec, Singapore, August 2009. Association for Computational Linguistics. URL http://www.aclweb.org/anthology/P/P09/P09-1068. ↗
  7. Eugene Charniak and Solomon Eyal Shimony. Probabilistic semantics for cost based abduction. Brown University, Department of Computer Science, 1990. URL https://www.aaai.org/Papers/AAAI/1990/AAAI90-016.pdf. ↗
  8. Qian Chen, Xiao-Dan Zhu, Zhen-Hua Ling, Si Wei, Hui Jiang, and Diana Inkpen. Enhanced lstm for natural language inference. In ACL, 2017. URL https://www.aclweb.org/anthology/P17-1152. ↗
  9. Peter E. Clark, Philip Harrison, John A. Thompson, William R. Murray, Jerry R. Hobbs, and Christiane Fellbaum. On the role of lexical and world knowledge in rte3. In ACL-PASCAL@ACL, 2007. URL https://www.aclweb.org/anthology/W07-1409. ↗
  10. Alexis Conneau, Douwe Kiela, Holger Schwenk, Loı̈c Barrault, and Antoine Bordes. Supervised learning of universal sentence representations from natural language inference data. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 670– 680, Copenhagen, Denmark, September 2017. Association for Computational Linguistics. doi: 10.18653/v1/D17-1070. URL https://www.aclweb.org/anthology/D17-1070. ↗
  11. Ido Dagan, Oren Glickman, and Bernardo Magnini. The pascal recognising textual entailment challenge. In Machine learning challenges. evaluating predictive uncertainty, visual object classification, and recognising tectual entailment, pp. 177–190. Springer, 2006. URL http://u.cs.biu.ac.il/˜dagan/publications/RTEChallenge.pdf. ↗
  12. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018. URL https://arxiv.org/abs/1810.04805. ↗
  13. Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel Bowman, and Noah A. Smith. Annotation artifacts in natural language inference data. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pp. 107–112, New Orleans, Louisiana, June 2018. Association for Computational Linguistics. doi: 10.18653/v1/N18-2017. URL https://www.aclweb.org/anthology/N18-2017. ↗
  14. Jerry R. Hobbs, Mark Stickel, Paul Martin, and Douglas Edwards. Interpretation as abduction. In Proceedings of the 26th Annual Meeting of the Association for Computational Linguistics, pp. 95–103, Buffalo, New York, USA, June 1988. Association for Computational Linguistics. doi: 10.3115/982023.982035. URL https://www.aclweb.org/anthology/P88-1012. ↗
  15. Bram Jans, Steven Bethard, Ivan Vulić, and Marie-Francine Moens. Skip n-grams and ranking functions for predicting script events. In Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics, pp. 336–344, Avignon, France, April 2012. Association for Computational Linguistics. URL http://www.aclweb.org/anthology/E12-1034. ↗
  16. Susan G. Josephson. Abductive inference: Computation , philosophy , technology. 2000. URL https://philpapers.org/rec/JOSAIC. ↗
  17. George Lakoff. Linguistics and natural logic. Synthese, 22(1-2):151–271, 1970. URL https://link.springer.com/article/10.1007/BF00413602. ↗
  18. Hector J. Levesque, Ernest Davis, and Leora Morgenstern. The winograd schema challenge. In KR, 2011. find
  19. Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. Text Summarization Branches Out, 2004. find
  20. Peter LoBue and Alexander Yates. Types of common-sense knowledge needed for recognizing textual entailment. In ACL, 2011. URL https://www.aclweb.org/anthology/P11-2057. ↗
  21. Bill MacCartney and Christopher D. Manning. Natural logic for textual inference. In Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Paraphrasing, pp. 193–200, Prague, June 2007. Association for Computational Linguistics. URL https://www.aclweb.org/anthology/W07-1431. ↗
  22. Bill MacCartney and Christopher D. Manning. An extended model of natural logic. In Proceedings of the Eight International Conference on Computational Semantics, pp. 140–156, Tilburg, The Netherlands, January 2009. Association for Computational Linguistics. URL https://www.aclweb.org/anthology/W09-3714. ↗
  23. Nicole Maslan, Melissa Roemmele, and Andrew S. Gordon. One hundred challenge problems for logical formalizations of commonsense psychology. In AAAI Spring Symposia, 2015. URL http://people.ict.usc.edu/˜gordon/publications/AAAI-SPRING15.PDF. ↗
  24. Nasrin Mostafazadeh, Nathanael Chambers, Xiaodong He, Devi Parikh, Dhruv Batra, Lucy Vanderwende, Pushmeet Kohli, and James Allen. A corpus and cloze evaluation for deeper understanding of commonsense stories. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL), pp. 839–849. Association for Computational Linguistics, 2016. doi: 10.18653/v1/N16-1098. URL http://aclweb.org/anthology/N16-1098. ↗
  25. Peter Norvig. Inference in text understanding. In AAAI, pp. 561–565, 1987. URL http://norvig.com/aaai87.pdf. ↗
  26. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In ACL, 2002. find our card
  27. Judea Pearl. Reasoning with cause and effect. AI Magazine, 23(1):95, 2002. URL https://ftp.cs.ucla.edu/pub/stat_ser/r265-ai-mag.pdf. ↗
  28. Judea Pearl and Dana Mackenzie. The Book of Why: The New Science of Cause and Effect. Basic Books, Inc., New York, NY, USA, 1st edition, 2018. ISBN 046509760X, 9780465097609. URL https://dl.acm.org/citation.cfm?id=3238230. ↗
  29. Charles Sanders Peirce. Collected papers of Charles Sanders Peirce, volume 5. Harvard University Press, 1965a. URL http://www.hup.harvard.edu/catalog.php?isbn=9780674138001. ↗
  30. Charles Sanders Peirce. Pragmatism and pragmaticism, volume 5. Belknap Press of Harvard University Press, 1965b. URL https://www.jstor.org/stable/224970. ↗
  31. Jeffrey Pennington, Richard Socher, and Christopher Manning. Glove: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543, Doha, Qatar, October 2014. Association for Computational Linguistics. doi: 10.3115/v1/D14-1162. URL https://www.aclweb.org/anthology/D14-1162. ↗
  32. Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pp. 2227–2237, New Orleans, Louisiana, June 2018. Association for Computational Linguistics. doi: 10.18653/v1/N18-1202. URL https://www.aclweb.org/anthology/N18-1202. ↗
  33. Karl Pichotta and Raymond Mooney. Statistical script learning with multi-argument events. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, pp. 220–229, Gothenburg, Sweden, April 2014. Association for Computational Linguistics. URL http://www.aclweb.org/anthology/E14-1024. ↗
  34. Adam Poliak, Jason Naradowsky, Aparajita Haldar, Rachel Rudinger, and Benjamin Van Durme. Hypothesis only baselines in natural language inference. In Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics, pp. 180–191, New Orleans, Louisiana, June 2018. Association for Computational Linguistics. doi: 10.18653/v1/S18-2023. URL https://www.aclweb.org/anthology/S18-2023. ↗
  35. Alec Radford. Improving language understanding by generative pre-training. 2018. find
  36. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. OpenAI Blog, 1(8), 2019. find
  37. Altaf Rahman and Vincent Ng. Resolving complex cases of definite pronouns: The winograd schema challenge. In EMNLP-CoNLL, 2012. find
  38. Rajat Raina, Andrew Y Ng, and Christopher D Manning. Robust textual inference via learning and abductive reasoning. In AAAI, pp. 1099–1105, 2005. URL https://nlp.stanford.edu/˜manning/papers/aaai05-learnabduction.pdf. ↗
  39. Rachel Rudinger, Pushpendre Rastogi, Francis Ferraro, and Benjamin Van Durme. Script induction as language modeling. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 1681–1686, Lisbon, Portugal, September 2015. Association for Computational Linguistics. URL http://aclweb.org/anthology/D15-1195. ↗
  40. Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adversarial winograd schema challenge at scale. In AAAI, 2020. find
  41. Maarten Sap, Ronan Le Bras, Emily Allaway, Chandra Bhagavatula, Nicholas Lourie, Hannah Rashkin, Brendan Roof, Noah A Smith, and Yejin Choi. Atomic: an atlas of machine commonsense for if-then reasoning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pp. 3027–3035, 2019. find
  42. Roger C. Schank and Robert P. Abelson. Scripts, plans, and knowledge. In Proceedings of the 4th International Joint Conference on Artificial Intelligence - Volume 1, IJCAI’75, pp. 151–157, San Francisco, CA, USA, 1975. Morgan Kaufmann Publishers Inc. URL http://dl.acm.org/citation.cfm?id=1624626.1624649. ↗
  43. Gary Shank. The extraordinary ordinary powers of abductive reasoning. Theory & Psychology, 8(6):841–860, 1998. URL https://journals.sagepub.com/doi/10.1177/0959354398086007. ↗
  44. Masatoshi Tsuchiya. Performance impact caused by hidden bias of training data for recognizing textual entailment. CoRR, abs/1804.08117, 2018. URL http://www.lrec-conf.org/proceedings/lrec2018/pdf/786.pdf. ↗
  45. Ramakrishna Vedantam, C Lawrence Zitnick, and Devi Parikh. Cider: Consensus-based image description evaluation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4566–4575, 2015. find
  46. Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pp. 353–355, Brussels, Belgium, November 2018. Association for Computational Linguistics. URL https://www.aclweb.org/anthology/W18-5446. ↗
  47. Adina Williams, Nikita Nangia, and Samuel Bowman. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pp. 1112–1122. Association for Computational Linguistics, 2018a. URL http://aclweb.org/anthology/N18-1101. ↗
  48. Adina Williams, Nikita Nangia, and Samuel Bowman. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pp. 1112–1122, New Orleans, Louisiana, June 2018b. Association for Computational Linguistics. doi: 10.18653/v1/N18-1101. URL https://www.aclweb.org/anthology/N18-1101. ↗
  49. Rowan Zellers, Yonatan Bisk, Roy Schwartz, and Yejin Choi. Swag: A large-scale adversarial dataset for grounded commonsense inference. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2018. URL https://aclweb.org/anthology/D18-1009. ↗
  50. Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can a machine really finish your sentence? In ACL, 2019. find
  51. Sheng Zhang, Rachel Rudinger, Kevin Duh, and Benjamin Van Durme. Ordinal common-sense inference. Transactions of the Association for Computational Linguistics, 5:379–395, 2017. doi: 10.1162/tacl a 00068. URL https://www.aclweb.org/anthology/Q17-1027. ↗
  52. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675, 2019. ↗

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/bhagavatula-2019-abductive-commonsense-reasoning/