Combining Event Semantics and Degree Semantics for Natural Language Inference

What it is about

A logic-based NLI system that combines two semantic frameworks: neo-Davidsonian event semantics (adverbs such as "slowly" are predicates of an event) and degree semantics (adjectives as tall(x, d), comparatives by the A-not-A analysis). CCG parsers feed semantic templates, which produce first-order formulas for a theorem prover, Vampire. Lexical axioms are added from seven relation types (antonym, hypernym, hyponym, synonym and others). Downward quantifiers such as "few" and non-monotone ones such as "exactly" get syntactic categories that let negation scope over the whole clause. The system is tested on FraCaS, MED, SICK, HANS and a new monotonicity-with-adverbs set. Accuracy on FraCaS generalised quantifiers is 0.97 (0.99 with hand rules), on MED 0.96 (generalised quantifiers) and 0.92 (with lexical knowledge), and on HANS "unknown" items 0.78 against 0.56 for RoBERTa. On the adjective and comparative set CAD+ it reaches 0.83 against 0.58 for RoBERTa.

Ideas

• I1. Quantifier table with scope: a closed table of quantifiers with a flag for downward and non-monotone scope, in world/src/qa.rs. Negation then scopes over the clause for "few" and "less than five". Measure on hand-written items of the MED kind.
• I2. Adverb drop as a rule: "John ran slowly" entails "John ran". The paper finds the rule over-applies to modal adverbs ("probably"), so add a list of adverb classes that block the drop, and measure the change.
• I3. Lexical axioms from the dictionary: compile the relations in en/src/dict.rs into the paper's fixed implication forms (hypernym gives one direction, synonym gives both). Keep the per-category accuracy table as the report format.
• I4. The per-dataset table is an external yardstick only. Do not claim a comparison with RoBERTa unless Mova has been run on the same items.

For Mova

The chapters are 27. Negation, 12. Pronouns, determiners and numerals and 15. Adjectives and adverbs. The code is world/src/qa.rs and en/src/dict.rs. The measure is per-category accuracy on hand-written inference items, run twice to confirm identical output.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.coling-main.156

@inproceedings{haruta2020combining,
  title = {{Combining Event Semantics and Degree Semantics for Natural Language Inference}},
  author = {Haruta, Izumi and Mineshima, Koji and Bekki, Daisuke},
  booktitle = {Proceedings of the 28th International Conference on Computational Linguistics},
  year = {2020},
  doi = {10.18653/v1/2020.coling-main.156},
  url = {https://doi.org/10.18653/v1/2020.coling-main.156}
}

References

32 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Lasha Abzianidze. 2016. Natural solution to FraCaS entailment problems. In Proceedings of the Fifth Joint Conference on Lexical and Computational Semantics (*SEM), pages 64–74. Association for Computational Linguistics. find
2. Johan Bos. 2008. Wide-coverage semantic analysis with Boxer. In Semantics in Text Processing. STEP 2008 Conference Proceedings, pages 277–286. College Publications. find
3. Brian Buccola and Benjamin Spector. 2016. Modified numerals and maximality. Linguistics and Philosophy, 39(3):151–199. find
4. Timothy Chklovski and Patrick Pantel. 2004. VerbOcean: Mining the web for fine-grained semantic verb relations. In Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 33–40. Association for Computational Linguistics. find
5. Stephen Clark and James R Curran. 2007. Wide-coverage efficient statistical parsing with CCG and log-linear models. Computational Linguistics, 33(4):493–552. find our card
6. Robin Cooper, Richard Crouch, Jan van Eijck, Chris Fox, Josef van Genabith, Jan Jaspers, Hans Kamp, Manfred Pinkal, Massimo Poesio, Stephen Pulman, et al. 1994. FraCaS–a framework for computational semantics. Deliverable, D6. find
7. Max J Cresswell. 1976. The semantics of degree. In Barbara Partee, editor, Montague Grammar, pages 261–292. Academic Press. find
8. Donald Davidson. 1967. The logical form of action sentences. In Nicholas Rescher, editor, The Logic of Decision and Action, pages 81–95. University of Pittsburgh Press, Pittsburgh, PA. find
9. Matt Gardner, Joel Grus, Mark Neumann, Oyvind Tafjord, Pradeep Dasigi, Nelson F. Liu, Matthew Peters, Michael Schmitz, and Luke Zettlemoyer. 2018. AllenNLP: A deep semantic natural language processing platform. In Proceedings of Workshop for NLP Open Source Software (NLP-OSS), pages 1–6. Association for Computational Linguistics. find
10. Martin Hackl. 2000. Comparative Quantifiers. Ph.D. thesis, Massachusetts Institute of Technology. find
11. Izumi Haruta, Koji Mineshima, and Daisuke Bekki. 2020. Logical inferences with comparatives and generalized quantifiers. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL): Student Research Workshop, pages 263–270. Association for Computational Linguistics. find
12. Christopher Kennedy. 1997. Projecting the Adjective: The Syntax and Semantics of Gradability and Comparison. Ph.D. thesis, University of California, Santa Cruz. find
13. Ewan Klein. 1982. The interpretation of adjectival comparatives. Journal of Linguistics, 18(1):113–136. find
14. Roger Levy and Galen Andrew. 2006. Tregex and Tsurgeon: tools for querying and manipulating tree data structures. In Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC’06), pages 2231–2234. find
15. Mike Lewis and Mark Steedman. 2014. A* CCG parsing with a supertag-factored model. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 990–1000. Association for Computational Linguistics. find our card
16. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized BERT pretraining approach. ArXiv, abs/1907.11692. ↗
17. Marco Marelli, Stefano Menini, Marco Baroni, Luisa Bentivogli, Raffaella Bernardi, and Roberto Zamparelli. 2014. A SICK cure for the evaluation of compositional distributional semantic models. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 216–223. find
18. Pascual Martı́nez-Gómez, Koji Mineshima, Yusuke Miyao, and Daisuke Bekki. 2017. On-demand injection of lexical knowledge for recognising textual entailment. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics (EACL): Volume 1, Long Papers, pages 710–720. Association for Computational Linguistics. find
19. Tom McCoy, Ellie Pavlick, and Tal Linzen. 2019. Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL), pages 3428–3448. Association for Computational Linguistics. find our card
20. George A. Miller. 1995. WordNet: A lexical database for english. Commun. ACM, 38(11):39–41. ↗ our card
21. Koji Mineshima, Pascual Martı́nez-Gómez, Yusuke Miyao, and Daisuke Bekki. 2015. Higher-order logical inference with compositional semantics. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2055–2061. Association for Computational Linguistics. find
22. Richard Montague. 1970. Universal grammar. Theoria, 36(3):373–398. find
23. Terence Parsons. 1990. Events in the Semantics of English. MIT press. find
24. Jessica Rett. 2018. The semantics of many, much, few, and little. Language and Linguistics Compass, 12(1). find
25. Roger Schwarzschild. 2008. The semantics of comparatives and other degree constructions. Language and Linguistics Compass, 2(2):308–331. find
26. Pieter A. M. Seuren. 1973. The comparative. In F. Kiefer and N. Ruwet, editors, Generative Grammar in Europe, pages 528–564. Riedel, Dordrecht. find
27. Arnim von Stechow. 1984. Comparing semantic theories of comparison. Journal of Semantics, 3(1-2):1–77. find
28. Mark Steedman. 2000. The Syntactic Process. MIT press. find
29. Johan van Benthem. 1986. Essays in logical semantics. Reidel. find
30. Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), pages 1112–1122. Association for Computational Linguistics. find
31. Hitomi Yanaka, Koji Mineshima, Daisuke Bekki, Kentaro Inui, Satoshi Sekine, Lasha Abzianidze, and Johan Bos. 2019. Can neural networks understand monotonicity reasoning? In Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 31–40. Association for Computational Linguistics. find
32. Masashi Yoshikawa, Hiroshi Noji, and Yuji Matsumoto. 2017. A* CCG parsing with a supertag and dependency factored model. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL), pages 277–287. Association for Computational Linguistics. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/haruta-2020-combining-event-semantics-degree-semantics/