Recognising Textual Entailment with Logical Inference

What it is about

The paper compares two domain-independent methods for recognising textual entailment on the RTE challenge data (567 development and 800 test pairs). The shallow method uses word overlap extended by WordNet synonyms and derivations, weighted by inverse document frequency. The deep method parses text and hypothesis with a CCG parser into discourse representations, translates them into first-order logic, and asks a theorem prover (Vampire) and a model builder (Paradox), with background axioms from WordNet and the CIA Factbook. Entailment is approximated by how much the model's domain grows when the hypothesis is added. A decision tree combines the features. On the test set the shallow method reaches 56.9%, the prover alone about 52%, the deep method with model building 56.2%, and the hybrid with the RTE task type as a feature 61.2%. The prover finds only 30 proofs, so its recall is 0.058 at precision 0.767; missing background knowledge is the main cause.

Ideas

• I1. Domain-size difference as a counting feature. Count the entities in the world built from the text and from text plus hypothesis; the absolute and relative growth are numbers Mova can explain ("the hypothesis adds two entities"). Test as a decision-list feature on the development set, run twice.
• I2. Task type as a feature. The paper gains about three points by adding the subset of the RTE data as input. A Mova rule combiner can take the lesson type the same way, chosen on cross-validation, with the test set touched once.
• I3. Explicit lexical rules: hyponym pairs entail, sister terms contradict. Keep them as a small lexicon in en/src/dict.rs, and log each rule that fires.
• I4. Report precision and recall of the proof route separately, use the proof only when it fires, and let the approximation handle the rest.

For Mova

Chapter: 27. Negation (negated hypotheses, the paper's Example 78 type). Code: prag/src/infer.rs for the decision, en/src/dict.rs for the lexical relations, world/src/qa.rs for the school questions. Measure: accuracy by route (proof, approximation, both), run twice for identical output.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.3115/1220575.1220654

@inproceedings{bos2005recognising,
  title = {{Recognising Textual Entailment with Logical Inference}},
  author = {Bos, Johan and Markert, Katja},
  booktitle = {Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing},
  year = {2005},
  doi = {10.3115/1220575.1220654},
  url = {https://doi.org/10.3115/1220575.1220654}
}

References

15 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. E. Akhmatova. 2005. Textual entailment resolution via atomic propositions. In PASCAL. Proc. of the First Challenge Workshop. Recognizing Textual Entailment. find
2. S. Bayer, J. Burger, L. Ferro, J. Henderson, and A. Yeh. 2005. Mitre’s submission to the eu pascal rte challenge. In PASCAL. Proc. of the First Challenge Workshop. Recognizing Textual Entailment. find
3. P. Blackburn and J. Bos. 2005. Representation and Inference for Natural Language. A First Course in Computational Semantics. CSLI. find
4. J. Bos, S. Clark, M. Steedman, J. Curran, and J. Hockenmaier. 2004. Wide-coverage semantic representations from a CCG parser. In Proc of COLING. find
5. K. Claessen and N. Sörensson. 2003. New techniques that improve mace-style model finding. In Model Computationa - Principles, Algorithms, Applications (Cade-19 Workshop), Miami, Florida. find
6. I. Dagan, O. Glickman, and B. Magnini. 2005. The pascal recognising textual entailment challenge. In PAS-CAL. Proc. of the First Challenge Workshop. Recognizing Textual Entailment. find
7. C. Fellbaum, editor. 1998. WordNet: An Electronic Lexical Database. MIT Press, Cambridge, Mass. find
8. A. Fowler, B. Hauser, D. Hodges, I. Niles, A. Novischi, and J. Stephan. 2005. Applying cogex to recognize textual entailment. In PASCAL. Proc. of the First Challenge Workshop. Recognizing Textual Entailment. find
9. H. Kamp and U. Reyle. 1993. From Discourse to Logic. Introduction to Modeltheoretic Semantics of Natural Language, Formal Logic and Discourse Representation Theory. Kluwer, Dordrecht, Netherlands. find
10. C. Manning and H. Schuetze. 1999. Foundations of Statistical Natural Language Processing. MIT Press. find
11. C. Monz and M. de Rijke. 2003. Light-weight entailment checking for computational semantics. In Proc. of ICOS-3. find
12. R. Raina, A.Y. Ng, and C. Manning. 2005. Robust textual inference via learning and abductive reasoning. In Proc. of AAAI 2005. find
13. A. Riazanov and A. Voronkov. 2002. The design and implementation of Vampire. AI Comm., 15(2-3). find
14. H. Saggion, R. Gaizauskas, M. Hepple, I. Roberts, and M Greenwood. 2004. Exploring the performance of boolean retrieval strategies for open domain question answering. In Proc. of the IR4QA Workshop at SIGIR. find
15. I. H. Witten and E. Frank. 2000. Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations. Morgan Kaufmann, San Diego, CA. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/bos-2005-recognising-textual-entailment-logical-inference/