Dependency-based paraphrasing for recognizing textual entailment

What it is about

The paper applies DIRT paraphrases, derived from about 1 GB of newspaper text, to the Minipar dependency trees of RTE-3 text. A paraphrase is applied only when the source path occurs in the text and the translation path occurs in the hypothesis, so the rewrite moves the text closer to the hypothesis. Of 38,207 text nodes, 6,173 have a paraphrase with the same lemma, 2,211 match a source path, and only 228 are actually substituted; about one text sentence in six is changed, and more often in true pairs than in false ones. The effect on the simple alignment-based RTE system is small: overall accuracy on the development set rises from 66.9 to 68.2 and on the test set from 58.6 to 59.1. Most paraphrases concern verbs, and the authors note that many required paraphrases are missing and that some DIRT rules are false.

Ideas

• I1. Keep the double constraint of the paper as a gate: accept a rewrite only if the source occurs in the text and the target occurs in the hypothesis. Put this in prag/src/infer.rs for every paraphrase table, and measure the dev gain, run twice.
• I2. Report paraphrase coverage by word class, as the paper's Table 1 does. Mova can see where its lexicon lacks nouns and adjectives, using the word-class chapter 10. Word classes.
• I3. Audit the rules for direction: the paper's examples "X feeds Y entails Y feeds X" show symmetric errors. Mark which pairs are symmetric and check them against the dictionary in world/src/dictionary.rs.
• I4. Use the count of applied rewrites per pair as a feature for a decision list, instead of a hard decision, and check whether it separates true from false pairs on the school lessons.

For Mova

The chapters are 19. The lexicon: vocabulary, multiword expressions and register and 10. Word classes. The code is prag/src/infer.rs and the dictionary world/src/dictionary.rs. Any paraphrase resource must have its licence checked in Data and model licences before it is used in a build. The measure is dev accuracy with and without the gate, run twice.

Cite

The paper itself, not this card. BibTeX from the library's record of the work.

@inproceedings{marsi2007dependency,
  title = {{Dependency-based paraphrasing for recognizing textual entailment}},
  author = {Marsi, Erwin and Krahmer, Emiel and Bosma, Wauter},
  booktitle = {Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Paraphrasing},
  year = {2007},
  url = {https://aclanthology.org/W07-1414/}
}

References

11 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. R. Bar-Haim, I. Dagan, B. Dolan, L. Ferro, D. Giampiccolo, B. Magnini, and I. Szpektor. 2006. The second pascal recognising textual entailment challenge. In Proceedings of the Second PASCAL Challenges Workshop on Recognising Textual Entailment, pages 1–9, Venice, Italy. find
2. W. Bosma and C. Callison-Burch. 2006. Paraphrase substitution for recognizing textual entailment. In Proceedings of CLEF. find
3. R. de Salvo Braz, R. Girju, V. Punyakanok, D. Roth, and M. Sammons. 2005. An inference model for semantic entailemnt in natural language. In Proceedings of the First Pascal Challenge Workshop on Recognizing Textual Entailment, pages 29–32. find
4. A. Hickl, J. Williams, J. Bensley, K. Roberts, B. Rink, and Y. Shi. 2006. Recognizing textual entailment with lccs groundhog system. In Proceedings of the Second PASCAL Challenges Workshop on Recognising Textual Entailment, pages 80–85, Venice, Italy. find
5. Dekang Lin and Patrick Pantel. 2001. Discovery of inference rules for question answering. Natural Language Engineering, 7(4):343–360. find
6. Dekang Lin. 1998. Dependency-based evaluation of minipar. In Proceedings of the Workshop on Evaluation of Parsing Systems at LREC 1998, pages 317–330, Granada, Spain. find
7. E. Marsi, E. Krahmer, W. Bosma, and M. Theune. 2006. Normalized alignment of dependency trees for detecting textual entailment. In Proceedings of the Second PASCAL Challenges Workshop on Recognising Textual Entailment, pages 56–61, venice, Italy. find
8. Adam Meyers, Roman Yangarber, and Ralph Grisham. 1996. Alignment of shared forests for bilingual corpora. In Proceedings of 16th International Conference on Computational Linguistics (COLING-96), pages 460–465, Copenhagen, Denmark. find
9. R. Nielsen, W. Ward, and J.H. Martin. 2006. Toward dependency path based entailment. In Proceedings of the Second PASCAL Challenges Workshop on Recognising Textual Entailment, pages 44–49, Venice, Italy. find
10. R. Raina, A. Haghighi, C. Cox, J. Finkel, J. Michels, K. Toutanova, B. MacCartney, M.C. de Marneffe, C.D. Manning, and A.Y. Ng. 2005. Robust textual inference using diverse knowledge sources. In Proceedings of PASCAL Recognising Textual Entailment Workshop. find
11. J. C. Reynar and A. Ratnaparkhi. 1997. A maximum entropy approach to identifying sentence boundaries. In Proceedings of the Fifth Conference on Applied Natural Language Processing, Washington, D.C. buy mod obj by ... pcomp-n ... Baikalfinansgroup mod nn known company whn i be which pred later subj {which} sell mod obj to ... pcomp-n ... Baikalfinansgroup sell s mod obj subj Baikalfinansgroup to {US$ 9.4 billion} {Baikalfinansgroup} mod nn rel pcomp-n known company {fin} Rosneft det whn i mod mod which the Russian be state-owned pred lex-mod lex-mod later - state subj {which} Figure 2: Alignment of paraphrase source to text (top), alignment of paraphrase translation to hypothesis (mid), and alignment of hypothesis to paraphrased text (bottom) for pair 1 from RTE3 dev set buy s mod subj obj by {Baikalfinansgroup} {US$ 9.4 billion} pcomp-n rel {fin} Rosneft det mod mod nn the Russian state-owned oil company lex-mod lex-mod lex-mod state - oil sell s be mod obj be to {Baikalfinansgroup} pcomp-n Rosneft sell s obj be mod Baikalfinansgroup be to {Baikalfinansgroup} nn pcomp-n oil company Rosneft lex-mod oil find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/marsi-2007-dependency-based-paraphrasing-recognizing-textual/