Multi-word expressions in textual inference: Much ado about nothing?

What it is about

The paper asks whether multi-word expressions (MWEs) matter for recognising textual entailment (RTE). It uses the manual Microsoft Research word alignments for the RTE2 data and counts the MWEs that appear as multi-word alignment links. Only 81 (dev) and 60 (test) true MWEs occur, in 8.2% of the 1,600 sentence pairs and in 1.2% of the SURE alignment links. The authors then check two paraphrase resources: 34% of the MWEs are present, but almost all with very low scores. On the 58 test pairs that contain a true MWE, the entailment accuracy of two aligners (62% to 74%) is not worse than on the full set. The conclusion is that wrongly aligned MWEs rarely change an entailment decision.

Ideas

• I1. Frequency audit before any MWE work: in the en parses of EWT test, count the multi-token units that data/lemmas.tsv already holds and the share of tokens they cover. Report it with en parse-eval. Measure: share of tokens inside listed MWEs, per treebank.
• I2. Gate for prag expressions: a small hand-written set of paraphrase pairs in the style of the paper's examples (take a pass on / not buy; kick the bucket / die). Check whether the expression base catches each pair before the agreer. Measure: catch rate, and the share of wrong catches.
• I3. Negative control for any MWE feature in world: before adding MWE handling, record the errors that trace to MWE splits. If that share is below a set threshold, the feature is not worth its cost. Measure on the existing world tale readers, two identical runs.

For Mova

The chapter is 19. The lexicon: vocabulary, multiword expressions and register, with 29. Ellipsis, word order and information structure only for paraphrase order. The code is en/src/lexgen.rs (what counts as an entry) and prag expressions. The RTE2 alignments and the paper's pairs are not Mova data; the pairs in I2 must be written by hand and kept held out.

Cite

The paper itself, not this card. BibTeX from the library's record of the work.

@inproceedings{marneffe2009multi,
  title = {{Multi-word expressions in textual inference: Much ado about nothing?}},
  author = {Marneffe, Marie-Catherine de and Padó, Sebastian and Manning, Christopher D.},
  booktitle = {Proceedings of the 2009 Workshop on Applied Textual Inference (TextInfer)},
  year = {2009},
  url = {https://aclanthology.org/W09-2501/}
}

References

23 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Colin Bannard and Chris Callison-Burch. 2005. Paraphrasing with bilingual parallel corpora. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics, pages 597–604, Ann Arbor, MI. find our card
2. Roy Bar-Haim, Idan Szpecktor, and Oren Glickman. 2005. Definition and analysis of intermediate entailment levels. In Proceedings of the ACL Workshop on Empirical Modeling of Semantic Equivalence and Entailment, pages 55–60, Ann Arbor, MI. find
3. Roy Bar-Haim, Ido Dagan, Bill Dolan, Lisa Ferro, Danilo Giampiccolo, Bernardo Magnini, and Idan Szpektor. 2006. The second PASCAL recognising textual entailment challenge. In Proceedings of the Second PASCAL Challenges Workshop on Recognising Textual Entailment, Venice, Italy. find
4. Francis Bond, Anna Korhonen, Diana McCarthy, and Aline Villavicencio, editors. 2003. Proceedings of the ACL 2003 workshop on multiword expressions: Analysis, acquisition and treatment. find
5. Chris Brockett. 2007. Aligning the RTE 2006 corpus. Technical Report MSR-TR-2007-77, Microsoft Research. find
6. Peter F. Brown, Stephen Della Pietra, Vincent J. Della Pietra, and Robert L. Mercer. 1993. The mathematic of statistical machine translation: Parameter estimation. Computational Linguistics, 19(2):263– 311. find
7. Aljoscha Burchardt, Nils Reiter, Stefan Thater, and Anette Frank. 2007. A semantic approach to textual entailment: System evaluation and task analysis. In Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Paraphrasing, pages 10– 15, Prague, Czech Republic. find
8. Michael Collins. 2002. Discriminative training methods for hidden Markov models. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. find
9. Ido Dagan, Oren Glickman, and Bernardo Magnini. 2006. The PASCAL recognising textual entailment challenge. In J. Quinonero-Candela, I. Dagan, B. Magnini, and F. d’Alch Buc, editors, Machine Learning Challenges. Lecture Notes in Computer Science, Vol. 3944, pages 177–190. Springer. find
10. Marie-Catherine de Marneffe, Trond Grenager, Bill MacCartney, Daniel Cer, Daniel Ramage, Chloé Kiddon, and Christopher D. Manning. 2007. Aligning semantic graphs for textual inference and machine reading. In Proceedings of the AAAI Spring Symposium. find
11. Georgiana Dinu and Rui Wang. 2009. Inference rules and their application to recognizing textual entailment. In Proceedings of the 12th Conference of the European Chapter of the ACL (EACL 2009), pages 211–219, Athens, Greece. find
12. Nicole Grégoire, Stefan Evert, and Su Nam Kim, editors. 2007. Proceedings of the ACL workshop: A broader perspective on multiword expressions. find
13. Andrew Hickl and Jeremy Bensley. 2007. A discourse commitment-based framework for recognizing textual entailment. In Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Paraphrasing, pages 171–176, Prague, Czech Republic. find
14. Adrian Iftene and Alexandra Balahur-Dobrescu. 2007. Hypothesis transformation and semantic variability rules used in recognizing textual entailment. In Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Paraphrasing, pages 125–130, Prague, Czech Republic. find
15. Dekang Lin and Patrick Pantel. 2002. Discovery of inference rules for question answering. Journal of Natural Language Engineering, 7(4):343–360. find
16. Dekang Lin. 1998. Automatic retrieval and clustering of similar words. In Proceedings of the joint Annual Meeting of the Association for Computational Linguistics and International Conference on Computational Linguistics, pages 768–774, Montréal, Canada. find
17. Bill MacCartney, Trond Grenager, Marie-Catherine de Marneffe, Daniel Cer, and Christopher D. Manning. 2006. Learning to recognize features of valid textual entailments. In Proceedings of NAACL. find
18. Bill MacCartney, Michel Galley, and Christopher D. Manning. 2008. A phrase-based alignment model for natural language inference. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, Honolulu, Hawaii. find
19. Erwin Marsi, Emiel Krahmer, and Wauter Bosma. 2007. Dependency-based paraphrasing for recognizing textual entailment. In Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Paraphrasing, pages 83–88, Prague, Czech Republic. find our card
20. Begona Villada Moirón, Aline Villavicencio, Diana McCarthy, Stefan Evert, and Suzanne Stevenson, editors. 2006. Proceedings of the ACL Workshop on Multiword Expressions: Identifying and Exploiting Underlying Properties. find
21. Ivan A. Sag, Timothy Baldwin, Francis Bond, Ann Copestake, and Dan Flickinger. 2002. Multi-word expressions: a pain in the neck for NLP. In Proceedings of CICLing. find
22. Takaaki Tanaka, Aline Villavicencio, Francis Bond, and Anna Korhonen, editors. 2004. Proceedings of the second ACL workshop on multiword expressions: Integrating processing. find
23. Fabio Massimo Zanzotto, Marco Pennacchiotti, and Alessandro Moschitti. 2007. Shallow semantic in fast textual entailment rule learners. In Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Paraphrasing, pages 72–77, Prague, Czech Republic. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2009-marneffe-multi-word-expressions-textual-inference/