PPDB 2.0: Better paraphrase ranking, fine-grained entailment relations, word embeddings, and style classification

What it is about

PPDB 1.0 ranked its pivot-extracted paraphrases with a fixed heuristic, and this paper replaces the ranking with a discriminative model. The model is trained on 26,000 human judgments of phrase pairs, sampled evenly across the range of the pivot probability. Its features include the pivot probability, a rarity penalty for how often a pair was seen, and similarity from word embeddings. Its rank correlates better with the human judgments than the pivot probability (Spearman 0.41), the 1.0 heuristic (0.41) or the cosine similarity of word2vec vectors (0.46), with the supervised ranking highest of the four in the paper's figure. The release also adds fine-grained entailment labels and style annotations to each pair, and it has over 100 million paraphrases in total. The lesson is that a cheap pivot probability is a weak ranking, and that a small number of human ratings can fix much of that.

Ideas

• I1. A small count-based ranker for the translator: take a few hundred merges that the residual has already judged (kept or undone) and fit a logistic model on features the translator already has, such as alignment counts, the number of verses where the two words meet, and rarity. Store the weights in a BTreeMap and check that two runs agree.
• I2. Keep a fixed list of human-judged merges as a held-out check. A new lexicon rule must raise the correlation on that list, measured twice, before it is accepted. The check should be kept apart from the rules that were fitted to it.
• I3. Use an entailment direction as a feature of a merge. The paper's example of a noise merge is be with have; a one-way label would keep a merge only where meaning goes one way, which addresses the second open problem directly.

For Mova

The problems are open problems 2 and 5: which merges are noise, and number and person features that a merge should not lose. The ranker is a count-based stand-in for the paper's model; the word embeddings are not for Mova, only the idea that similarity can be counted from context. Summary only, as the licence is non-commercial.

Topics: Machine translation

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.3115/v1/p15-2070

@inproceedings{pavlick2015ppdb,
  title = {{PPDB 2.0: Better paraphrase ranking, fine-grained entailment relations, word embeddings, and style classification}},
  author = {Pavlick, Ellie and Rastogi, Pushpendre and Ganitkevitch, Juri and Durme, Benjamin Van and Callison-Burch, Chris},
  booktitle = {Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)},
  year = {2015},
  doi = {10.3115/v1/p15-2070},
  url = {https://doi.org/10.3115/v1/p15-2070}
}

References

28 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Colin Bannard and Chris Callison-Burch. 2005. Paraphrasing with bilingual parallel corpora. In ACL, pages 597–604. find our card
2. Islam Beltagy, Stephen Roller, Gemma Boleda, Katrin Erk, and Raymond J Mooney. 2014. Utexas: Natural language semantics using distributional semantics and probabilistic logic. In SemEval. find
3. Jonathan Berant, Ido Dagan, and Jacob Goldberger. 2011. Global learning of typed entailment rules. In ACL. find
4. Rahul Bhagat, Patrick Pantel, Eduard H Hovy, and Marina Rey. 2007. LEDIR: An unsupervised algorithm for learning directionality of inference rules. In EMNLP-CoNLL, pages 161–170. find
5. Johannes Bjerva, Johan Bos, Rob van der Goot, and Malvina Nissim. 2014. The meaning factory: Formal semantics for recognizing textual entailment and determining semantic similarity. In SemEval. find
6. Chris Callison-Burch. 2008. Syntactic constraints on paraphrases extracted from parallel corpora. In EMNLP, pages 196–205. find
7. Tsz Ping Chan, Chris Callison-Burch, and Benjamin Van Durme. 2011. Reranking bilingually extracted paraphrases using monolingual distributional similarity. In GEMS, pages 33–42. find
8. Paramveer S. Dhillon, Dean Foster, and Lyle Ungar. 2011. Multi-view learning of word embeddings via CCA. In NIPS. find
9. Manaal Faruqui, Jesse Dodge, Sujay K Jauhar, Chris Dyer, Eduard Hovy, and Noah A Smith. 2015. Retrofitting word vectors to semantic lexicons. In NAACL. find
10. Juri Ganitkevitch, Chris Callison-Burch, Courtney Napoles, and Benjamin Van Durme. 2011. Learning sentential paraphrases from bilingual parallel corpora for text-to-text generation. In EMNLP, pages 1168–1179. find
11. Juri Ganitkevitch, Benjamin Van Durme, and Chris Callison-Burch. 2013. PPDB: The paraphrase database. In NAACL-HLT, pages 758–764, Atlanta, Georgia, June. find our card
12. Lushan Han, Abhay Kashyap, Tim Finin, James Mayfield, and Jonathan Weese. 2013. Umbc ebiquity-core: Semantic textual similarity systems. In Proceedings of the Second Joint Conference on Lexical and Computational Semantics, volume 1, pages 44– 52. find
13. Yangfeng Ji and Jacob Eisenstein. 2013. Discriminative improvements to distributional sentence similarity. In EMNLP, pages 891–896. find
14. Bill MacCartney. 2009. Natural language inference. Ph.D. thesis, Citeseer. find
15. Prodromos Malakasiotis and Ion Androutsopoulos. 2011. A generate and rank approach to sentence paraphrasing. In EMNLP, pages 96–106. find
16. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. In Workshop at ICLR. find
17. Ellie Pavlick and Ani Nenkova. 2015. Inducing lexical style properties for paraphrase and genre differentiation. In NAACL. find
18. Ellie Pavlick, Johan Bos, Malvina Nissim, Charley Beller, Benjamin Van Durme, and Chris Callison-Burch. 2015. Adding semantics to data-driven paraphrasing. In ACL. find
19. Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word representation. Proceedings of the Empiricial Methods in Natural Language Processing (EMNLP 2014), 12. find
20. Pushpendre Rastogi, Benjamin Van Durme, and Raman Arora. 2015. Multiview LSA: Representation learning via generalized CCA. In NAACL. find
21. Rion Snow, Daniel Jurafsky, and Andrew Y Ng. 2004. Learning syntactic patterns for automatic hypernym discovery. In NIPS. Md. find
22. Arafat Sultan, Steven Bethard, and Tamara Sumner. 2014a. Back to basics for monolingual alignment: Exploiting word similarity and contextual evidence. Transactions of the Association for Computational Linguistics, 2:219–230. find
23. Md Arafat Sultan, Steven Bethard, and Tamara Sumner. 2014b. Dls@cu: Sentence similarity from word alignment. In SemEval. find
24. Wei Xu, Alan Ritter, Bill Dolan, Ralph Grishman, and Colin Cherry. 2012. Paraphrasing for style. In COLING. find our card
25. Wei Xu, Chris Callison-Burch, and Courtney Napoles. 2015. Problems in current text simplification research: New data can help. TACL. find
26. Xuchen Yao, Benjamin Van Durme, Chris Callison-Burch, and Peter Clark. 2013. Semi-markov phrase-based monolingual alignment. In EMNLP, pages 590–600. find
27. Mo Yu and Mark Dredze. 2014. Improving lexical embeddings with semantic knowledge. In ACL, volume 2, pages 545–550. find
28. Shiqi Zhao, Cheng Niu, Ming Zhou, Ting Liu, and Sheng Li. 2008. Combining multiple resources to improve SMT-based paraphrasing model. In ACL. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/pavlick-2015-ppdb-2-0-better-paraphrase/