Learning Scalar Adjective Intensity from Paraphrases

What it is about

The paper orders scalar adjectives (warm, hot, scalding) by intensity from paraphrase pairs such as "really hot" ↔ "scalding". It uses 36,756 such pairs from the Paraphrase Database over 3,704 adjectives. Intensifying adverbs are found by bootstrapping from base and comparative forms (610 adverbs), and a logistic regression over adverb features scores each pair. This score is combined, by back-off, with a pattern score and the SO-CAL lexicon (2,826 adjectives, weights -5 to 5). The combined scores feed a global ranking (MILP) and a yes/no rule for indirect answers on 123 question-answer pairs. Paraphrase evidence gives much higher coverage; on the answer task the all-"YES" baseline still has the highest accuracy (0.691), and the best combination reaches 0.642.

Ideas

• I1. en/src/dict.rs: find intensifiers without a list. Take the base and comparative pairs in our own lexicon (hard → harder), and mark each adverb that links a base form to a stronger one, in a deterministic count table (BTreeMap). Check the list by hand against 50 known intensifiers and 50 de-intensifiers.
• I2. Global ordering without an ILP: sum the signed pairwise scores of a scale and sort (a Borda-style rule), breaking ties by the lemma in order. Compare its pairwise accuracy with the paper's MILP on the school adjective scales, to see whether the heavy solver is needed.
• I3. prag (polarity): the paper's decision list for indirect answers, with an explicit UNCERTAIN output when an adjective is missing from the vocabulary, and a negation flip. Report precision, recall and F for each output, not only accuracy, since the all-"YES" baseline wins on accuracy.
• I4. Coverage as a reported number: for each intensity rule, report the share of pairs it covers and its accuracy on those pairs, separately. The paper shows that a precise, low-coverage source and a noisy, high-coverage one are complementary; Mova should measure that split rather than assume it.

For Mova

The adjective scale and intensifier list belong in en/src/dict.rs and 15. Adjectives and adverbs. The polarity decision for indirect answers belongs with prag and in 31. Speech acts and 26. Questions. Measures: intensifier list precision on a hand-checked set, the pairwise accuracy of the Borda ordering against a gold scale, and precision, recall and F for the answer-polarity rule, with the all-"YES" baseline as the negative control. Run twice for determinism.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/d18-1202

@inproceedings{cocos2018learning,
  title = {{Learning Scalar Adjective Intensity from Paraphrases}},
  author = {Cocos, Anne and Wharton, Skyler and Pavlick, Ellie and Apidianaki, Marianna and Callison-Burch, Chris},
  booktitle = {Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing},
  year = {2018},
  doi = {10.18653/v1/d18-1202},
  url = {https://doi.org/10.18653/v1/d18-1202}
}

References

28 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Theodore W Anderson and Donald A Darling. 1954. A test of goodness of fit. Journal of the American statistical association, 49(268):765–769. find
2. Thorsten Brants and Alex Franz. 2009. Web 1T 5-gram, 10 European languages version 1. Linguistic Data Consortium, Philadelphia. find
3. Ido Dagan, Oren Glickman, and Bernardo Magnini. 2006. Machine Learning Challenges. Evaluating Predictive Uncertainty, Visual Object Classification, and Recognising Tectual Entailment: First PASCAL Machine Learning Challenges Workshop, MLCW 2005, Southampton, UK, April 11-13, 2005, Revised Selected Papers, chapter The PASCAL Recognising Textual Entailment Challenge. Springer Berlin Heidelberg, Berlin, Heidelberg. find
4. Rotem Dror, Gili Baumer, Segev Shlomov, and Roi Reichart. 2018. The hitchhiker’s guide to testing statistical significance in natural language processing. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL), pages 1383–1392. find
5. Christiane Fellbaum. 1998. WordNet. Wiley Online Library. find
6. Ronald Aylmer Fisher. 1935. The design of experiments. find
7. Juri Ganitkevitch, Benjamin Van Durme, and Chris Callison-Burch. 2013. PPDB: The paraphrase database. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL HLT), pages 758–764, Atlanta, Georgia. find our card
8. Derek Gross and Katherine J Miller. 1990. Adjectives in wordnet. International Journal of lexicography, 3(4):265–277. find
9. Vasileios Hatzivassiloglou and Kathleen R. McKeown. 1993. Towards the automatic identification of adjectival scales: Clustering adjectives according to meaning. In Proceedings of the 31st Annual Meeting on Association for Computational Linguistics (ACL), pages 172–182, Columbus, Ohio. find
10. Marti A Hearst. 1992. Automatic acquisition of hyponyms from large text corpora. In Proceedings of the 14th conference on Computational linguistics-Volume 2 (COLING), pages 539–545, Nantes, France. find
11. Hans Kamp and Barbara Partee. 1995. Prototype theory and compositionality. Cognition, 57(2):129– 191. find
12. Joo-Kyung Kim and Marie-Catherine de Marneffe. 2013. Deriving adjectival scales from continuous space word representations. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1625–1630, Seattle, Washington. find
13. Marie-Catherine de Marneffe, Christopher D. Manning, and Christopher Potts. 2010. Was it good? It was provocative. Learning the meaning of scalar adjectives. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics (ACL), pages 167–176, Uppsala, Sweden. find our card
14. Gerard de Melo and Mohit Bansal. 2013. Good, great, excellent: Global inference of semantic intensities. Transactions of the Association for Computational Linguistics, 1:279–290. find
15. George A. Miller. 1995. Wordnet: A lexical database for english. Commun. ACM, 38(11):39–41. ↗ our card
16. Marcin Morzycki. 2015. Modification. Cambridge University Press. find
17. Bo Pang, Lillian Lee, et al. 2008. Opinion mining and sentiment analysis. Foundations and Trends in Information Retrieval, 2(1–2):1–135. find
18. Ellie Pavlick, Pushpendre Rastogi, Juri Ganitkevitch, Benjamin Van Durme, and Chris Callison-Burch. 2015. PPDB 2.0: Better paraphrase ranking, fine-grained entailment relations, word embeddings, and style classification. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (ACL) (Volume 2: Short Papers), pages 425–430, Beijing, China. find our card
19. Sven Rill, J. vom Scheidt, Johannes Drescher, Oliver Schütz, Dirk Reinel, and Florian Wogenstein. 2012. A generic approach to generate opinion lists of phrases for opinion mining applications. In Proceedings of the First International Workshop on Issues of Sentiment Discovery and Opinion Mining (WISDOM), Beijing, China. find
20. Josef Ruppenhofer, Jasper Brandes, Petra Steiner, and Michael Wiegand. 2015. Ordering adverbs by their scaling effect on adjective intensity. In Proceedings of the International Conference Recent Advances in Natural Language Processing, pages 545–554, Hissar, Bulgaria. find
21. Josef Ruppenhofer, Michael Wiegand, and Jasper Brandes. 2014. Comparing methods for deriving intensity scores for adjectives. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics (EACL), Gothenburg, Sweden. find
22. Raksha Sharma, Mohit Gupta, Astha Agarwal, and Pushpak Bhattacharyya. 2015. Adjective intensity and sentiment analysis. In Proceedings of the 2015 Conference on Empirical Methods for Natural Language Processing (EMNLP), Lisbon, Portugal. find
23. Vera Sheinman, Christiane Fellbaum, Isaac Julien, Peter Schulam, and Takenobu Tokunaga. 2013. Large, huge or gigantic? Identifying and encoding intensity relations among adjectives in WordNet. Language resources and evaluation, 47(3):797–816. find
24. Vera Sheinman and Takenobu Tokunaga. 2009. Ad-jscales: Visualizing differences between adjectives for language learners. IEICE TRANSACTIONS on Information and Systems, 92(8):1542–1550. find
25. Chaitanya P. Shivade, Marie-Catherine de Marneffe, Eric Fosler-Lussier, and Albert M. Lai. 2015. Corpus-based discovery of semantic intensity scales. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics Human Language Technologies (NAACL HLT), pages 483–493, Denver, Colorado. find
26. Maite Taboada, Julian Brooke, Milan Tofiloski, Kimberly Voll, and Manfred Stede. 2011. Lexicon-based methods for sentiment analysis. Computational linguistics, 37(2):267–307. find
27. Bryan Wilkinson. 2017. Identifying and Ordering Scalar Adjectives Using Lexical Substitution. Ph.D. thesis, University of Maryland, Baltimore County. find
28. Bryan Wilkinson and Tim Oates. 2016. A gold standard for scalar adjectives. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC), Portoro, Slovenia. find our card

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/cocos-2018-learning-scalar-adjective-intensity-paraphrases/