Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfer

What it is about

The paper transfers an attribute such as sentiment or style without parallel data. It assumes that a sentence's attribute shows up as a few distinctive phrases, such as a comparative phrase for a negative review. The method has three steps. Delete: score each n-gram by how much more often it appears in one attribute class than in the other, using smoothed relative frequencies, and remove those above a threshold, leaving the content words. Retrieve: find the sentence in the target class whose content is most similar to the remaining words, by TF-IDF overlap or embedding distance. Generate: combine the content with the target phrases, either by a fixed template or by a small neural model. The threshold can be changed at test time to control how many markers are removed. On human ratings the best variant yields grammatical and appropriate outputs on about 22 percent more inputs than the best earlier system, averaged over three datasets: Yelp and Amazon reviews, and image captions.

Ideas

• I1. Find the author's markers by salience. For an author against the standard, compute the smoothed ratio of n-gram counts and keep the top ones (thou, hath, unto). This gives the candidate list for the author delta without a hand-written list, and a threshold tuned on dev sets the cut.
• I2. Report a curve, not one number. Vary the marker threshold and report content match against style match. The curve fits exact mode (strict) and normal mode (loose) better than a single score.
• I3. Retrieve then fill for unseen concepts. For a slot with no known chunk, retrieve a sentence from the target author with similar meta tokens and take its chunk. This is a count-based answer to open problem 3, which asks for trees rather than strings.
• I4. Keep the template fill as a deterministic route to compare against the encoder. The paper's template baseline is cheap to build and gives a floor for each delta.

For Mova

The code is tools/translate (profile.rs for the chunk choice, measure.rs for the threshold curve) and en/src/morph.rs for marker morphology. The neural generator is not for Mova. The marker-salience step and the threshold curve are count-based and fit the author delta directly, addressing open problem 6.

Topics: Machine translation

Cite

The paper itself, not this card. BibTeX from the library's record of the work. arXiv 1804.06437

@misc{li2018delete,
  title = {{Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfer}},
  author = {Li, Juncen and Jia, Robin and He, He and Liang, Percy},
  year = {2018},
  eprint = {1804.06437},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1804.06437}
}

References

23 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. M. Arjovsky and L. Bottou. 2017. Towards principled methods for training generative adversarial networks. In International Conference on Learning Representations (ICLR). ↗
2. M. Artetxe, G. Labaka, E. Agirre, and K. Cho. 2017. Unsupervised neural machine translation. arXiv preprint arXiv:1710.11041 . ↗
3. K. Bousmalis, N. Silberman, D. Dohan, D. Erhan, and D. Krishnan. 2017. Unsupervised pixel-level domain adaptation with generative adversarial networks. In Computer Vision and Pattern Recognition (CVPR). find
4. Z. Fu, X. Tan, N. Peng, D. Zhao, and R. Yan. 2018. Style transfer in text: Exploration and evaluation. In Association for the Advancement of Artificial Intelligence (AAAI). find
5. C. Gan, Z. Gan, X. He, J. Gao, and L. Deng. 2017. Stylenet: Generating attractive visual captions with styles. In Computer Vision and Pattern Recognition (CVPR). find
6. A. Globerson and S. Roweis. 2006. Nightmare at test time: robust learning by feature deletion. In International Conference on Machine Learning (ICML). pages 353–360. find
7. I. Goodfellow, D. Warde-farley, M. Mirza, A. Courville, and Y. Bengio. 2013. Maxout networks. In International Conference on Machine Learning (ICML). pages 1319–1327. find
8. K. Guu, T. B. Hashimoto, Y. Oren, and P. Liang. 2018. Generating sentences by editing prototypes. Transactions of the Association for Computational Linguistics (TACL) 0. ↗
9. R. He and J. McAuley. 2016. Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. In World Wide Web (WWW). ↗
10. Z. Hu, Z. Yang, X. Liang, R. Salakhutdinov, and E. P. Xing. 2017. Toward controlled generation of text. In International Conference on Machine Learning (ICML). ↗
11. Z. Jun-Yan, P. Taesung, I. Phillip, and E. A. A. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. In International Conference on Computer Vision (ICCV). find
12. G. Lample, L. Denoyer, and M. Ranzato. 2017. Unsupervised machine translation using monolingual corpora only. arXiv preprint arXiv:1711.00043 . ↗
13. I. Melnyk, C. N. dos Santos, K. Wadhawan, I. Padhi, and A. Kumar. 2017. Improved neural text attribute transfer with non-parallel data. arXiv preprint arXiv:1711.09395 . ↗
14. M. L. Newman, J. W. Pennebaker, D. S. Berry, and J. M. Richards. 2003. Lying words: Predicting deception from linguistic styles. Personality and Social Psychology Bulletin 29. ↗
15. E. Pavlick and J. Tetreault. 2017. An empirical analysis of formality in online communication. Transactions of the Association for Computational Linguistics (TACL) 4. ↗ our card
16. M. Recasens, C. Danescu-Niculescu-Mizil, and D. Jurafsky. 2013. Linguistic models for analyzing and detecting biased language. In Association for Computational Linguistics (ACL). find
17. T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and X. Chen. 2016. Improved techniques for training gans. In Advances in Neural Information Processing Systems (NIPS). ↗
18. R. Schwartz, M. Sap, Y. Konstas, L. Zilles, Y. Choi, and N. A. Smith. 2017. The effect of different writing tasks on linguistic style: A case study of the ROC story cloze task. In Computational Natural Language Learning (CoNLL). find
19. T. Shen, T. Lei, R. Barzilay, and T. Jaakkola. 2017. Style transfer from non-parallel text by cross-alignment. In Advances in Neural Information Processing Systems (NIPS). ↗ our card
20. P. Vincent, H. Larochelle, Y. Bengio, , and P. Manzagol. 2008. Extracting and composing robust features with denoising autoencoders. In International Conference on Machine Learning (ICML). find
21. S. Wang and C. D. Manning. 2012. Baselines and bigrams: Simple, good sentiment and topic classification. In Association for Computational Linguistics (ACL). find
22. M. D. Zeiler. 2012. ADADELTA: an adaptive learning rate method. arXiv preprint arXiv:1212.5701 . ↗
23. J. Zhao, Y. Kim, K. Zhang, A. M. Rush, and Y. Le-Cun. 2018. Adversarially regularized autoencoders. In International Conference on Learning Representations (ICLR). find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/li-2018-delete-retrieve-generate-simple-approach/