Mova0.5 pre-release⌕
☰
home / docs / papers / Machine translation / Evaluating prose style transfer with the Bible

Evaluating prose style transfer with the Bible

What it is about

The paper proposes the Bible as a style-transfer corpus. It collects 34 English versions, of which eight can be redistributed, and aligns them by book, chapter and verse, so no automatic sentence alignment is needed. That gives over 33 million verse pairs in the full data and a standard train, dev and test split by book. Two baselines are trained: an attention-based encoder-decoder with a subword vocabulary and a target-version tag prepended to each source verse, and a phrase-based statistical system (Moses) trained on several source and target pairings. Both are scored with BLEU, which measures similarity to the target, and PINC, which measures how different the output is from the source. Moses changes the input less and leans on exact pairings; the neural model gains from pooling many versions. For the KJV-to-ASV pair, Moses reaches a BLEU of about 71 and the neural model about 66, with the neural model showing much higher PINC.

Ideas

For Mova

The code is tools/translate (hebrew.rs and profile.rs for the data side, measure.rs for the metrics). The paper's models are not for Mova, but its corpus design is: verse-aligned versions are the cleanest source of author pairs the translator has. The PINC pairing and the pooling test address open problems 6 and the measures for normal mode. Only the summary may be used, since the paper is read-only in the library.

Topics: Machine translation

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.1098/rsos.171920 · arXiv 1711.04731

@misc{carlson2017evaluating,
  title = {{Evaluating prose style transfer with the Bible}},
  author = {Carlson, Keith and Riddell, Allen and Rockmore, Daniel},
  year = {2017},
  doi = {10.1098/rsos.171920},
  eprint = {1711.04731},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.1098/rsos.171920}
}

References

53 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Pang B, Lee L. 2008 Opinion mining and sentiment analysis. Found. Trends w Inf. Retr. 2, 1 –135. (doi:10.1561/150 0000011) R. Soc. open sci. 5: 171920 ↗
  2. Hughes JM, Foti NJ, Krakauer DC, Rockmore DN. 2012 Quantitative patterns of stylistic influence in the evolution of literature. Proc. Natl Acad. Sci. USA 109, 7682–7686. (doi:10.1073/pnas.1115407109) ↗
  3. Carlson K, Livermore MA, Rockmore D. 2015 A quantitative analysis of writing style on the US supreme court. SSRN. (doi:10.2139/ssrn. 2554516) ↗
  4. Sheikha FA, Inkpen D. 2010 Automatic classification of documents by formality. In 2010 Int. Conf. on Natural Language Processing and Knowledge Engineering (NLP-KE), Beijing, China, August, pp. 1–5. IEEE. (doi:10.1109/nlpke.2010. 5587767) ↗
  5. Specia L. 2010 Translating from complex to simplified sentences. In Computational Processing of the Portuguese Language (eds TAS Pardo, A Branco, A Klautau, R Vieira, VL Strube de Lima), pp. 30 –39. Berlin, Germany: Springer. find
  6. Sennrich R, Haddow B, Birch A. 2016 Controlling politeness in neural machine translation via side constraints. In Proc. of the 2016 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego, CA, June, pp. 35 –40. Association for Computational Linguistics. (doi:10.18653/v1/n16-1005) ↗
  7. Yamagishi H, Kanouchi S, Sato T, Komachi M. 2016 Controlling the voice of a sentence in Japanese-to-English neural machine translation. In Proc. of the 3rd Workshop on Asian Translation (WAT2016), Osaka, Japan, December, pp. 203–210. The COLING 2016 Organizing Committee. find
  8. Yatskar M, Pang B, Danescu-Niculescu-Mizil C, Lee L. 2010 For the sake of simplicity: unsupervised extraction of lexical simplifications from Wikipedia. In Human Language Technologies: The 2010 Annual Conf. of the North American Chapter of the Association for Computational Linguistics, Los Angeles, CA, June, pp. 365–368. Association for Computational Linguistics. find
  9. Ficler J, Goldberg Y. 2017 Controlling linguistic style aspects in neural language generation. In Proc. of the Workshop on Stylistic Variation, Copenhagen, Denmark, September, pp. 94 –104. Association for Computational Linguistics. (doi:10.18653/v1/w17-4912) ↗
  10. Xu W, Ritter A, Dolan WB, Grishman R, Cherry C. 2012 Paraphrasing for style. In 24th Int. Conf. on Computational Linguistics, COLING 2012, Mumbai, India, December, pp. 2899– 2914. The COLING 2012 Organizing Committee. find our card
  11. Jhamtani H, Gangal V, Hovy E, Nyberg E. 2017 Shakespearizing modern language using copy-enriched sequence-to-sequence models. In Proc. of the Workshop on Stylistic Variation, Copenhagen, Denmark, September, pp. 10– 19. Association for Computational Linguistics. (doi:10.18653/v1/w17-4902) ↗ our card
  12. Hu Z, Yang Z, Liang X, Salakhutdinov R, Xing EP. 2017 Toward controlled generation of text. In Int. Conf. on Machine Learning, Sydney, Australia, August, pp. 1587 – 1596. PMLR. find
  13. Shen T, Lei T, Barzilay R, Jaakkola T. 2017 Style transfer from non-parallel text by cross-alignment. In Advances in neural information processing systems (eds I Guyon, UV Luxburg, S Bengio, H Wallach, R Fergus, S Vishwanathan, R Garnett), pp. 6833 –6844. Red Hook, NY: Curran Associates. find our card
  14. Li J, Jia R, He H, Liang P. 2018 Delete, retrieve, generate: a simple approach to sentiment and style transfer. In Proc. of the 2018 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1, New Orleans, LA, June, pp. 1865– 1874. Association for Computational Linguistics. (doi:10.18653/v1/n18-1169) ↗ our card
  15. Chen DL, Dolan WB. 2011 Collecting highly parallel data for paraphrase evaluation. In Proc. of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1, pp. 190– 200. Association for Computational Linguistics. find
  16. Wubben S, Van Den Bosch A, Krahmer E. 2012 Sentence simplification by monolingual machine translation. In Proc. of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers-Volume 1, pp. 1015 –1024. Association for Computational Linguistics. find our card
  17. Kajiwara T, Komachi M. 2016 Building a monolingual parallel corpus for text simplification using sentence similarity based on alignment between word embeddings. In Proc. of COLING 2016, the 26th Int. Conf. on Computational Linguistics: Technical Papers, pp. 1147– 1158. find
  18. Rao S, Tetreault J. 2018 Dear sir or madam, may I introduce the YAFC corpus: corpus, benchmarks and metrics for formality style transfer. In Proc. of the 2018 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1, New Orleans, LA, June, pp. 129–140. Association for Computational Linguistics. (doi:10.18653/v1/ n18-1012) find
  19. Xu W, Napoles C, Pavlick E, Chen Q, Callison-Burch C. 2016 Optimizing statistical machine translation for text simplification. Trans. Assoc. Comput. Linguist. 4, 401– 415. find our card
  20. Fu Z, Tan X, Peng N, Zhao D, Yan R. 2017 Style transfer in text: exploration and evaluation. Preprint. (http://arxiv.org/abs/171106861). ↗
  21. Artetxe M, Labaka G, Agirre E, Cho K. 2017 Unsupervised neural machine translation. Preprint. (http://arxiv.org/abs/171011041). ↗
  22. Lample G, Denoyer L, Ranzato M. 2017 Unsupervised machine translation using monolingual corpora only. Preprint. (http://arxiv.org/abs/171100043). ↗
  23. Koehn P et al. 2007 Moses: open source toolkit for statistical machine translation. In Proc. of the 45th Annual Meeting of the ACL on Interactive Poster and Demonstration Sessions, Prague, Czech Republic, June, pp. 177– 180. Association for Computational Linguistics. (doi:10.3115/ 1557769.1557821) find
  24. Papineni K, Roukos S, Ward T, Zhu WJ. 2002 BLEU: a method for automatic evaluation of machine translation. In Proc. of the 40th Annual Meeting on Association for Computational Linguistics, Philadelphia, PA, July, pp. 311 –318. Association for Computational Linguistics. (doi:10.3115/1073083.1073135) ↗ our card
  25. Zhu Z, Bernhard D, Gurevych I. 2010 A monolingual tree-based translation model for sentence simplification. In Proc. of the 23rd Int. Conf. on Computational Linguistics, Beijing, China, August, pp. 1353– 1361. Association for Computational Linguistics. find our card
  26. Hwang W, Hajishirzi H, Ostendorf M, Wu W. 2015 Aligning sentences from standard wikipedia to simple wikipedia. In Proc. of the 2015 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Denver, CO, May–June, pp. 211– 217. Association for Computational Linguistics. (doi:10.3115/v1/n15-1022) ↗
  27. Xu W, Callison-Burch C, Napoles C. 2015 Problems in current text simplification research: new data can help. Trans. Assoc. Comput. Linguist. 3, 283– 297. find
  28. Johnson M et al. 2017 Google’s multilingual neural machine translation system: enabling zero-shot translation. Trans. Assoc. Comput. Linguist. 5, 339– 351. find
  29. Cho K, van Merrienboer B, Ç Gülçehre, Bougares F, Schwenk H, Bengio Y. 2014 Learning phrase representations using RNN encoder-decoder for statistical machine translation. In Proc. of the 2014 Conf. on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar, October, pp. 1724 –1734. Association for Computational Linguistics. (doi:10.3115/v1/d14-1179) ↗
  30. Hochreiter S, Schmidhuber J. 1997 Long short-term memory. Neural Comput. 9, 1735 –1780. find
  31. Graves A. 2013 Generating sequences with recurrent neural networks. CoRR. (http://arxiv.org/abs/1308.0850). ↗
  32. Sutskever I, Vinyals O, Le QV. 2014 Sequence to sequence learning with neural networks. In Advances in neural information processing systems 27 (eds Z Ghahramani, M Welling, C Cortes, ND Lawrence, KQ Weinberger), pp. 3104 –3112. Red Hook, NY: Curran Associates. find
  33. Bahdanau D, Cho K, Bengio Y. 2014 Neural machine translation by jointly learning to align and translate. CoRR. (http://arxiv.org/abs/1409.0473). ↗
  34. Firat O, Cho K, Bengio Y. 2016 Multi-way, multilingual neural machine translation with a shared attention mechanism. In Proc. of the 2016 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego, CA, June, pp. 866–875. Association for Computational Linguistics. (doi:10.18653/v1/n16-1101) ↗
  35. Quirk C, Brockett C, Dolan WB. 2004 Monolingual machine translation for paraphrase generation. In Proceedings of EMNLP 2004, Barcelona, Spain, July, pp. 142–149. Association for Computational Linguistics. find our card
  36. Wubben S, Van Den Bosch A, Krahmer E. 2010 Paraphrase generation as monolingual translation: data and evaluation. In Proc. of the 6th Int. Natural Language Generation Conf., Meath, Ireland, July, pp. 203–207. Association for Computational Linguistics. find
  37. Kabbara J, Cheung JCK. 2016 Stylistic transfer in natural language generation systems using recurrent neural networks. In Proc. of the Workshop on Uphill Battles in Language Processing: Scaling Early Achievements to Robust Methods, Austin, TX, November, pp. 43–47. Association for Computational Linguistics. (doi:10. 18653/v1/w16-6010) R. Soc. open sci. 5: 171920 find
  38. Wang T, Chen P, Amaral KM, Qiang J. 2016 An experimental study of LSTM encoder-decoder model for text simplification. CoRR. (http://arxiv.org/abs/1609.03663). ↗
  39. Coster W, Kauchak D. 2011 Simple English Wikipedia: a new text simplification task. In Proc. of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: Short Papers-Volume 2, Portland, OR, June, pp. 665–669. Association for Computational Linguistics. find
  40. Och FJ. 2003 Minimum error rate training in statistical machine translation. In Proc. of the 41st Annual Meeting on Association for Computational Linguistics-Volume 1, Sapporo, Japan, pp. 160–167. Association for Computational Linguistics. (doi:10.3115/ 1075096.1075117) find
  41. Heafield K. 2011 KenLM: faster and smaller language model queries. In Proc. of the Sixth Workshop on Statistical Machine Translation, Edinburgh, UK, July, pp. 187 –197. Association for Computational Linguistics. find
  42. Dong D, Wu H, He W, Yu D, Wang H. 2015 Multi-task learning for multiple language translation. In Proc. of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th Int. Joint Conf. on Natural Language Processing (Volume 1: Long Papers), Beijing, China, July, pp. 1723 – 1732. Association for Computational Linguistics. (doi:10.3115/v1/ p15-1166) find
  43. Vinyals O, Kaiser L, Koo T, Petrov S, Sutskever I, Hinton GE. 2014 Grammar as a foreign language. In Advances in neural information processing systems 28 (eds C Cortes, ND Lawrence, DD Lee, M Sugiyama, R Garnett), pp. 2773– 2781. Red Hook, NY: Curran Associates. find
  44. Prakash A, Hasan SA, Lee K, Datla VV, Qadir A, Liu J, Farri O. 2016 Neural paraphrase generation with stacked residual LSTM networks. In Proc. of COLING 2016, the 26th Int. Conf. on Computational Linguistics: Technical Papers, Osaka, Japan, December, pp. 2923 – 2934. The COLING 2016 Organizing Committee. find
  45. Sennrich R, Haddow B, Birch A. 2015 neural machine translation of rare words with subword units. In Proc. of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Berlin, Germany, August, pp. 1715– 1725. Association for Computational Linguistics. (doi:10.18653/v1/ p16-1162) find
  46. Zaremba W, Sutskever I, Vinyals O. 2014 Recurrent neural network regularization. CoRR. (http://arxiv.org/abs/1409.2329). ↗
  47. Schuster M, Paliwal KK. 1997 Bidirectional recurrent neural networks. IEEE Trans. Signal Process. 45, 2673 – 2681. find
  48. He K, Zhang X, Ren S, Sun J. 2015 Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR, Las Vegas, NV, June, pp. 770– 778. IEEE. (doi:10.1109/CVPR.2016.90) ↗
  49. Kingma DP, Ba J. 2014 Adam: a method for stochastic optimization. CoRR. (http://arxiv.org/abs/1412.6980). ↗
  50. Britz D, Goldie A, Luong T, Le Q. 2017 Massive Exploration of Neural Machine Translation Architectures. In Proc. of the 2017 Conf. on Empirical Methods in Natural Language Processing, Copenhagen, Denmark, September, pp. 1442 – 1451. Association for Computational Linguistics. (doi:10.18653/v1/ d17-1151) find
  51. Abadi M et al. 2015 TensorFlow: large-scale machine learning on heterogeneous systems. Software available from tensorflow.org. See http://tensorflow.org/. ↗
  52. Tu Z, Lu Z, Liu Y, Liu X, Li H. 2016 Modeling coverage for neural machine translation. In Proc. of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Berlin, Germany, August, pp. 76– 85. Association for Computational Linguistics. (doi:10.18653/v1/p16-1008) ↗
  53. Merity S, Xiong C, Bradbury J, Socher R. 2016 Pointer sentinel mixture models. Preprint. (http://arxiv.org/abs/160907843) ↗

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/carlson-2017-evaluating-prose-style-transfer-bible/