Unsupervised Syntactically Controlled Paraphrase Generation with Abstract Meaning Representations

What it is about

AMRPG generates a paraphrase that follows a requested syntactic form without any paraphrase pairs. It encodes the input twice: an AMR graph gives a semantic embedding, and the constituency parse gives a syntactic embedding. A decoder learns to reconstruct the input from the two. At test time the syntactic embedding of a different sentence, or of a template, is used, so the output keeps the meaning and takes the requested form. Training uses only the source sentences of ParaNMT. Among unsupervised systems, AMRPG gives the best syntactic control and the best BLEU on ParaNMT, Quora, MRPC and PAN test sets, and it beats the earlier unsupervised baseline SynPG. Its outputs also serve as augmentation that improves robustness on GLUE tasks under syntactic attack. The gap to a supervised model trained on paraphrase pairs remains, and the authors say it narrows when the model is fine-tuned on target-domain sources.

Ideas

• I1. Count-based disentanglement. The paper separates meaning (AMR) from form (parse). In the meta-world, keep frame content and tree shape in two counted tables, and recombine the frame of one sentence with the shape of another, choosing shapes by counts per predicate arity. Check each output by exact frame match. Where: tools/translate and tools/roundtrip.
• I2. Shape-control measure. Given a requested UD shape, measure the share of linearised outputs that realise it. This is the paper's control accuracy, computed without any learned model, and it tests en/src/lin.rs directly.
• I3. Stability under shape change. Vary the tree shape of the translator's test verses and check whether the chosen words stay the same. Unstable word choice under reshaping signals a reader or encoder that depends on attachment noise, which is open problem 1.

For Mova

The idea of separating semantics from syntax maps onto the meta-world with its tree heads. The paper's encoders and decoder are not for Mova. Its controls (I2, I3) are the useful part, because they can run on the existing linearizer and reader, with two runs for determinism.

Topics: Machine translation

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/2022.findings-emnlp.111

@inproceedings{huang2022unsupervised,
  title = {{Unsupervised Syntactically Controlled Paraphrase Generation with Abstract Meaning Representations}},
  author = {Huang, Kuan-Hao and Iyer, Varun and Kumar, Anoop and Venkatapathy, Sriram and Chang, Kai-Wei and Galstyan, Aram},
  booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2022},
  year = {2022},
  doi = {10.18653/v1/2022.findings-emnlp.111},
  url = {https://doi.org/10.18653/v1/2022.findings-emnlp.111}
}

References

34 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Xuefeng Bai, Yulong Chen, Linfeng Song, and Yue Zhang. 2021. Semantic representation for dialogue modeling. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (ACL/IJCNLP). find
2. Laura Banarescu, Claire Bonial, Shu Cai, Madalina Georgescu, Kira Griffitt, Ulf Hermjakob, Kevin Knight, Philipp Koehn, Martha Palmer, and Nathan Schneider. 2013. Abstract meaning representation for sembanking. In Proceedings of the 7th Linguistic Annotation Workshop and Interoperability with Discourse (LAW-ID@ACL). find our card
3. Yu Bao, Hao Zhou, Shujian Huang, Lei Li, Lili Mou, Olga Vechtomova, Xin-Yu Dai, and Jiajun Chen. 2019. Generating sentences from disentangled syntactic and semantic spaces. In Proceedings of the 57th Conference of the Association for Computational Linguistics (ACL). find
4. Regina Barzilay and Lillian Lee. 2003. Learning to paraphrase: An unsupervised approach using multiple-sequence alignment. In Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT). find our card
5. Michele Bevilacqua, Rexhina Blloshmi, and Roberto Navigli. 2021. One SPRING to rule them both: Symmetric AMR semantic parsing and generation without a complex pipeline. In Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI). find
6. Deng Cai and Wai Lam. 2020. AMR parsing via graph-sequence iterative inference. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL). find
7. Yue Cao and Xiaojun Wan. 2020. Divgan: Towards diverse paraphrase generation via diversified generative adversarial network. In Findings of the Association for Computational Linguistics: (EMNLP-Findings). find
8. Ziqiang Cao, Chuwei Luo, Wenjie Li, and Sujian Li. 2017. Joint copying and restricted generation for paraphrase. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI). find
9. Mingda Chen, Qingming Tang, Sam Wiseman, and Kevin Gimpel. 2019. Controllable paraphrase generation with a syntactic exemplar. In Proceedings of the 57th Conference of the Association for Computational Linguistics (ACL). find
10. Jishnu Ray Chowdhury, Yong Zhuang, and Shuyi Wang. 2022. Novelty controlled paraphrase generation with retrieval augmented conditional prompt tuning. In Proceedings of the Thirty-Sixth AAAI Conference on Artificial Intelligence (AAAI). find
11. Bill Dolan, Chris Quirk, and Chris Brockett. 2004. Unsupervised construction of large paraphrase corpora: Exploiting massively parallel news sources. In 20th International Conference on Computational Linguistics (COLING). find
12. Yao Fu, Yansong Feng, and John P. Cunningham. 2019. Paraphrase generation with latent bag of words. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019 (NeurIPS). find
13. Silin Gao, Yichi Zhang, Zhijian Ou, and Zhou Yu. 2020. Paraphrase augmented task-oriented dialog generation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL). sequence-to-sequence models for parsing and generation. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL). find
14. Ashutosh Kumar, Kabir Ahuja, Raghuram Vadapalli, and Partha P. Talukdar. 2020. Syntax-guided controlled generation of paraphrases. Transactions of the Association for Computational Linguistics, 8:330– 345. find
15. Ashutosh Kumar, Satwik Bhattamishra, Manik Bhandari, and Partha P. Talukdar. 2019. Submodular optimization-based diverse paraphrasing and its effectiveness in data augmentation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT). find
16. Zhe Lin and Xiaojun Wan. 2021. Pushing paraphrase away from original sentence: A multi-round paraphrase generation approach. In Findings of the Association for Computational Linguistics (ACL/IJCNLP-Findings). find
17. Mingtong Liu, Erguang Yang, Deyi Xiong, Yujie Zhang, Yao Meng, Changjian Hu, Jinan Xu, and Yufeng Chen. 2020. A learning-exploring method to generate diverse paraphrases with multi-objective deep reinforcement learning. In Proceedings of the 28th International Conference on Computational Linguistics (COLING). find
18. Tanya Goyal and Greg Durrett. 2020. Neural syntactic preordering for controlled paraphrase generation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL). find
19. Nitin Madnani, Joel R. Tetreault, and Martin Chodorow. 2012. Re-examining machine translation metrics for paraphrase identification. In Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT). find
20. Ankush Gupta, Arvind Agarwal, Prawaan Singh, and Piyush Rai. 2018. A deep generative framework for paraphrase generation. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence (AAAI). find
21. Christopher D. Manning, Mihai Surdeanu, John Bauer, Jenny Rose Finkel, Steven Bethard, and David Mc-Closky. 2014. The stanford corenlp natural language processing toolkit. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, System Demonstrations. find
22. Kuan-Hao Huang and Kai-Wei Chang. 2021. Generating syntactically controlled paraphrases without using annotated parallel pairs. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics (EACL). find
23. Leonardo F. R. Ribeiro, Martin Schmitt, Hinrich Schütze, and Iryna Gurevych. 2021. Investigating pretrained language models for graph-to-text generation. In Proceedings of the 3rd Workshop on Natural Language Processing for Conversational AI. find
24. Shankar Iyer, Nikhil Dandekar, and Korn el Csernai. 2017. First quora dataset release: Question pairs. data.quora.com. find
25. Aurko Roy and David Grangier. 2019. Unsupervised paraphrasing without translation. In Proceedings of the 57th Conference of the Association for Computational Linguistics (ACL). find
26. Mohit Iyyer, John Wieting, Kevin Gimpel, and Luke Zettlemoyer. 2018. Adversarial example generation with syntactically controlled paraphrase networks. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT). find
27. Ioannis Konstas, Srinivasan Iyer, Mark Yatskar, Yejin Choi, and Luke Zettlemoyer. 2017. Neural AMR: Mrinmaya Sachan and Eric P. Xing. 2016. Machine comprehension using rich semantic representations. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL). find
28. Jiao Sun, Xuezhe Ma, and Nanyun Peng. 2021. AESOP: paraphrase generation with adaptive syntactic control. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP). find
29. Yufei Tian, Arvind Krishna Sridhar, and Nanyun Peng. 2021. Hypogen: Hyperbole generation with commonsense and counterfactual knowledge. In Findings of the Association for Computational Linguistics: (EMNLP-Findings). find
30. Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. 2019. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In 7th International Conference on Learning Representations (ICLR). find
31. John Wieting and Kevin Gimpel. 2018. Paranmt-50m: Pushing the limits of paraphrastic sentence embeddings with millions of machine translations. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL). find
32. Xinyuan Zhang, Yi Yang, Siyang Yuan, Dinghan Shen, and Lawrence Carin. 2019. Syntax-infused variational autoencoder for text generation. In Proceedings of the 57th Conference of the Association for Computational Linguistics (ACL). find
33. Zixuan Zhang, Nikolaus Nova Parulian, Heng Ji, Ahmed Elsayed, Skatje Myers, and Martha Palmer. 2021. Fine-grained information extraction from biomedical literature based on knowledge-enriched abstract meaning representation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (ACL/IJCNLP). find
34. Qiji Zhou, Yue Zhang, Donghong Ji, and Hao Tang. 2020. AMR parsing with latent structural information. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL). find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/huang-2022-unsupervised-syntactically-controlled-paraphrase-generation/