Monolingual Machine Translation for Paraphrase Generation

What it is about

The paper treats paraphrase generation as translation within one language. Training pairs are sentences from news stories that different web sources wrote about the same event, clustered and aligned at sentence level. Word alignment uses the GIZA++ IBM models and an HMM, with alignment error rate 11.58% on 250 hand-aligned sentences (interrater agreement 93.1%). A monotone phrasal decoder then generates contextual replacements, and the paper argues that statistical MT tools are a route to better coverage and scale than hand-built paraphrase rules. Human judges rated the paraphrases as plausible more often than outputs of a WordNet synonym baseline, an unsupervised clustering baseline and the Barzilay and Lee system on the same 59-sentence set. A trigram language model did not help the WordNet baseline, which the authors take to mean that lexical choice, not fluency, is the bottleneck.

Ideas

• I1. Use the verse-aligned Bible translations as the monolingual parallel corpus, aligned with the IBM Model 1 already in tools/translate/ibm.rs. Report the alignment error rate on a hand-aligned sample of verses, as the paper does, before using the alignment for anything else.
• I2. A monotone decoder is the count form of the standard encoder: if the source order is kept by default, only local replacements are needed, and a phrase table counted by concept and neighbours gives each replacement its evidence.
• I3. Test whether a merge is justified by its context alone: compare the replacement table with and without the neighbour context, and drop merges that the context does not support. The paper's WordNet result suggests lexical choice is where the errors lie.

For Mova

The paper's data and its alignment step are close to the translator's verse pairs and its IBM alignment. The standard encoder in profile.rs and the lexicon in meta.rs are the places for the replacement table. Only counts are proposed here, not the neural decoders the paper also discusses. Summary only.

Topics: Machine translation

Cite

The paper itself, not this card. BibTeX from the library's record of the work.

@inproceedings{quirk2004monolingual,
  title = {{Monolingual Machine Translation for Paraphrase Generation}},
  author = {Quirk, Chris and Brockett, Chris and Dolan, William},
  booktitle = {Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing},
  year = {2004},
  url = {https://aclanthology.org/W04-3219/}
}

References

24 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. R. Barzilay and K. R. McKeown. 2001. Extracting Paraphrases from a parallel corpus. In Proceedings of the ACL/EACL. find our card
2. R. Barzilay and L. Lee. 2003. Learning to Paraphrase; an unsupervised approach using multiple-sequence alignment. In Proceedings of HLT/NAACL. find our card
3. P. Brown, S. A. Della Pietra, V. J. Della Pietra and R. L. Mercer. 1993. The Mathematics of Statistical Machine Translation. Computational Linguistics 19(2): 263-311. find
4. W. Dolan, C. Quirk and C. Brockett. 2004. Unsupervised Construction of Large Paraphrase Corpora: Exploiting Massively Parallel News Sources. To appear in Proceedings of COLING-2004. find
5. C. Fellbaum, ed. 1998. WordNet: An Electronic Lexical Database. MIT Press, Cambridge, MA. find
6. J. Goodman. 2002. JCLUSTER. Software available at http://research.microsoft.com/~joshuago/ ↗
7. A. Ibrahim. 2002. Extracting Paraphrases from Aligned Corpora. Master of Engineering Thesis, MIT. find
8. A. Ibrahim, B. Katz, and J. Lin. 2003. Extracting Structural Paraphrases from Aligned Monolingual Corpora. In Proceedings of the Second International Workshop on Paraphrasing (IWP 2003). Sapporo, Japan. find
9. R. Kneser and H. Ney. 1995. Improved backing-off for N-gram language modeling. In Proc. Int. Conf. on Acoustics, Speech and Signal Processing: 181-184. Detroit, MI. find
10. P. Koehn, F. Och, and D. Marcu. 2003. Statistical Phrase-Based Translation. In Proceedings of HLT/NAACL. find
11. V. Levenshtein. 1966. Binary codes capable of correcting deletions, insertions, and reversals. Soviet Physice-Doklady 10: 707-710. find
12. D. Lin and P. Pantel. 2001. DIRT - Discovery of Inference Rules from Text. In Proceedings of ACM SIGKDD Conference on Knowledge Discovery and Data Mining: 323-328. find
13. I. D. Melamed. 2001. Empirical Methods for Exploiting Parallel Texts. The MIT Press. find
14. R. Mihalcea and T. Pedersen. 2003. An Evaluation Exercise for Word Alignment. In HLT/NAACL Workshop: Building and Using Parallel Texts: 1-10. find
15. F. Och and H. Ney. 2000. Improved Statistical Alignment Models. In Proceedings of the ACL: 440-447. Hong Kong, China. find
16. F. Och and H. Ney. 2003. A Systematic Comparison of Various Statistical Alignment Models. Computational Linguistics 29(1): 19-52. find
17. B. Pang, K. Knight, and D. Marcu. 2003. Syntax-based Alignment of Multiple Translations: Extracting Paraphrases and Generating New Sentences. Proceedings of HLT/NAACL. find our card
18. Y. Shinyama, S. Sekine and K. Sudo. 2002. Automatic Paraphrase Acquisition from News Articles. In Proceedings of NAACL-HLT. find
19. F. K. Soong and E. F. Huang. 1991. A tree-trellis based fast search for finding the n-best sentence hypotheses in continuous speech recognition. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing 1: 705-708. Toronto, Canada. find
20. E. Sumita. 2001. Example-based machine translation using DP-matching between work sequences. In Proceedings of the ACL 2001 Workshop on Data-Driven Methods in Machine Translation: 1–8. find
21. C. Tillmann, S. Vogel, H. Ney, and A. Zubaiga. 1997. A DP Based Search Using Monotone Alignments in Statistical Translation. In Proceedings of the ACL. find
22. L. Vita, A. Ittycheriah, S. Roukos, and N. Kambhatla. 2003. tRuEcasing. In Proceedings of the ACL: 152-159. Sapporo, Japan. find
23. S. Vogel, H. Ney and C. Tillmann. 1996. HMM-Based Word Alignment in Statistical Translation. In Proceedings of the ACL: 836-841. Copenhagen Denmark. find
24. S. Vogel, Y. Zhang, F. Huang, A. Venugopal, B. Zhao, A. Tribble, M. Eck, and A. Waibel. 2003. The CMU Statistical Machine Translation System. In Proceedings of MT Summit IX, New Orleans, Louisiana, USA. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/quirk-2004-monolingual-machine-translation-paraphrase-generation/