Unsupervised Morphology Rivals Supervised Morphology for Arabic MT

What it is about

The question is whether an unsupervised Arabic segmenter (the Lee et al. Bayesian model) can match supervised segmenters such as MADA and Sakhr when it feeds a hierarchical string-to-tree MT system. The authors add maximum marginal decoding (MM) over 25 Gibbs-sampling runs. Sampling alone changes 13% of token segmentations between runs; MM agrees across two runs on over 95% of tokens, cuts the induced affixes for the large MT lexicon, and raises segmentation F1 on the Arabic Treebank from a mean of 86.2% to 88.2%. On NIST MT-08 (Modern Standard Arabic) the MM segmenter gives higher BLEU than MADA on the full corpus and on the 1.3M-word subset (BLEU values are in the paper's Table 4), and on a Levantine dialect corpus it gives an 18% relative BLEU gain over no segmentation.

Ideas

• I1. Measure run-to-run agreement before trusting any stochastic step: the paper's Table 1 counts the share of tokens and types that two runs segment alike. Mova is deterministic, so this is a check for any sampled or randomized component of tools/hbo: two runs must match exactly, and any sampled step must report its agreement.
• I2. Prune rare affix classes by coverage: the paper keeps only the prefixes and suffixes that cover 95% of the affix tokens (Top-95) and sees the inventory shrink. In tools/hbo, the same count over the pre and suf tables shows which rare splits can be dropped.
• I3. Score a segmenter by its downstream effect, not only its F1: the paper reports the segmenter's gain in MT BLEU as well as its segmentation accuracy. For Mova, report gparse LAS on treebanks/hbo/ptnk next to each segmentation variant, and keep the test set held out until the release.

For Mova

I1 and I3 are the measurement rules for tools/hbo (segmenter plus gparse). I2 is a table audit for the segmenter tables pre and suf in tools/hbo. The unsupervised model itself is not a Mova idea: Mova's segmenter learns its splits from MACULA rather than sampling them.

Topics: Treebanks · Machine translation

Cite

The paper itself, not this card. BibTeX from the library's record of the work.

@inproceedings{stallard2012unsupervised,
  title = {{Unsupervised Morphology Rivals Supervised Morphology for Arabic MT}},
  author = {Stallard, David and Devlin, Jacob and Kayser, Michael and Lee, Yoong Keok and Barzilay, Regina},
  booktitle = {Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)},
  year = {2012},
  url = {https://aclanthology.org/P12-2063/}
}

References

26 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Eleftherios Avramidis and Philipp Koehn. 2008. Enriching morphologically poor languages for statistical machine translation. In Proceedings of ACL-08: HLT. find
2. Ibrahim Badr, Rabih Zbib, and James Glass. 2008. Segmentation for English-to-Arabic statistical machine translation. In Proceedings of ACL-08: HLT, Short Papers. find
3. Ann Clifton and Anoop Sarkar. 2011. Combining morpheme-based machine translation with post-processing morpheme prediction. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies. find
4. Mathias Creutz and Krista Lagus. 2007. Unsupervised models for morpheme segmentation and morphology learning. ACM Trans. Speech Lang. Process., 4:3:1– 3:34, February. find
5. Nizar Habash and Owen Rambow. 2005. Arabic tokenization, part-of-speech tagging and morphological disambiguation in one fell swoop. In Proceedings of ACL. find our card
6. Nizar Habash. 2008. Four techniques for online handling of out-of-vocabulary words in Arabic-English statistical machine translation. In Proceedings of ACL-08: HLT, Short Papers. find
7. Mark Johnson and Sharon Goldwater. 2009. Improving nonparametric bayesian inference: experiments on unsupervised word segmentation with adaptor grammars. In Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics. find
8. Philipp Koehn and Hieu Hoang. 2007. Factored translation models. In Proceedings of EMNLP-CoNLL, pages 868–876. find
9. Philipp Koehn. 2004. Statistical significance tests for machine translation evaluation. In Proceedings of EMNLP 2004. find
10. Yoong Keok Lee, Aria Haghighi, and Regina Barzilay. 2011. Modeling syntactic context improves morphological segmentation. In Proceedings of the Fifteenth Conference on Computational Natural Language Learning. find
11. Minh-Thang Luong, Preslav Nakov, and Min-Yen Kan. 2010. A hybrid morpheme-word representation for machine translation of morphologically rich languages. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing. find
12. Coşkun Mermer and Ahmet Afşın Akın. 2010. Unsupervised search for the optimal segmentation for statistical machine translation. In Proceedings of the ACL 2010 Student Research Workshop, pages 31–36, Uppsala, Sweden, July. Association for Computational Linguistics. find
13. Coşkun Mermer and Murat Saraclar. 2011. Unsupervised Turkish morphological segmentation for statistical machine translation. In Workshop on Machine Translation and Morphologically-rich languages, January. find
14. Preslav Nakov and Hwee Tou Ng. 2011. Translating from morphologically complex languages: A paraphrase-based approach. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies. find
15. Jason Naradowsky and Kristina Toutanova. 2011. Unsupervised bilingual morpheme segmentation and alignment with context-rich hidden semi-Markov models. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies. find
16. NIST. 2010. NIST 2008 Open Machine Translation (Open MT) Evaluation. http://www.ldc.upenn.edu/Catalog/catalogEntry.jsp? catalogId=LDC2010T21/. ↗
17. Kemal Oflazer. 1993. Two-level description of Turkish morphology. In Proceedings of the Sixth Conference of the European Chapter of the Association for Computational Linguistics. find
18. Hoifung Poon, Colin Cherry, and Kristina Toutanova. 2009. Unsupervised morphological segmentation with log-linear models. In Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics. find
19. Lawrence R. Rabiner. 1989. A tutorial on hidden Markov models and selected applications in speech recognition. In Proceedings of the IEEE, pages 257– 286. find
20. Fatiha Sadat and Nizar Habash. 2006. Combination of Arabic preprocessing schemes for statistical machine translation. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics. find
21. Libin Shen, Jinxi Xu, and Ralph Weischedel. 2008. A new string-to-dependency machine translation algorithm with a target dependency language model. In Proceedings of ACL-08: HLT. find our card
22. Sami Virpioja, Jaakko J. Väyrynen, Mathias Creutz, and Markus Sadeniemi. 2007. Morphology-aware statistical machine translation based on morphs induced in an unsupervised manner. In Proceedings of the Machine Translation Summit XI. find
23. Larry Wasserman. 2006. All of Nonparametric Statistics. Springer. find
24. Mei Yang and Katrin Kirchhoff. 2006. Phrase-based backoff models for machine translation of highly inflected languages. In Proceedings of EACL. find
25. Rabih Zbib, Michael Kayser, Spyros Matsoukas, John Makhoul, Hazem Nader, Hamdy Soliman, and Rami Safadi. 2012a. Methods for integrating rule-based and statistical systems for Arabic to English machine translation. Machine Translation, 26(1-2):67–83. find
26. Rabih Zbib, Erika Malchiodi, Jacob Devlin, David Stallard, Spyros Matsoukas, Richard Schwartz, John Makhoul, Omar F. Zaidan, and Chris Callison-Burch. 2012b. Machine translation of Arabic dialects. In NAACL 2012: Proceedings of the 2012 Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics, Montreal, Quebec, Canada, June. Association for Computational Linguistics. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2012-stallard-unsupervised-morphology-rivals-supervised-morphology/