Not an Interlingua, But Close: Comparison of English AMRs to Chinese and Czech
What it is about
The authors take 100 English sentences that already have AMR graphs, translate them into Chinese and Czech, and write AMR graphs for the translations. The question is whether AMR could serve as a minimally divergent transfer layer in machine translation. The comparison is manual and qualitative. English–Chinese graphs are more compatible than English–Czech ones. For Czech, only 29 of the 100 sentences have structurally identical graphs (apart from argument labels), and 18 more differ only locally, so more than half differ in a deeper way. Causes include multiword expressions that are one concept in one language and several in another, and language-specific concept inventories that do not line up. The conclusion is that AMR is close to an interlingua for some language pairs but not for all, and its guidelines need refinement.
Ideas
- I1. Compatibility count per author pair: for each verse, check whether two translations give identical meta-trees (same concepts and relations), and report the share. The Czech figures (29 identical, 18 local) are a template for a control table in
measure.rs. Compare KJV with WEB and KJV with JPS, to test whether close authors are more compatible. - I2. Treat a multiword phrase that is one concept in one translation and two in another as a phrase node with its own identifier. Test the reader on the verses where this happens.
- I3. Keep each author's concept inventory separate and map it to shared identifiers only through the lexicon, so that an unaligned inventory does not leak into the meta-world.
For Mova
The code is tools/translate/meta.rs and measure.rs. I1 gives the Bible pairs a measure that is comparable to the paper's, and I2 addresses open problem 2 (phrases that carry function words). The paper's Chinese pair is closer than its Czech pair; the same pattern is worth checking for Mova's author pairs.
Cite
@inproceedings{xue2014not,
title = {{Not an Interlingua, But Close: Comparison of English AMRs to Chinese and Czech}},
author = {Xue, Nianwen and Bojar, Ondřej and Hajič, Jan and Palmer, Martha and Urešová, Zdeňka and Zhang, Xiuhong},
booktitle = {Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)},
year = {2014},
url = {https://aclanthology.org/L14-1332/}
}References
- Banarescu, L., Bonial, C., Cai, S., Georgescu, M., Griffitt, K., Hermjakob, U., Knight, K., Koehn, P., Palmer, M., and Schneider, N. (2013). Abstract Meaning Representation for Sembanking. In Proceedings of the 7th Linguistic Annotation Workshop, Sophia, Bulgaria. find our card
- Bohnet, Bernd, Nivre, Joakim, Boguslavsky, Igor, Farkas, Richard, Ginter, Filip, and Hajič, Jan. (2013). Joint morphological and syntactic analysis for richly inflected languages. Transactions of the Association for Computational Linguistics, 1:415–428. find
- Charniak, Eugene. (2000). A maximum-entropy-inspired parser. In Proceedings of the 1st North American chapter of the Association for Computational Linguistics conference, pages 132–139. Association for Computational Linguistics. find
- Collins, Michael. (1999). Head-driven Statistical Models for Natural Language Parsing. Ph.D. thesis, University of Pennsylvania. find our card
- Copestake, Ann, Flickinger, Dan, Pollard, Carl, and Sag, IvanA. (2005). Minimal recursion semantics: An introduction. Research on Language and Computation, 3(2-3):281–332. find
- Gildea, Daniel and Jurafsky, Daniel. (2002). Automatic labeling of semantic roles. Computational linguistics, 28(3):245–288. find our card
- Hajič, Jan, Panevová, Jarmila, Urešová, Zdeňka, Bémová, Alevtina, Kolářová, Veronika, and Pajas, Petr. (2003). PDT-VALLEX: Creating a large-coverage valency lexicon for treebank annotation. In Nivre, Joakim and Hinrichs, Erhard, editors, Proceedings of The Second Workshop on Treebanks and Linguistic Theories, volume 9 of Mathematical Modeling in Physics, Engineering and Cognitive Sciences, pages 57–68, Vaxjo, Sweden. Vaxjo University Press. find
- Hajič, Jan, Ciaramita, Massimiliano, Johansson, Richard, Kawahara, Daisuke, Martı́, Maria, Màrquez, Lluı́s, Meyers, Adam, Nivre, Joakim, Padó, Sebastian, Štěpánek, Jan, Straňák, Pavel, Surdeanu, Mihai, Xue, Nianwen, and Zhang, Yi. (2009). The CoNLL-2009 Shared Task: Syntactic and Semantic Dependencies in Multiple Languages. In Hajič, Jan, editor, Proceedings of the Thirteenth Conference on Computational Natural Language Learning (CoNLL): Shared Task, pages 1–18, Boulder, CO, USA. Association for Computational Linguistics. find our card
- Hajič, Jan, Hajičová, Eva, Panevová, Jarmila, Sgall, Petr, Bojar, Ondřej, Cinková, Silvie, Fučı́ková, Eva, Mikulová, Marie, Pajas, Petr, Popelka, Jan, Semecký, Jiřı́, Šindlerová, Jana, Štěpánek, Jan, Toman, Josef, Urešová, Zdeňka, and Žabokrtský, Zdeněk. (2012). Announcing Prague Czech-English Dependency Treebank 2.0. In Proceedings of the Eighth International Language Resources and Evaluation Conference (LREC’12), pages 3153–3160, Istanbul, Turkey, May. ELRA, European Language Resources Association. find
- Hajič, Jan, Böhmová, Alena, Hajicová, Eva, and Hladká, Barbora. (2003). The Prague Dependency Treebank: A Three Level Annotation Scenario. In Abeillé, Anne, editor, Treebanks: Building and Using Annotated Corpora. Kluwer Academic Publishers. find
- Jones, Bevan, Andreas, Jacob, Bauer, Daniel, Hermann, Karl Moritz, and Knight, Kevin. (2012). Semantics-based machine translation with hyperedge replacement grammars. In COLING, pages 1359–1376. find our card
- Marcus, Mitchell P., Santorini, Beatrice, and Marcinkiewicz, Mary Ann. (1993). Building a large annotated corpus of english: The penn treebank. Computational Linguistics, 19(2):313–330. find our card
- Meyers, A., Reeves, R., Macleod, C., Szekely, R., Zielinska, V., Young, B., and Grishman, R. (2004). The Nom-Bank Project: An Interim Report. In Proceedings of the NAACL/HLT Workshop on Frontiers in Corpus Annotation, pages 24–31, Boston, Massachusetts. find
- Palmer, Martha, Gildea, Daniel, and Kingsbury, Paul. (2005). The Proposition Bank: An annotated corpus of semantic roles. Computational Linguistics, 31(1):71– 106. find our card
- Petrov, Slav and Klein, Dan. (2007). Improved inference for unlexicalized parsing. In HLT-NAACL, pages 404– 411. find
- Pradhan, Sameer S, Ward, Wayne, Hacioglu, Kadri, Martin, James H, and Jurafsky, Daniel. (2004). Shallow semantic parsing using support vector machines. In HLT-NAACL, pages 233–240. find
- Prasad, Rashmi, Dinesh, Nikhil, Lee, Alan, Miltsakaki, Eleni, Robaldo, Livio, Joshi, Aravind, and Webber, Bonnie. (2008). The Penn Discourse Treebank 2.0. In Proceedings of the 6th International Conference on Language Resources and Evaluation (LREC 2008), Marrakech, Morocco. find our card
- Pustejovsky, James, Hanks, Patrick, Saurı́, Roser, See, Andrew, Day, David, Ferro, Lisa, Gaizauskas, Robert, Lazo, Marcia, Setzer, Andrea, and Sundheim, Beth. (2003). The TimeBank Corpus. Corpus Linguistics, pages 647– 656. find
- Saurı́, Roser and Pustejovsky, James. (2009). Factbank: a corpus annotated with event factuality. Language resources and evaluation, 43(3):227–268. find
- Sgall, Petr, Hajičová, Eva, and Panevová, Jarmila. (1986). The Meaning of the Sentence and Its Semantic and Pragmatic Aspects. Academia/Reidel Publishing Company, Prague, Czech Republic/Dordrecht, Netherlands. find
- Srikumar, Vivek and Roth, Dan. (2013). Modeling semantic relations expressed by prepositions. Transactions of the Association for Computational Linguistics, 1:231– 242. find
- Urešová, Zdeňka. (2011). Valenčnı́ slovnı́k Pražského závislostnı́ho korpusu (PDT-Vallex). Studies in Computational and Theoretical Linguistics. Ústav formálnı́ a aplikované lingvistiky, Praha, Czechia. find
- Xue, Nianwen and Palmer, Martha. (2004). Calibrating features for semantic role labeling. In EMNLP, pages 88–94. find
- Xue, Nianwen, Xia, Fei, dong Chiou, Fu, and Palmer, Martha. (2005). The Penn Chinese TreeBank: Phrase Structure Annotation of a Large Corpus. Natural Language Engineering, 11(2):207–238. find
This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/xue-2014-not-interlingua-but-close-comparison/