RIGOTRIO at SemEval-2017 Task 9: Combining Machine Learning and Grammar Engineering for AMR Parsing and Generation

What it is about

The paper reports a team's entry to the SemEval 2017 AMR task, with two parts. For parsing, a neural AMR parser from the previous year was retrained on more data and given a gazetteer of organic compound names, which raised Smatch to 54.0 on the biomedical set. For generation, about 200 hand-written tree-pattern rules map an AMR to an abstract syntax tree of the Grammatical Framework resource grammar, and the grammar then linearises the tree into English. The rules cover only 12.3% of the 1,293 test AMRs, so the grammar output was combined with the JAMR statistical generator, which handled the rest. The combination reached a human Trueskill score of 107.2 and 18.82 BLEU. Scored apart, the grammar sentences reached 11.35 BLEU against 19.18 for JAMR, which shows that BLEU favours the statistical system's closer copying.

Ideas

• I1. Coverage-gated hybrid: take the rule path only when every meta token of a sentence is covered, and fall back otherwise. Report the covered share and the score of each path separately, because the combined score hides the gap.
• I2. Merge multi-word names into one meta token: a chain of operator relations under a name node becomes one concept with its source words kept, so the encoder reads "New York City" as one chunk.
• I3. Deletion rules (named-entity type nodes, wiki links) remove nodes. Count how many meta tokens each deletion removes on the Bible pairs and check the residual before and after each rule.
• I4. Names gazetteer: the paper's gazetteer came only from training data and hurt named-entity recall. For the translator's name lexicon, use an outside list and keep training-only names as a separate, measured class.

For Mova

The coverage gate and the separate path scores fit tools/translate (the standard path and any fallback), and the name merge fits tools/translate/meta.rs. The grammar itself is not for Mova. The useful part is the table of rules with coverage counts, which is the transparent form Mova already wants.

Topics: Machine translation

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/s17-2159

@inproceedings{gruzitis2017rigotrio,
  title = {{RIGOTRIO at SemEval-2017 Task 9: Combining Machine Learning and Grammar Engineering for AMR Parsing and Generation}},
  author = {Gruzitis, Normunds and Gosko, Didzis and Barzdins, Guntis},
  booktitle = {Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017)},
  year = {2017},
  doi = {10.18653/v1/s17-2159},
  url = {https://doi.org/10.18653/v1/s17-2159}
}

References

10 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Laura Banarescu, Claire Bonial, Shu Cai, Madalina Georgescu, Kira Griffitt, Ulf Hermjakob, Kevin Knight, Philipp Koehn, Martha Palmer, and Nathan Schneider. 2013. Abstract Meaning Representation for Sembanking. In Proceedings of the 7th Linguistic Annotation Workshop and Interoperability with Discourse. find our card
2. Guntis Barzdins and Didzis Gosko. 2016. RIGA at SemEval-2016 Task 8: Impact of Smatch extensions and character-level neural translation on AMR parsing accuracy. In Proceedings of the 10th International Workshop on Semantic Evaluation. find
3. Alastair Butler. 2016. Deterministic natural language generation from meaning representations for machine translation. In Proceedings of the 2nd Workshop on Semantics-Driven Machine Translation. find our card
4. Jeffrey Flanigan, Chris Dyer, Noah A. Smith, and Jaime Carbonell. 2016. Generation from Abstract Meaning Representation using Tree Transducers. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. find our card
5. Normunds Gruzitis and Guntis Barzdins. 2016. The role of CNL and AMR in scalable abstractive summarization for multilingual media monitoring. In Controlled Natural Language, Springer, volume 9767 of LNCS. find
6. Roger Levy and Galen Andrew. 2006. Tregex and Tsurgeon: Tools for querying and manipulating tree data structures. In Proceedings of the 5th International Conference on Language Resources and Evaluation. find
7. Jonathan May and Jay Priyadarshi. 2017. Semeval-2017 task 9: Abstract meaning representation parsing and generation. In Proceedings of the 11th International Workshop on Semantic Evaluation. find
8. Martha Palmer, Daniel Gildea, and Paul Kingsbury. 2005. The Proposition Bank: An Annotated Corpus of Semantic Roles. Computational Linguistics 31(1). find our card
9. Aarne Ranta. 2004. Grammatical Framework: A Type-Theoretical Grammar Formalism. Journal of Functional Programming 14(2). find
10. Aarne Ranta. 2009. The GF Resource Grammar Library. Linguistic Issues in Language Technology 2(2). find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/gruzitis-2017-rigotrio-semeval-2017-task-9/