The Role of PropBank Sense IDs in AMR-to-text Generation and Text-to-AMR Parsing

What it is about

The paper asks how much the PropBank sense IDs in AMR (the suffixes such as -01) help generation, and how accurately parsers produce them. On the AMR 3.0 test split the authors alter the sense IDs in several ways: remove them (four variants), set them to unrealistic or realistic other senses, swap each to the most or least frequent sense of its verb, and swap verbs to or from verb-particle constructions such as turn in. Four generators are evaluated with BLEU, METEOR and BERTScore. Removing or changing senses lowers the scores only slightly, though significantly. Changing a verb into a verb-particle construction is the largest drop (about 0.012 BERTScore on average), because the root verb itself changes. For parsing, five parsers get the sense right on 98.0% to 98.9% of the verbs they match with gold, including rare senses such as run-04.

Ideas

• I1. Build a perturbation suite for the concept layer: remove the sense suffix in several ways, swap to a random realistic sense, swap to the most and least frequent sense. Check whether the merges in meta.rs and the encoder choices change, and run twice.
• I2. Keep particles as part of the concept: the largest drop in the paper comes from swapping a verb for a particle verb (turn in, go out). The translator should keep the particle with its verb, and the suite above tests that.
• I3. Sense-free keys: since sense IDs cost little for generation, count the Hebrew reader's words by Strong's number or gloss class, and compare with a table keyed by the finer sense. Pick the key by the residual, not by the argument.
• I4. Negative control: include a row in each run where senses are deliberately mismatched, so that a suspiciously good result from the reader cannot pass unnoticed.

For Mova

The perturbation suite fits tools/translate/meta.rs and profile.rs, and the round-trip check fits tools/roundtrip. The paper's parsers are not for Mova. Its method, measuring how much each annotation layer changes the output, is the method Mova's residual already uses.

Topics: Semantic role labeling · Machine translation

Cite

The paper itself, not this card. BibTeX from the library's record of the work.

@inproceedings{hoang2025role,
  title = {{The Role of PropBank Sense IDs in AMR-to-text Generation and Text-to-AMR Parsing}},
  author = {Hoang, Thu and Yang, Mina and Wein, Shira},
  booktitle = {Proceedings of the Sixth International Workshop on Designing Meaning Representations},
  year = {2025},
  url = {https://aclanthology.org/2025.dmr-1.2/}
}

References

2 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Xuefeng Bai, Yulong Chen, and Yue Zhang. 2022. Graph pre-training for AMR parsing and generation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6001–6015, Dublin, Ireland. Association for Computational Linguistics. find
2. Laura Banarescu, Claire Bonial, Shu Cai, Madalina Georgescu, Kira Griffitt, Ulf Hermjakob, Kevin Knight, Philipp Koehn, Martha Palmer, and Nathan Schneider. 2013. Abstract Meaning Representation for sembanking. In Proceedings of the 7th Linguistic Annotation Workshop and Interoperability with Discourse, pages 178–186, Sofia, Bulgaria. Association for Computational Linguistics. find our card

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/hoang-2025-role-propbank-sense-ids-amr/