Open-source Frame Semantic Parsing

What it is about

Frame Semantic Transformer is an open-source Python library for frame semantic parsing, aimed at ease of use. It treats parsing as three text-to-text subtasks for a T5 model: trigger identification (the model marks the trigger word with an asterisk), frame classification and argument extraction. For frame classification, candidate frames come from the FrameNet lexical units. Trigger and lexical-unit words are normalised with several stemmers and a WordNet lemmatiser, and bigrams around the trigger are added, so the candidates are passed to the model as hints in the prompt. The model is pre-trained on PropBank (from OntoNotes and the English Web Treebank) and on about 100,000 FrameNet exemplar sentences, and trained with text augmentations (synonyms, misspellings, keyboard typos, case changes, punctuation deletion). Trigger identification is sampled three times as often as the other tasks. The paper evaluates on FrameNet 1.7 with the Open Sesame split and reports that it exceeds Open Sesame. The scores are in a figure, which this summary has not read; no score is quoted here.

Ideas

I1. The lexical-unit hint is a rule that needs no learning. Normalise the trigger and the frame's lexical units with the same stemming and lemma rules, and use the matching frames as a candidate set in prag/src/frames.rs. Measure frame accuracy with and without the candidate restriction on school sentences.
I2. Trigger identification as a separate measured step. In Mova this is a content-word rule in world/src/events.rs; report its precision and recall on its own, so that errors in the frame step are not blamed on triggers.
I3. Use the augmentations as a robustness test rather than as training data: apply the same kinds of change (case, punctuation removal, typos, quote style) to school sentences and measure the drop in the frame and role outputs. The paper itself notes that augmentation slightly hurts argument extraction, so the trade-off should be measured.

For Mova

The chapters are 06. Spelling rules: inflection, derivation, compounds and national standards (normalisation) and 13. Verbs. The code is prag/src/frames.rs, en/src/tok.rs for tokens and world/src/events.rs for triggers. Measure on school sentences; run twice to check determinism.

Topics: Semantic role labeling

Cite

The paper itself, not this card. BibTeX from the library's record of the work. arXiv 2303.12788

@misc{chanin2023open,
  title = {{Open-source Frame Semantic Parsing}},
  author = {Chanin, David},
  year = {2023},
  eprint = {2303.12788},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2303.12788}
}

References

5 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

Python package index - pypi. URL https://pypi.org/. C. F. Baker, C. J. Fillmore, and J. B. Lowe. The berkeley framenet project. In COLING 1998 Volume 1: The 17th International Conference on Computational Linguistics, 1998. ↗ our card
A. Bies, J. Mott, C. Warner, and S. Kulick. English web treebank dataset ldc2012t13. Linguistic Data Consortium, Philadelphia, PA, 2012. find
S. Bird, E. Klein, and E. Loper. Natural language processing with Python: analyzing text with the natural language toolkit. "O’Reilly Media, Inc.", 2009. Q. Chen, Z. Zhuo, and W. Wang. Bert for joint intent classification and slot filling. arXiv preprint arXiv:1902.10909, 2019. Y.-N. Chen, W. Y. Wang, and A. I. Rudnicky. Unsupervised induction and filling of semantic slots for spoken dialogue systems using frame-semantic parsing. In 2013 IEEE Workshop on Automatic Speech Recognition and Understanding, pages 120–125. ↗
IEEE, 2013. D. Das, N. Schneider, D. Chen, and N. A. Smith. Probabilistic frame-semantic parsing. In Human language technologies: The 2010 annual conference of the North American chapter of the association for computational linguistics, pages 948–956, 2010. D. Gildea and D. Jurafsky. Automatic labeling of semantic roles. Computational linguistics, 28(3):245–288, 2002. A. Kalyanpur, O. Biran, T. Breloff, J. Chu-Carroll, A. Diertani, O. Rambow, and M. Sammons. Open-domain frame semantic parsing using transformers. arXiv preprint arXiv:2010.10998, 2020. P. R. Kingsbury and M. Palmer. From treebank to propbank. In LREC, pages 1989–1993, 2002. M. Kshirsagar, S. Thomson, N. Schneider, J. G. Carbonell, N. A. Smith, and C. Dyer. Frame-semantic role labeling with heterogeneous annotations. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 218–224, 2015. E. Ma. Nlp augmentation. https://github.com/makcedward/nlpaug, 2019. G. A. Miller. Wordnet: a lexical database for english. Communications of the ACM, 38(11):39–41, 1995. C. D. Paice. Another stemmer. In ACM Sigir Forum, volume 24, pages 56–61. ↗ our card
ACM New York, NY, USA, 1990. H. Peng, S. Thomson, S. Swayamdipta, and N. A. Smith. Learning joint semantic parsers from disjoint data. arXiv preprint arXiv:1804.05990, 2018. M. F. Porter. An algorithm for suffix stripping. Program, 14(3):130–137, 1980. C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. The Journal of Machine Learning Research, 21(1): 5485–5551, 2020. X. Su, R. Li, X. Li, J. Z. Pan, H. Zhang, Q. Chai, and X. Han. A knowledge-guided framework for frame identification. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 5230–5240, 2021. S. Swayamdipta, S. Thomson, C. Dyer, and N. A. Smith. Frame-semantic parsing with softmax-margin segmental rnns and a syntactic scaffold. arXiv preprint arXiv:1706.09528, 2017. R. Weischedel, M. Palmer, M. Marcus, E. Hovy, S. Pradhan, L. Ramshaw, N. Xue, A. Taylor, J. Kaufman, M. Franchini, et al. Ontonotes release 5.0 ldc2013t19. Linguistic Data Consortium, Philadelphia, PA, 23, 2013. T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz, J. Davison, S. Shleifer, P. von Platen, C. Ma, Y. Jernite, J. Plu, C. Xu, T. L. Scao, S. Gugger, M. Drame, Q. Lhoest, and A. M. Rush. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45, Online, Oct. 2020. Association for Computational Linguistics. URL https://www.aclweb.org/anthology/2020.emnlp-demos.6. X. Zhao, X. Walton, S. Shrestha, and A. Rios. Bike frames: Understanding the implicit portrayal of cyclists in the news. arXiv preprint arXiv:2301.06178, 2023. C. Zheng, X. Chen, R. Xu, and B. Chang. A double-graph based framework for frame semantic parsing. arXiv preprint arXiv:2206.09158, 2022. ↗

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/chanin-2023-open-source-frame-semantic-parsing/