A Simple and Accurate Syntax-Agnostic Neural Model for Dependency-based Semantic Role Labeling

What it is about

The model labels dependency-based roles without any syntactic input, or with predicted POS tags only. A bidirectional LSTM encodes the sentence; for each predicate the sentence is encoded again with a binary flag on the predicate token, and the role of each candidate word is predicted from the states of the two endpoints. Role labels are predicate-specific and built compositionally from the predicate lemma and the role. On the English in-domain CoNLL-2009 test the model reaches 87.7 F1, above the best local model (86.7), and it reaches 77.7 on the out-of-domain test, above the ensemble of PathLSTMs (76.5). In the development ablation, removing the predicate-specific encoding drops F1 from 86.6 to 80.4, and removing POS tags drops it to 85.9.

Ideas

• I1. A per-predicate pass in the rule-based labeler: for each predicate run the argument classifier with a predicate flag on each token. Measure dev F1 with and without the flag, and use a negative control in which the flag is moved to a random token, which must score worse.
• I2. Store role labels as (predicate lemma, role) pairs, with a backoff to the role alone for rare lemmas. This is the paper's compositional idea in count form, and the gain should show mostly on rare predicates, which can be reported separately.
• I3. The out-of-domain result suggests a linear-distance feature set as a fallback when the UD parse is unreliable. Measure whether the fallback recovers arguments on the out-of-domain lessons.
• I4. Use a component ablation table, as the paper does (Table 7), for every new role feature. Remove one feature at a time, measure, and delete the feature if it does not move dev F1.

For Mova

The chapters are 13. Verbs (predicates and their frames) and 28. Voice and the passive (argument alternations). The code is prag/src/roles.rs and world/src/events.rs. The measure is dev argument F1 with the flag ablation, run twice for determinism. The Early Modern lessons in languages/en-x-emode/ are a natural out-of-domain test.

Topics: Semantic role labeling · Part-of-speech tagging

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/k17-1041

@inproceedings{marcheggiani2017simple,
  title = {{A Simple and Accurate Syntax-Agnostic Neural Model for Dependency-based Semantic Role Labeling}},
  author = {Marcheggiani, Diego and Frolov, Anton and Titov, Ivan},
  booktitle = {Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017)},
  year = {2017},
  doi = {10.18653/v1/k17-1041},
  url = {https://doi.org/10.18653/v1/k17-1041}
}

References

36 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Collin F. Baker, Charles J. Fillmore, and John B. Lowe. 1998. The Berkeley FrameNet project. In Proceedings of COLING-ACL. find our card
2. Jan Hajic, Massimiliano Ciaramita, Richard Johansson, Daisuke Kawahara, Maria Antònia Martı́, Lluı́s Màrquez, Adam Meyers, Joakim Nivre, Sebastian Padó, Jan Stepánek, Pavel Stranák, Mihai Surdeanu, Nianwen Xue, and Yi Zhang. 2009. The CoNLL-2009 shared task: Syntactic and semantic dependencies in multiple languages. In Proceedings of CoNLL. find our card
3. Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Janvin. 2003. A neural probabilistic language model. Journal of Machine Learning Research 3:1137–1155. find
4. Anders Björkelund, Bernd Bohnet, Love Hafdell, and Pierre Nugues. 2010. A high-performance syntactic and semantic dependency parser. In Proceedings of COLING. find
5. James Henderson, Paola Merlo, Gabriele Musillo, and Ivan Titov. 2008. A latent variable model of synchronous parsing for syntactic and semantic dependencies. In Proceedings of CoNLL. find
6. Anders Björkelund, Love Hafdell, and Pierre Nugues. 2009. Multilingual semantic role labeling. In Proceedings of CoNLL. find
7. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Computation 9(8):1735–1780. find
8. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2016. Enriching word vectors with subword information. arXiv preprint arXiv:1607.04606 . ↗
9. Mohit Iyyer, Varun Manjunatha, Jordan Boyd-Graber, and Hal Daumé III. 2015. Deep unordered composition rivals syntactic methods for text classification. In Proceedings of ACL. find
10. Janara Christensen, Mausam, Stephen Soderland, and Oren Etzioni. 2011. An analysis of open information extraction based on semantic role labeling. In Proceedings of the 6th International Conference on Knowledge Capture (K-CAP) 2011. find
11. Diederik Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In Proceedings of ICLR. find
12. Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. JMLR 12:2493–2537. find
13. Eliyahu Kiperwasser and Yoav Goldberg. 2016. Simple and accurate dependency parsing using bidirectional LSTM feature representations. TACL . find our card
14. James Cross and Liang Huang. 2016. Incremental parsing with minimal features using bi-directional LSTM. In Proceedings of ACL. find
15. Tao Lei, Yuan Zhang, Lluı́s Màrquez, Alessandro Moschitti, and Regina Barzilay. 2015. High-order low-rank tensors for semantic role labeling. In Proceedings of NAACL. find
16. Chris Dyer, Miguel Ballesteros, Wang Ling, Austin Matthews, and Noah A. Smith. 2015. Transition-based dependency parsing with stack long short-term memory. In Proceedings of ACL. find our card
17. Wang Ling, Chris Dyer, Alan W Black, and Isabel Trancoso. 2015. Two/too simple adaptations of word2vec for syntax problems. In Proceedings of NAACL. find
18. Ivan Titov, James Henderson, Paola Merlo, and Gabriele Musillo. 2009. Online projectivisation for synchronous parsing of semantic and syntactic dependencies. In Proceedings of IJCAI. find
19. Tal Linzen, Emmanuel Dupoux, and Yoav Goldberg. 2016. Assessing the ability of LSTMs to learn syntax-sensitive dependencies. TACL 4:521–535. find
20. Jason Naradowsky, Sebastian Riedel, and David A Smith. 2012. Improving nlp through marginalization of hidden syntactic structure. In Proceedings of EMNLP. find
21. Kristina Toutanova, Aria Haghighi, and Christopher D. Manning. 2008. A global joint model for semantic role labeling. Computational Linguistics 34(2):161– 191. find
22. Martha Palmer, Paul Kingsbury, and Daniel Gildea. 2005. The proposition bank: An annotated corpus of semantic roles. Computational Linguistics 31(1):71–106. find our card
23. Hai Zhao, Wenliang Chen, Jun’ichi Kazama, Kiyotaka Uchimoto, and Kentaro Torisawa. 2009. Multilingual dependency learning: Exploiting rich features for tagging syntactic and semantic dependencies. In Proceedings of CoNLL. find
24. Sameer Pradhan, Kadri Hacioglu, Wayne H. Ward, James H. Martin, and Daniel Jurafsky. 2005. Semantic role chunking combining complementary syntactic views. In Proceedings of CoNLL. find
25. Jie Zhou and Wei Xu. 2015. End-to-end learning of semantic role labeling using recurrent neural networks. In Proceedings of ACL. find
26. Vasin Punyakanok, Dan Roth, and Wen-tau Yih. 2008. The importance of syntactic parsing and inference in semantic role labeling. Computational Linguistics 34(2):257–287. find
27. Lance A. Ramshaw and Mitchell P. Marcus. 1995. Text chunking using transformation-based learning. In Proceedings of the Third ACL Workshop on Very Large Corpora. find
28. Dan Roth and Wen-tau Yih. 2005. Integer linear programming inference for conditional random fields. In Proceedings of ICML. find
29. Michael Roth and Mirella Lapata. 2016. Neural semantic role labeling with dependency path embeddings. In Proceedings of ACL. find
30. Michael Roth and Kristian Woodsend. 2014. Composition of word representations improves semantic role labelling. In Proceedings of EMNLP. find
31. Dan Shen and Mirella Lapata. 2007. Using semantic roles to improve question answering. In Proceedings of EMNLP-CoNLL. find
32. Mihai Surdeanu, Richard Johansson, Adam Meyers, Lluı́s Màrquez, and Joakim Nivre. 2008. find
33. The CoNLL 2008 shared task on joint parsing of syntactic and semantic dependencies. In Proceedings of CoNLL. find our card
34. Mihai Surdeanu, Lluı́s Màrquez, Xavier Carreras, and Pere Comas. 2007. Combination strategies for semantic role labeling. Journal of Artificial Intelligence Research 29:105–151. find
35. Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to sequence learning with neural networks. In Proceedings of NIPS. find
36. Swabha Swayamdipta, Miguel Ballesteros, Chris Dyer, and Noah A. Smith. 2016. Greedy, joint syntactic-semantic parsing with stack LSTMs. In Proceedings of CoNLL. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/marcheggiani-2017-simple-accurate-syntax-agnostic-neural/