Dependency-based Semantic Role Labeling of PropBank

What it is about

The paper builds a PropBank role labeler on top of a dependency parser. A syntactic model (second-order projective, with pseudo-projective handling of non-projective links) and a semantic model (a pipeline of logistic classifiers for sense, argument identification and labeling, followed by a global model) each produce candidates, and a joint reranker picks the complete syntactic-semantic output from the top 16 trees. Training uses the online passive-aggressive algorithm. On the CoNLL-2005 WSJ+Brown test set the segment-based F1 is 77.97 (78.84 after punctuation normalisation), close to the best published 79.22. On the CoNLL-2008 test set the dependency-based F1 is 84.29, above the constituent-based system used as reference (83.45). Precision drops sharply on the Brown out-of-domain set, which the authors trace to prepositional attachment errors of the parser.

Ideas

• I1. Global constraints as deterministic filters on candidate frames: no core role twice, a continuation label C-X needs an X, and a relative-clause reference R-X needs an X. Implement them as checks in prag/src/roles.rs, count how many candidates each one removes, and measure dev argument F1 with and without them (run twice).
• I2. Keep a candidate pool of the top k structures from the rule-based labeler and rerank it with a log-linear score over the parts. Report dev F1 for k = 1, 4 and 16 to see where the reranker starts to pay.
• I3. Report the score in two ways, segment-based and head-based. The paper shows that the same output can rank differently under the two metrics, so a single number hides the choice of metric.
• I4. Keep an out-of-domain split in the school tests. The paper's Brown result shows that attachment accuracy falls off quickly out of domain, which a Mova lesson set can expose.

For Mova

The chapters are 21. The simple clause and 22. Phrases: noun, adjective, adverb and prepositional phrases (attachment), with 28. Voice and the passive for argument alternations. The code is prag/src/roles.rs and world/src/events.rs. The measure is argument F1 on dev, with an out-of-domain check on the Early Modern lessons in languages/en-x-emode/, where the parser is expected to be weaker.

Topics: Semantic role labeling

Cite

The paper itself, not this card. BibTeX from the library's record of the work.

@inproceedings{johansson2008dependency,
  title = {{Dependency-based Semantic Role Labeling of PropBank}},
  author = {Johansson, Richard and Nugues, Pierre},
  booktitle = {Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing},
  year = {2008},
  url = {https://aclanthology.org/D08-1008/}
}

References

31 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Collin Baker, Michael Ellsworth, and Katrin Erk. 2007. SemEval task 19: Frame semantic structure extraction. In Proceedings of SemEval-2007. find
2. Xavier Carreras and Lluís Màrquez. 2004. Introduction to the CoNLL-2004 shared task: Semantic role labeling. In Proceedings of CoNLL-2004. find
3. Xavier Carreras and Lluís Màrquez. 2005. Introduction to the CoNLL-2005 shared task: Semantic role labeling. In Proceedings of CoNLL-2005. find our card
4. Xavier Carreras. 2007. Experiments with a higher-order projective dependency parser. In Proceedings of CoNLL-2007. find
5. David M. Chickering, Dan Geiger, and David Heckerman. 1994. Learning Bayesian networks: The combination of knowledge and statistical data. Technical Report MSR-TR-94-09, Microsoft Research. find
6. Ronan Collobert and Jason Weston. 2007. Fast semantic extraction using a novel neural network architecture. In Proceedings of ACL-2007. find
7. Koby Crammer, Ofer Dekel, Joseph Keshet, Shai Shalev-Schwartz, and Yoram Singer. 2006. Online passive-aggressive algorithms. JMLR, 2006(7):551–585. find
8. Jason M. Eisner. 1996. Three new probabilistic models for dependency parsing: An exploration. In Proceedings of ICCL. find our card
9. Daniel Gildea and Daniel Jurafsky. 2002. Automatic labeling of semantic roles. Computational Linguistics, 28(3):245–288. find our card
10. Daniel Gildea and Martha Palmer. 2002. The necessity of syntactic parsing for predicate argument recognition. In Proceedings of the ACL-2002. find
11. Aria Haghighi, Andrew Y. Ng, and Christopher D. Manning. 2005. Robust textual inference via graph matching. In Proceedings of EMNLP-2005. find
12. Andrew Hickl, Jeremy Bensley, John Williams, Kirk Roberts, Bryan Rink, and Ying Shi. 2006. Recognizing textual entailment with LCC’s G ROUNDHOG systems. In Proceedings of the Second PASCAL Recognizing Textual Entailment Challenge. find
13. Liang Huang. 2008. Forest reranking: Discriminative parsing with non-local features. In Proceedings of ACL-2008. find
14. Richard Johansson and Pierre Nugues. 2008a. Dependency-based syntactic–semantic analysis with PropBank and NomBank. In Proceedings of the Shared Task Session of CoNLL-2008. find
15. Richard Johansson and Pierre Nugues. 2008b. The effect of syntactic representation on semantic role labeling. In Proceedings of COLING-2008. find
16. Chih-Jen Lin, Ruby C. Weng, and S. Sathiya Keerthi. 2008. Trust region Newton method for large-scale logistic regression. JMLR, 2008(9):627–650. find
17. Ken Litkowski. 2004. Senseval-3 task: Automatic labeling of semantic roles. In Proceedings of Senseval-3. find our card
18. Mitchell P. Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of English: the Penn Treebank. Computational Linguistics, 19(2):313–330. ↗ our card
19. Ryan McDonald and Fernando Pereira. 2006. Online learning of approximate dependency parsing algorithms. In Proceedings of EACL-2006. find our card
20. Alessandro Moschitti, Paul Morărescu, and Sanda Harabagiu. 2003. Open domain information extraction via automatic semantic labeling. In Proceedings of FLAIRS. find
21. Joakim Nivre and Jens Nilsson. 2005. Pseudo-projective dependency parsing. In Proceedings of ACL-2005. find
22. Jacob Persson, Richard Johansson, and Pierre Nugues. 2008. Text categorization using predicate–argument structures. Submitted. find
23. Sameer Pradhan, Wayne Ward, Kadri Hacioglu, James Martin, and Daniel Jurafsky. 2005. Semantic role labeling using different syntactic views. In Proceedings of ACL-2005. find
24. Vasin Punyakanok, Dan Roth, and Wen-tau Yih. 2008. The importance of syntactic parsing and inference in semantic role labeling. Computational Linguistics, 34(2):257–287. find
25. Mihai Surdeanu, Sanda Harabagiu, John Williams, and Paul Aarseth. 2003. Using predicate-argument structures for information extraction. In Proceedings of ACL-2003. find
26. Mihai Surdeanu, Lluís Màrquez, Xavier Carreras, and Pere R. Comas. 2007. Combination strategies for semantic role labeling. Journal of Artificial Intelligence Research, 29:105–151. find
27. Mihai Surdeanu, Richard Johansson, Adam Meyers, Lluís Màrquez, and Joakim Nivre. 2008. The CoNLL–2008 shared task on joint parsing of syntactic and semantic dependencies. In Proceedings of CoNLL–2008. find our card
28. Reid Swanson and Andrew S. Gordon. 2006. A comparison of alternative parse tree paths for labeling semantic roles. In Proceedings of COLING/ACL-2006. find
29. Ben Taskar, Carlos Guestrin, and Daphne Koller. 2003. Max-margin Markov networks. In Proceedings of NIPS-2003. find
30. Kristina Toutanova, Aria Haghighi, and Christopher D. Manning. 2005. Joint learning improves semantic role labeling. In Proceedings of ACL-2005. find
31. Nianwen Xue and Martha Palmer. 2004. Calibrating features for semantic role labeling. In Proceedings of EMNLP-2004. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/johansson-2008-dependency-based-semantic-role-labeling/