Mova0.5 pre-release⌕
☰
home / docs / papers / English / LSTM CCG Parsing

LSTM CCG Parsing

What it is about

The paper asks whether a CCG parser needs an explicit model of bilexical dependencies. Its parser has none: a bidirectional LSTM supertagger assigns lexical categories, a deterministic grammar combines them, and A* search finds the best parse. Semi-supervised tri-training adds training signal. On CCGbank section 23 the parser improves on the earlier state of the art by 1.1 F1 in domain and by up to 4.5 F1 out of domain (BioInfer). It parses over 2600 sentences per second on a GPU. When all supertags of a sentence are correct, the parse F1 is 97.7; when some supertags are wrong, it drops to 80.3.

Ideas

For Mova

The chapters are 20. Syntax: the Universal Dependencies frame and 21-clause-structure.md. The code is en/src/ptag.rs and en/src/parse.rs. The paper is neural and GPU-fast, so only the error split and the length check transfer to our CPU pipeline. Its data are CCGbank (WSJ-derived), not in our licence list, so use no part of them in the build.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/n16-1026

@inproceedings{lewis2016lstm,
  title = {{LSTM CCG Parsing}},
  author = {Lewis, Mike and Lee, Kenton and Zettlemoyer, Luke},
  booktitle = {Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies},
  year = {2016},
  doi = {10.18653/v1/n16-1026},
  url = {https://doi.org/10.18653/v1/n16-1026}
}

References

39 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Martın Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, et al. 2015. TensorFlow: Large-scale Machine Learning on Heterogeneous Systems. Software available from tensorflow.org. find
  2. Bharat Ram Ambati, Tejaswini Deoskar, Mark Johnson, and Mark Steedman. 2015. An Incremental Algorithm for Transition-based CCG Parsing. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for C… find
  3. Bharat Ram Ambati, Tejaswini Deoskar, and Mark Steedman. 2016. Shift-Reduce CCG Parsing using Neural Network Models. In Proceedings of the Human Language Technology Conference of the NAACL, Companion Volume: Short Papers. find
  4. Michael Auli and Adam Lopez. 2011a. A Comparison of Loopy Belief Propagation and Dual Decomposition for Integrated CCG Supertagging and Parsing. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1. find
  5. Michael Auli and Adam Lopez. 2011b. Training a log-linear parser with loss functions via softmax-margin. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. find
  6. Jason Baldridge and Geert-Jan M Kruijff. 2003. Multimodal Combinatory Categorial Grammar. In Proceedings of the tenth conference on European chapter of the Association for Computational Linguistics-Volume 1. find
  7. Srinivas Bangalore and Aravind K. Joshi. 1999. Supertagging: An approach to almost parsing. Computational Linguistics, 25(2). find
  8. John Canny, David Hall, and Dan Klein. 2013. A multi-teraflop constituency parser using gpus. Architecture, 3:3–5. find
  9. Ciprian Chelba, Tomas Mikolov, Mike Schuster, Qi Ge, Thorsten Brants, Phillipp Koehn, and Tony Robinson. 2013. One Billion Word Benchmark for Measuring Progress in Statistical Language Modeling. Technical report, Google. find
  10. Danqi Chen and Christopher D Manning. 2014. A Fast and Accurate Dependency Parser using Neural Networks. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), volume 1, pages 740–750. find our card
  11. Jinho D. Choi, Joel Tetreault, and Amanda Stent. 2015. It Depends: Dependency Parser Comparison Using A Web-based Evaluation Tool. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Vol… find our card
  12. Stephen Clark and James R Curran. 2007. Wide-coverage Efficient Statistical Parsing with CCG and Log-Linear Models. Computational Linguistics, 33(4). find our card
  13. Stephen Clark. 2002. Supertagging for Combinatory Categorial Grammar. In Proceedings of the 6th International Workshop on Tree Adjoining Grammars and Related Frameworks (TAG+ 6), pages 19–24. find
  14. Greg Durrett and Dan Klein. 2015. Neural CRF Parsing. In Proceedings of the Association for Computational Linguistics. find
  15. Chris Dyer, Miguel Ballesteros, Wang Ling, Austin Matthews, and Noah A. Smith. 2015. Transition-based Dependency Parsing with Stack Long Short-Term Memory. In Proc. find our card
  16. ACL. Timothy AD Fowler and Gerald Penn. 2010. Accurate Context-free Parsing with Combinatory Categorial Grammar. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics. find
  17. David Hall, Taylor Berg-Kirkpatrick, and Dan Klein. 2014. Sparser, better, faster gpu parsing. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 208–217, Baltimore, Maryland, June. Association for Computational Linguistics. find
  18. Hua He, Jimmy Lin, and Adam Lopez. 2015. Gappy Pattern Matching on GPUs for On-Demand Extraction of Hierarchical Translation Grammars. TACL, 3:87– 100. find
  19. James Henderson, Paola Merlo, Ivan Titov, and Gabriele Musillo. 2013. Multi-lingual Joint Parsing of Syntactic and Semantic Dependencies with a Latent Variable Model. find
  20. Computational Linguistics. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long Short-term Memory. Neural computation, 9(8):1735– 1780. find
  21. Julia Hockenmaier and Mark Steedman. 2007. CCGbank: a Corpus of CCG derivations and Dependency Structures Extracted from the Penn Treebank. Computational Linguistics, 33(3). find our card
  22. Julia Hockenmaier. 2003. Data and models for statistical parsing with Combinatory Categorial Grammar. Ph.D. thesis, University of Edinburgh. College of Science and Engineering. School of Informatics. find
  23. Mark Johnson. 2011. Parsing in Parallel on Multiple Cores and GPUs. In Proceedings of the Australasian Language Technology Association Workshop 2011, pages 29–37, Canberra, Australia, December. find
  24. Mike Lewis and Mark Steedman. 2014a. A* CCG Parsing with a Supertag-factored Model. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. find our card
  25. Mike Lewis and Mark Steedman. 2014b. Improved CCG parsing with Semi-supervised Supertagging. Transactions of the Association for Computational Linguistics. find
  26. Mike Lewis, Luheng He, and Luke Zettlemoyer. 2015. Joint A* CCG Parsing and Semantic Role Labelling. In Empirical Methods in Natural Language Processing. find
  27. Wang Ling, Chris Dyer, Alan W. Black, Isabel Trancoso, Ramon Fermandez, Silvio Amir, Luís Marujo, and Tiago Luís. 2015. Finding function in form: Compositional character models for open vocabulary word representation. In EMNLP, pages 1520–1530. The Association for Computational Linguistics. find
  28. Ryan T McDonald, Joakim Nivre, Yvonne Quirmbach-Brundage, Yoav Goldberg, Dipanjan Das, Kuzman Ganchev, Keith B Hall, Slav Petrov, Hao Zhang, Oscar Täckström, et al. 2013. Universal Dependency Annotation for Multilingual Parsing. In ACL (2), pages 92–97. find
  29. Laura Rimell and Stephen Clark. 2008. Adapting a Lexicalized-grammar Parser to Contrasting Domains. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. find
  30. Richard Socher, John Bauer, Christopher D Manning, and Andrew Y Ng. 2013. Parsing with Compositional Vector Grammars. In Proceedings of the ACL conference. find
  31. The Syntactic Process. MIT Press. Joseph Turian, Lev Ratinov, and Yoshua Bengio. 2010. Word representations: A Simple and General Method for Semi-supervised Learning. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics. find
  32. Ashish Vaswani, Yonatan Bisk, Kenji Sagae, and Ryan Musa. 2016. Supertagging With LSTMs . In Proceedings of the Human Language Technology Conference of the NAACL. find
  33. Oriol Vinyals, Lukasz Kaiser, Terry Koo, Slav Petrov, Ilya Sutskever, and Geoffrey Hinton. 2015. Grammar as a Foreign Language. In Advances in Neural Information Processing Systems. find
  34. David Weiss, Chris Alberti, Michael Collins, and Slav Petrov. 2015. Structured Training for Neural Network Transition-Based Parsing. In Proceedings of ACL 2015, pages 323–333. find
  35. Wenduan Xu, Stephen Clark, and Yue Zhang. 2014. Shift-Reduce CCG Parsing with a Dependency Model. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (ACL 2014). find
  36. Wenduan Xu, Michael Auli, and Stephen Clark. 2015. CCG Supertagging with a Recurrent Neural Network. Volume 2: Short Papers, page 250. find
  37. Wenduan Xu, Michael Auli, and Stephen Clark. 2016. Shift-Reduce CCG Parsing with Recurrent Neural Networks and Expected F-Measure Training. In Proceedings of the Human Language Technology Conference of the NAACL. find
  38. Youngmin Yi, Chao-Yue Lai, Slav Petrov, and Kurt Keutzer. 2011. Efficient parallel cky parsing on gpus. In Proceedings of the 12th International Conference on Parsing Technologies, IWPT ’11, pages 175–185, Stroudsburg, PA, USA. Association for Computational Linguistics. find
  39. Yue Zhang and Stephen Clark. 2011. Shift-reduce CCG Parsing. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2016-lewis-lstm-ccg-parsing/