Investigating NP-Chunking with Universal Dependencies for English

What it is about

Chunks are normally taken from constituency trees, but UD has no chunk layer. The paper derives minimal noun-phrase chunks (B-NP, I-NP, O tags) from EWT dependency trees by rules over the core token (noun, proper noun, some pronouns) and its children, excluding prepositional phrases. A manual check of 233 chunks in 50 EWT sentences gives 98.7% accuracy. The chunks then feed a bi-LSTM tagger trained with PoS and feature tags in a multi-task setting. Multi-task training helps PoS tagging on EWT and ParTUT but not LinES (up to +0.6 PoS and +1.73 chunking on ParTUT), and NP-chunk tags as parser features help mainly on the smallest treebank (ParTUT, +0.96 LAS; LinES +0.22 with chunks alone), while the baseline wins on EWT. The author concludes that the chunk layer is useful, but its benefit for parsing depends on the size of the training data.

Ideas

I1. en/src/ud.rs or a new en/src/chunk.rs: implement the deduction rules from the paper (minimal NP, PP excluded, conj between adjectives split) and measure the agreement with the en output on EWT test, plus a 233-chunk hand check of our own.
I2. en/src/ptag.rs: add a deduced NP-chunk tag as a feature in the perceptron tagger's window (no network needed); metric: PTB tag accuracy on EWT test, with two runs to check determinism.
I3. gparse: add an NP-chunk output head as an auxiliary task on the 4-layer transformer; metric: UPOS and LAS on EWT test against the single-task model, with two seeds, since the paper's gains are small and mixed.

For Mova

The chapters are 22. Phrases: noun, adjective, adverb and prepositional phrases and 10-word-classes.md. The code is en/src/ud.rs (a derived chunk layer costs no new data) and gparse for the multi-task head. The paper's own finding is that small training sets gain most, so measure on the small treebanks (ParTUT-size) before EWT.

Topics: Universal Dependencies · Part-of-speech tagging

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/w18-6010

@inproceedings{lacroix2018investigating,
  title = {{Investigating NP-Chunking with Universal Dependencies for English}},
  author = {Lacroix, Ophélie},
  booktitle = {Proceedings of the Second Workshop on Universal Dependencies (UDW 2018)},
  year = {2018},
  doi = {10.18653/v1/w18-6010},
  url = {https://doi.org/10.18653/v1/w18-6010}
}

References

7 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

Yoav Goldberg and Joakim Nivre. 2013. Training Deterministic Parsers with Non-Deterministic Oracles. Transactions of the Association of Computational Linguistics, 1:403–414. ↗
Alex Graves and Jürgen Schmidhuber. 2005. Frame-wise Phoneme Classification with Bidirectional LSTM and Other Neural Network Architectures. Neural Networks, pages 5–6. find
Jan Hajič and Dan Zeman, editors. 2017. Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies. Association for Computational Linguistics. find our card
Lê Hồng, Alessandro Lenci, Saran Lertpradit, Herman Leung, Cheuk Ying Li, Josie Li, Keying Li, Nikola Ljubešić, Olga Loginova, Olga Lyashevskaya, Teresa Lynn, Vivien Macketanz, Aibek Makazhanov, Michael Mandl, Christopher Manning, Cătălina Mărănduc, David Mareček, Katrin Marheinecke, Héctor Martínez Alonso, André Martins, Jan Mašek, Yuji Matsumoto, Ryan McDonald, Gustavo Mendonça, Niko Miekka, Anna Missilä, Cătălin Mititelu, Yusuke Miyao, Simonetta Montemagni, Amir More, Laura Moreno Romero, Shinsuke Mori, Bohdan Moskalevskyi, Kadri Muischnek, Kaili Müürisep, Pinkey Nainwani, Anna Nedoluzhko, Gunta Nešpore-Bērzkalne, Lương Nguyễn Thi., Huyền Nguyễn Thi. Minh, Vitaly Nikolaev, Hanna Nurmi, Stina Ojala, Petya Osenova, Robert Östling, Lilja Øvrelid, Elena Pascual, Marco Passarotti, Cenel-Augusto Perez, Guy Perrier, Slav Petrov, Jussi Piitulainen, Emily Pitler, Barbara Plank, Martin Popel, Lauma Pretkalnin , a, Prokopis Prokopidis, Tiina Puolakainen, Sampo Pyysalo, Alexandre Rademaker, Loganathan Ramasamy, Taraka Rama, Vinit Ravishankar, Livy Real, Siva Reddy, Georg Rehm, Larissa Rinaldi, Laura Rituma, Mykhailo Romanenko, Rudolf Rosa, Davide Rovati, Benoît Sagot, Shadi Saleh, Tanja Samardžić, Manuela Sanguinetti, Baiba Saulı̄te, Sebastian Schuster, Djamé Seddah, Wolfgang Seeker, Mojgan Seraji, Mo Shen, Atsuko Shimada, Dmitry Sichinava, Natalia Silveira, Maria Simi, Radu Simionescu, Katalin Simkó, Mária Šimková, Kiril Simov, Aaron Smith, Antonio Stella, Milan Straka, Jana Strnadová, Alane Suhr, Umut Sulubacak, Zsolt Szántó, Dima Taji, Takaaki Tanaka, Trond Trosterud, Anna Trukhina, Reut Tsarfaty, Francis Tyers, Sumire Uematsu, Zdeňka Urešová, Larraitz Uria, Hans Uszkoreit, Sowmya Vajjala, Daniel van Niekerk, Gertjan van Noord, Viktor Varga, Eric Villemonte de la Clergerie, Veronika Vincze, Lars Wallin, Jonathan North Washington, Mats Wirén, Tak-sum Wong, Zhuoran Yu, Zdeněk Žabokrtský, Amir Zeldes, Daniel Zeman, and Hanzhi Zhu. 2017. Universal dependencies 2.1. LINDAT/CLARIN digital library at the Institute of Formal and Applied Linguistics (ÚFAL), Faculty of Mathematics and Physics, Charles University. find
Lance Ramshaw and Mitch Marcus. 1995. Text Chunking using Transformation-Based Learning. In Third Workshop on Very Large Corpora. find
Anders Søgaard and Yoav Goldberg. 2016. Deep multi-task learning with low level tasks supervised at lower layers. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL 2016). find
Erik F. Tjong Kim Sang and Sabine Buchholz. 2000. Introduction to the CoNLL-2000 shared task: Chunking. In Proceedings of the 2nd Workshop on Learning language in logic and the 4th conference on Computational Natural Language Learning. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/ud-2018-lacroix-investigating-np-chunking-universal-dependencies/