Parsing Tweets into Universal Dependencies

What it is about

The authors extend the UD guidelines to tweets: emoticons and retweet markers become non-syntactic or discourse tokens, abbreviations such as "idc" stay unnormalized as single words, multiword expressions can be merged, and tweets can have several roots. They build Tweebank v2, 3,550 tweets (four times the earlier unlabelled Tweebank v1), with agreement of 96.6% on POS, 88.8% on unlabelled and 84.3% on labelled dependencies. A character-level bi-LSTM tokenizer reaches 98.3 F1, above Stanford CoreNLP (97.3). For parsing, a 20-parser ensemble reaches 79.4 LAS over a 75.7 stack-LSTM baseline, and a single greedy parser distilled from the ensemble reaches 77.8 (77.9 when learning from the ensemble's exploration). The full pipeline scores 74.0 LAS against 71.4 for the best off-the-shelf pipeline, and losing gold tokenization costs about four points.

Ideas

• I1. en/src/tok.rs: add tweet rules (keep @-mentions, hashtags and emoticons whole, split UD-style contractions such as "gon na" and "it s", keep acronyms whole); metric: token F1 on the Tweebank v2 test set against the current tokenizer; test only, unless the licence allows more.
• I2. en/src/parse.rs: train the arc-standard perceptron with exploration (sample a transition from the model at some steps, with a dynamic oracle), as in the paper; metric: LAS on EWT dev and on the Tweebank test, run twice for determinism.
• I3. annot and treeagree: use the disagreement between two annotators or teachers as a noise flag for tweet silver labels, keeping only tweets where the teachers agree on the arc; metric: the share of kept tweets that a human confirms.
• I4. en parse-eval: report LAS on raw tweets through the full en pipeline (tokenizer, tagger, parser), not only on gold tokens; the paper shows the tokenization loss is about four points.

For Mova

The chapters are 37. Register and variation and 07-punctuation-capitals-numbers.md. The code is en/src/tok.rs, en/src/parse.rs and the annot teacher pipeline. Tweets are a register test, and the distillation idea is useful only as a measure of exploration training, not as a second model.

Topics: Universal Dependencies

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/n18-1088 · arXiv 1804.08228

@inproceedings{liu2018parsing,
  title = {{Parsing Tweets into Universal Dependencies}},
  author = {Liu, Yijia and Zhu, Yi and Che, Wanxiang and Qin, Bing and Schneider, Nathan and Smith, Noah A.},
  booktitle = {Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)},
  year = {2018},
  doi = {10.18653/v1/n18-1088},
  eprint = {1804.08228},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/n18-1088}
}

References

38 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. 2016; Ballesteros et al., 2016). We find that this approach outperforms the conventional distillation model, coming in 1.5 points behind the ensemble (last line of Table 6). find
2. Waleed Ammar, George Mulcaire, Miguel Ballesteros, Chris Dyer, and Noah A. Smith. 2016. Many languages, one parser. TACL 4. find
3. Miguel Ballesteros, Chris Dyer, and Noah A. Smith. 2015. Improved transition-based parsing by modeling characters instead of words with LSTMs. In Proc. of EMNLP. find
4. Miguel Ballesteros, Yoav Goldberg, Chris Dyer, and Noah A. Smith. 2016. Training with exploration improves a greedy stack LSTM parser. In Proc. of EMNLP. find
5. Peter F. Brown, Peter V. deSouza, Robert L. Mercer, Vincent J. Della Pietra, and Jenifer C. Lai. 1992. Class-based n-gram models of natural language. Computational Linguistics 18(4). find our card
6. Leon Derczynski, Alan Ritter, Sam Clark, and Kalina Bontcheva. 2013. Twitter part-of-speech tagging for all: Overcoming sparse and noisy data. In Proc. of RANLP 2013. find
7. Thomas G. Dietterich. 2000. An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting, and randomization. Machine Learning 40(2):139–157. https://doi.org/10.1023/A:1007607513941.14 We choose the Stanford CoreNLP tokenizer in the spirit of comparing rule-based and statistical methods. ↗
8. Adhiguna Kuncoro, Miguel Ballesteros, Lingpeng Kong, Chris Dyer, and Noah A. Smith. 2016. Distilling an ensemble of greedy dependency parsers into one MST parser. In Proc. of EMNLP. find
9. Timothy Dozat, Peng Qi, and Christopher D. Manning. 2017. Stanford’s graph-based neural dependency parser at the CoNLL 2017 shared task. In Proc. of CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies. find our card
10. Jacob Eisenstein. 2013. What to do about bad language on the internet. In Proc. of NAACL. find
11. Xuezhe Ma and Eduard Hovy. 2016. End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF. In Proc. of ACL. find
12. Jennifer Foster, Özlem Çetinoǧlu, Joachim Wagner, Joseph Le Roux, Stephen Hogan, Joakim Nivre, Deirdre Hogan, and Josef Van Genabith. 2011. #hardtoparse: POS tagging and parsing the Twitter-verse. In Proc. of the 5th AAAI Conference on Analyzing Microtext. find
13. Christopher D. Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven J. Bethard, and David Mc-Closky. 2014. The Stanford CoreNLP natural language processing toolkit. In ACL System Demonstrations. find
14. Benoît Frénay and Michel Verleysen. 2014. Classification in the presence of label noise: A survey. IEEE Transactions on Neural Networks and Learning Systems 25:845–869. find
15. Mitchell P. Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of English: The Penn Treebank. Computational Linguistics 19(2):313–330. ↗ our card
16. Kim Gerdes. 2013. Collaborative dependency annotation. In Proc. of DepLing. find
17. Joakim Nivre, Marie-Catherine de Marneffe, Filip Ginter, Yoav Goldberg, Jan Hajic, Christopher D. Manning, Ryan McDonald, Slav Petrov, Sampo Pyysalo, Natalia Silveira, Reut Tsarfaty, and Daniel Zeman. 2016. Universal Dependencies v1: A multilingual treebank collection. In Proc. of LREC. find our card
18. Kevin Gimpel, Nathan Schneider, Brendan O’Connor, Dipanjan Das, Daniel Mills, Jacob Eisenstein, Michael Heilman, Dani Yogatama, Jeffrey Flanigan, and Noah A. Smith. 2011. Part-of-speech tagging for twitter: Annotation, features, and experiments. In Proc. of ACL. find
19. Brendan O’Connor, Michel Krieger, and David Ahn. 2010. TweetMotif: Exploratory search and topic summarization for Twitter. find
20. Yoav Goldberg and Joakim Nivre. 2012. A dynamic oracle for arc-eager dependency parsing. In Proc. of COLING. find
21. Olutobi Owoputi, Brendan O’Connor, Chris Dyer, Kevin Gimpel, Nathan Schneider, and Noah A. Smith. 2013. Improved part-of-speech tagging for online conversational text with word clusters. In Proc. of NAACL. find
22. Yoav Goldberg and Joakim Nivre. 2013. Training deterministic parsers with non-deterministic oracles. TACL 1. find
23. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. GloVe: Global vectors for word representation. In Proc. of EMNLP. find
24. Tao Gui, Qi Zhang, Haoran Huang, Minlong Peng, and Xuanjing Huang. 2017. Part-of-speech tagging for Twitter with adversarial neural networks. In Proc. of EMNLP. find
25. Slav Petrov, Dipanjan Das, and Ryan McDonald. 2012. A universal part-of-speech tagset. In Proc. of LREC. find
26. Jiang Guo, Wanxiang Che, David Yarowsky, Haifeng Wang, and Ting Liu. 2015. Cross-lingual dependency parsing based on distributed representations. In Proc. of ACL. find
27. Nils Reimers and Iryna Gurevych. 2017. Reporting score distributions makes a difference: Performance study of LSTM-networks for sequence tagging. In Proc. of EMNLP. find
28. Geoffrey E. Hinton, Oriol Vinyals, and Jeffrey Dean. 2015. Distilling the knowledge in a neural network. CoRR abs/1503.02531. ↗
29. Alan Ritter, Sam Clark, Mausam, and Oren Etzioni. 2011. Named entity recognition in tweets: An experimental study. In Proc. of EMNLP. find
30. Zhiheng Huang, Wei Xu, and Kai Yu. 2015. Bidirectional LSTM-CRF models for sequence tagging. CoRR abs/1508.01991. ↗
31. Manuela Sanguinetti, Cristina Bosco, Alessandro Mazzei, Alberto Lavelli, and Fabio Tamburini. 2017. Annotating Italian social media texts in Universal Dependencies. In Proc. of Depling. Pisa, Italy. find
32. Yoon Kim and Alexander M. Rush. 2016. Sequence-level knowledge distillation. In Proc. of EMNLP. find
33. Eliyahu Kiperwasser and Yoav Goldberg. 2016. Simple and accurate dependency parsing using bidirectional LSTM feature representations. TACL 4. find our card
34. Nathan Schneider, Brendan O’Connor, Naomi Saphra, David Bamman, Manaal Faruqui, Noah A. Smith, Chris Dyer, and Jason Baldridge. 2013. A framework for (under)specifying dependency syntax without overloading annotators. In Proc. of the 7th Linguistic Annotation Workshop and Interoperability with Discourse. find
35. Lingpeng Kong, Nathan Schneider, Swabha Swayamdipta, Archna Bhatia, Chris Dyer, and Noah A. Smith. 2014. A dependency parser for tweets. In Proc. of EMNLP. find our card
36. Milan Straka and Jana Straková. 2017. Tokenizing, POS tagging, lemmatizing and parsing UD 2.0 with UDPipe. In Proc. of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies. find our card
37. Hongmin Wang, Yue Zhang, GuangYong Leonard Chan, Jie Yang, and Hai Leong Chieu. 2017. Universal Dependencies parsing for colloquial Singaporean English. In Proc. of ACL. find our card
38. Hiroyasu Yamada and Yuji Matsumoto. 2003. Statistical dependency analysis with support vector machines. In Proc. of IWPT. find our card

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/ud-2018-liu-parsing-tweets-into-universal-dependencies/