Twitter Universal Dependency Parsing for African-American and Mainstream American English

What it is about

The authors ask how well UD parsers handle tweets written in African-American English (AAE) compared with Mainstream American English. They annotate 500 tweets in UD 2.0, 250 sampled as strongly AAE and 250 as strongly White, with explicit rules for Twitter conventions and for AAE constructions such as dropped copulas, habitual "be", "finna" and "gone"/"done" auxiliaries. Two parsers (UDPipe and Deep Biaffine) are trained on the UD English treebank with cross-domain tricks: coarse Twitter tags from the ARK tagger instead of morphological tags, Twitter word2vec embeddings, and rule-generated synthetic data. The ARK setting shrinks the AA/WH LAS gap from 14.0–15.7 points to 5.9–8.4, and training on the UD treebank together with the in-domain tweets raises LAS by 8.1 points over tweets alone. The gain comes mostly on the AA tweets, and the synthetic AAE data did not help the lexical variants.

Ideas

• I1. en/src/ptag.rs: test a coarse, Twitter-style tag setting (no tense or number, as in the ARK tagger) as the tag input to en parsing; metric: LAS on the 250 AA and 250 WH tweets separately, reporting the gap (test only, unless the tweet licence allows more).
• I2. en/seeds/lexicon: a seed list of AAE and Twitter spellings (da/tha, dis, ion, yo, finna, bouta, tryna, gonna) mapped to standard lemmas and UPOS; metric: lemma and UPOS accuracy on the AA subset, and the number of variant tokens that change tag.
• I3. en/src/morph.rs and the en explain output: keep the copula check (nsubj attached to the predicate when the copula is dropped) as a measured feature; metric: dropped-copula attachment accuracy on the 44 AA cases in the paper's set, never on training data.
• I4. Synthetic AAE data as a training input is a rejected idea here: the paper saw it hurt lexical variants, so the Mova rule is to test any rule-made training data on a held-out tweet set first, and keep it only if the gap and the variant accuracy both improve.

For Mova

The chapters are 08. National varieties and 37-register-and-variation.md, with 19-lexicon-mwe-register.md for the spellings. The code is en/src/ptag.rs, en/src/morph.rs and the seed lexicon. Tweets are a dialect and register test for the shipped en model, not a training source. Measure the AA/WH gap first.

Topics: Dependency parsing

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/p18-1131

@inproceedings{blodgett2018twitter,
  title = {{Twitter Universal Dependency Parsing for African-American and Mainstream American English}},
  author = {Blodgett, Su Lin and Wei, Johnny and O'Connor, Brendan},
  booktitle = {Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2018},
  doi = {10.18653/v1/p18-1131},
  url = {https://doi.org/10.18653/v1/p18-1131}
}

References

35 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Su Lin Blodgett, Lisa Green, and Brendan O’Connor. 2016. Demographic dialectal variation in social media: A case study of African-American English. Proceedings of EMNLP. find
2. Su Lin Blodgett and Brendan O’Connor. 2017. Racial disparity in natural language processing: A case study of social media African-American English. arXiv preprint arXiv:1707.00061; presented at Fairness, Accountability, and Transparency in Machine Learning (FAT/ML) workshop at KDD 2017. ↗
3. David Chiang, Mona Diab, Nizar Habash, Owen Rambow, and Safiullah Shareef. 2006. Parsing Arabic dialects. In Proceedings of EACL. find our card
4. Sam Corbett-Davies, Emma Pierson, Avi Feller, Sharad Goel, and Aziz Huq. 2017. Algorithmic decision making and the cost of fairness. In Proceedings of the KDD. ACM. find
5. Joachim Daiber and Rob van der Goot. 2016. The de-noised web treebank: Evaluating dependency parsing under noisy input conditions. In Proceedings of LREC. find
6. Leon Derczynski, Alan Ritter, Sam Clark, and Kalina Bontcheva. 2013. Twitter part-of-speech tagging for all: Overcoming sparse and noisy data. In Recent Advances in Natural Language Processing, pages 198–206. find
7. Timothy Dozat and Christopher D Manning. 2016. Deep biaffine attention for neural dependency parsing. Proceedings of ICLR. find our card
8. Timothy Dozat, Peng Qi, and Christopher D. Manning. 2017. Stanford’s graph-based neural dependency parser at the CoNLL 2017 shared task. In Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies. find our card
9. Jacob Eisenstein. 2013. What to do about bad language on the internet. In HLT-NAACL, pages 359–369. find
10. Jennifer Foster, Özlem Çetinoglu, Joachim Wagner, Joseph Le Roux, Stephen Hogan, Joakim Nivre, Deirdre Hogan, and Josef Van Genabith. 2011. # hardtoparse: Pos tagging and parsing the twitter-verse. In AAAI 2011 Workshop on Analyzing Microtext. find
11. Rob van der Goot and Gertjan van Noord. 2017. Parser adaptation for social media by integrating normalization. In Proceedings of ACL. find
12. Lisa Green, Toya A Wyatt, and Qiuana Lopez. 2007. Event arguments and ‘be’ in child African American English. University of Pennsylvania Working Papers in Linguistics, 13(2):8. find
13. Lisa J Green. 2002. African American English: A linguistic introduction. Cambridge University Press. find
14. Spence Green and Christopher D Manning. 2010. Better arabic parsing: Baselines, evaluations, and analysis. In Proceedings of COLING. ACL. find our card
15. Taylor Jones. 2015. Toward a description of African American Vernacular English dialect regions using “Black Twitter”. American Speech, 90(4). find
16. Anna Jørgensen, Dirk Hovy, and Anders Søgaard. 2016. Learning a pos tagger for aave-like language. In Proceedings of NAACL. Association for Computational Linguistics. find our card
17. David Jurgens, Yulia Tsvetkov, and Dan Jurafsky. 2017. Incorporating dialectal variability for socially equitable language identification. In Proceedings of ACL. find
18. Mohammad Khan, Markus Dickinson, and Sandra Kübler. 2013. Towards domain adaptation for parsing web data. In RANLP. find
19. Jon Kleinberg, Sendhil Mullainathan, and Manish Raghavan. 2017. Inherent trade-offs in the fair determination of risk scores. Proceedings of ITCS. find
20. Lingpeng Kong, Nathan Schneider, Swabha Swayamdipta, Archna Bhatia, Chris Dyer, and Noah A. Smith. 2014. A dependency parser for tweets. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1001–1012, Doha, Qatar. Association for Computational Linguistics. find our card
21. Yijia Liu, Yi Zhu, Wanxiang Che, Bing Qin, Nathan Schneider, and Noah A Smith. 2018. Parsing tweets into universal dependencies. Proceedings of NAACL. find our card
22. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781. ↗
23. Alexis Nasr, Geraldine Damnati, Aleksandra Guerraz, and Frederic Bechet. 2016. Syntactic parsing of chat language in contact center conversation corpus. In Annual SIGdial Meeting on Discourse and Dialogue. find
24. Joakim Nivre, Marie-Catherine de Marneffe, Filip Ginter, Yoav Goldberg, Jan Hajic, Christopher D Manning, Ryan McDonald, Slav Petrov, Sampo Pyysalo, Natalia Silveira, Reut Tsarfaty, and Daniel Zeman. 2016. Universal Dependencies v1: A multilingual treebank collection. In Proceedings of the 10th International Conference on Language Resources and Evaluation (LREC 2016). find our card
25. Brendan O’Connor, Michel Krieger, and David Ahn. 2010. TweetMotif: Exploratory search and topic summarization for Twitter. In Proceedings of the International AAAI Conference on Weblogs and Social Media. find
26. Olutobi Owoputi, Brendan O’Connor, Chris Dyer, Kevin Gimpel, Nathan Schneider, and Noah A. Smith. 2013. Improved part-of-speech tagging for online conversational text with word clusters. In Proceedings of NAACL. find
27. Manuela Sanguinetti, Cristina Bosco, Alessandro Mazzei, Alberto Lavelli, and Fabio Tamburini. 2017. Annotating Italian social media texts in universal dependencies. In Proceedings of Depling 2017. find
28. Yves Scherrer. 2011. Syntactic transformations for Swiss German dialects. In Proceedings of the First Workshop on Algorithms and Resources for Modelling of Dialects and Language Varieties. ACL. find
29. Nathan Schneider, Brendan O’Connor, Naomi Saphra, David Bamman, Manaal Faruqui, Noah A. Smith, Chris Dyer, and Jason Baldridge. 2013. A framework for (under)specifying dependency syntax without overloading annotators. In Proceedings of the 7th Linguistic Annotation Workshop and Interoperability with Discourse, pages 51–60, Sofia, Bulgaria. Association for Computational Linguistics. find
30. Ian Stewart. 2014. Now we stronger than ever: African-american english syntax in twitter. In Proceedings of the Student Research Workshop at the 14th Conference of the European Chapter of the Association for Computational Linguistics, pages 31– 37, Gothenburg, Sweden. Association for Computational Linguistics. find
31. Milan Straka, Jan Hajic, and Jana Straková. 2016. Udpipe: Trainable pipeline for processing conll-u files performing tokenization, morphological analysis, pos tagging and parsing. In Proceedings of LREC. find our card
32. Hongmin Wang, Yue Zhang, GuangYong Leonard Chan, Jie Yang, and Hai Leong Chieu. 2017. Universal dependencies parsing for colloquial Singaporean english. Proceedings of ACL. find our card
33. Aaron Steven White, Drew Reisinger, Keisuke Sakaguchi, Tim Vieira, Sheng Zhang, Rachel Rudinger, Kyle Rawlins, and Benjamin Van Durme. 2016. Universal decompositional semantics on universal dependencies. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1713–1723, Austin, Texas. Association for Computational Linguistics. find our card
34. Daniel Zeman, Martin Popel, Milan Straka, Jan Hajic, Joakim Nivre, Filip Ginter, Juhani Luotolahti, Sampo Pyysalo, Slav Petrov, Martin Potthast, et al. 2017. CoNLL 2017 shared task: Multilingual parsing from raw text to universal dependencies. Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies. find our card
35. Congle Zhang, Tyler Baldwin, Howard Ho, Benny Kimelfeld, and Yunyao Li. 2013. Adaptive parser-centric text normalization. In Proceedings of ACL. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/ud-2018-blodgett-twitter-universal-dependency-parsing-african/