Mova0.5 pre-release⌕
☰
home / docs / papers / English / AMALGUM – A Free, Balanced, Multilayer English Web Corpus

AMALGUM – A Free, Balanced, Multilayer English Web Corpus

What it is about

AMALGUM is an English web corpus of about 4 million tokens, built from openly licensed online sources in eight genres (news, interviews, travel guides, how-to guides, academic papers, fiction, biographies and forum posts), with about 500k tokens per genre. Its documents follow the size and style rules of the GUM corpus. The text is annotated automatically with tokens, tags, lemmas, UD parses, entities and coreference, and RST discourse trees. The central claim is that retraining on genre data, stacking several taggers and using features from other layers gives output better than out-of-the-box NLP: tagging accuracy on hand-checked AMALGUM snippets rises from 93.99 to 97.37, and parsing LAS from 81.41 to 85.77. Coreference and discourse parsing remain much weaker, and the authors say so.

Ideas

For Mova

The corpus is NC, so it can be used for tests and measurements only, and nothing trained may be taken from it. The ideas touch the annot crate, corpus/, tools/treeagree and the genre chapter 37. Register and variation. The first measurement is I3, because it needs no training.

Topics: Part-of-speech tagging

Cite

The paper itself, not this card. BibTeX from the library's record of the work.

@inproceedings{gessler2020amalgum,
  title = {{AMALGUM – A Free, Balanced, Multilayer English Web Corpus}},
  author = {Gessler, Luke and Peng, Siyao and Liu, Yang and Zhu, Yilun and Behzad, Shabnam and Zeldes, Amir},
  booktitle = {Proceedings of the Twelfth Language Resources and Evaluation Conference},
  year = {2020},
  url = {https://aclanthology.org/2020.lrec-1.648/}
}

References

38 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Akbik, A., Bergmann, T., and Vollgraf, R. (2019). Pooled contextualized embeddings for named entity recognition. In Proceedings of NAACL 2019, pages 724–728. find
  2. Asheghi, N. R., Sharoff, S., and Markert, K. (2014). Designing and evaluating a reliable corpus of web genres via crowd-sourcing. In LREC, pages 1339–1346. find
  3. Baroni, M., Bernardini, S., Ferraresi, A., and Zanchetta, E. (2009). The WaCky wide web: A collection of very large linguistically processed web-crawled corpora. LREC, 43(3):209–226. find
  4. Berzak, Y., Huang, Y., Barbu, A., Korhonen, A., and Katz, B. (2016). Anchoring and agreement in syntactic annotations. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2215–2224. Association for Computational Linguistics. find
  5. Biber, D. and Conrad, S. (2009). Register, Genre, and Style. Cambridge Textbooks in Linguistics. Cambridge University Press, Cambridge. find
  6. Carlson, L., Marcu, D., and Okurowski, M. E. (2003). Building a discourse-tagged corpus in the framework of rhetorical structure theory. In Current and New Directions in Discourse and Dialogue, Text, Speech and Language Technology 22, pages 85–112. Kluwer, Dordrecht. find our card
  7. Charniak, E., Blaheta, D., Ge, N., Hall, K., and Johnson, M. (2000). BLLIP Corpus. LDC2000T43. find
  8. Chen, T. and Guestrin, C. (2016). XGBoost: A scalable tree boosting system. In KDD ’16 Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 785–794, San Francisco, CA. find
  9. Dalan, E. and Sharoff, S. (2016). Genre classification for a corpus of academic webpages. In Proceedings of the 10th Web as Corpus Workshop, pages 90–98. find
  10. Derczynski, L., Ritter, A., Clark, S., Bontcheva, K., and Bontcheva, K. (2012). Twitter part-of-speech tagging for all: Overcoming sparse and noisy data. In Proceedings of RANLP 2012, pages 198–206, Hissar, Bulgaria. find
  11. Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. ↗
  12. Hundt, M. (2008). Text corpora. In Anke Lüdeling et al., editors, Corpus Linguistics. An International Handbook, pages 168–187. Mouton de Gruyter. find
  13. Ide, N., Baker, C., Fellbaum, C., and Passonneau, R. (2010). The manually annotated sub-corpus: A community resource for and by the people. In Proceedings of ACL 2010, pages 68–73, Uppsala, Sweden. find
  14. Ji, Y. and Eisenstein, J. (2014). Representation learning for text-level discourse parsing. In Proceedings of ACL 2014, pages 13–24, Baltimore, MD. find
  15. Joshi, M., Levy, O., Weld, D. S., and Zettlemoyer, L. (2019). Bert for coreference resolution: Baselines and analysis. ArXiv, abs/1908.09091. ↗
  16. Kennedy, G. (1998). An Introduction to Corpus Linguistics. Longman, London. find
  17. Kuncoro, A., Ballesteros, M., Kong, L., Dyer, C., and Smith, N. A. (2016). Distilling an ensemble of greedy dependency parsers into one MST parser. In Proceedings of EMNLP 2016, pages 1744–1753, Austin, TX. find
  18. Lee, K., He, L., and Zettlemoyer, L. (2018). Higher-order coreference resolution with coarse-to-fine inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 687–692, New Orleans, Louisiana, June. Association for Computational Linguistics. find
  19. Leech, G., McEnery, T., and Weisser, M. (2003). Spaac speech-act annotation scheme. Technical report, Lancaster University, Lancaster University. find
  20. Mann, W. C. and Thompson, S. A. (1988). Rhetorical structure theory: Toward a functional theory of text organization. Text, 8(3):243–281. find our card
  21. Marcu, D. and Echihabi, A. (2002). An unsupervised approach to recognizing discourse relations. In Proceedings of ACL 2002, pages 368–375, Philadelphia, PA. find
  22. Morey, M., Muller, P., and Asher, N. (2017). How much progress have we made on RST discourse parsing? a replication study of recent results on the RST-DT. In Proceedings of EMNLP 2017, pages 1319–1324, Copenhagen, Denmark. find our card
  23. Muller, P., Braud, C., and Morey, M. (2019). ToNy: Contextual embeddings for accurate multilingual discourse segmentation of full documents. In Proceedings of Discourse Relation Treebanking and Parsing (DISRPT 2019), pages 115–124, Minneapolis, MN. Nivre, J., Željko Agić, Ahrenberg, L., Aranzabe, M. J., Asahara, M., Atutxa, A., Ballesteros, M., Bauer, J., Bengoetxea, K., Bhat, R. A., Bick, E., Bosco, C., Bouma, G., Bowman, S., Candito, M., Eryiğit, G. C., Celano, G. G. A., Chalub, F., Choi, J., Çağrı Çöltekin, Connor, M., Davidson, E., de Marneffe, M.-C., de Paiva, V., de Ilarraza, A. D., Dobrovoljc, K., Dozat, T., Droganova, K., Dwivedi, P., Eli, M., Erjavec, T., Farkas, R., Foster, J., Freitas, C., Gajdošová, K., Galbraith, D., Garcia, M., Ginter, F., Goenaga, I., Gojenola, K., Gökırmak, M., Goldberg, Y., Guinovart, X. G., Saavedra, B. G., Grioni, M., Grūzı̄tis, N., Guillaume, B., Habash, N., Hajič, J., Mỹ, L. H., Haug, D., Hladká, B., Hohle, P., Ion, R., Irimia, E., Johannsen, A., Jørgensen, F., Kaşıkara, H., Kanayama, H., Kanerva, J., Kotsyba, N., Krek, S., Laippala, V., H`ông, L., Lenci, A., Ljubešić, N., Lyashevskaya, O., Lynn, T., Makazhanov, A., Manning, C., Mărănduc, C., Mareček, D., Alonso, H. M., Martins, A., Mašek, J., Matsumoto, Y., McDonald, R., Missilä, A., Mititelu, V., Miyao, Y., Montemagni, S., More, A., Mori, S., Moskalevskyi, B., Muischnek, K., Mustafina, N., Müürisep, K., Thi., L. N., Minh, H. N. T., Nikolaev, V., Nurmi, H., Ojala, S., Osenova, P., Øvrelid, L., Pascual, E., Passarotti, M., Perez, C.-A., Perrier, G., Petrov, S., Piitulainen, J., Plank, B., Popel, M., Pretkalniņa, L., Prokopidis, P., Puolakainen, T., Pyysalo, S., Rademaker, A., Ramasamy, L., Real, L., Rituma, L., Rosa, R., Saleh, S., Sanguinetti, M., Saulı̄te, B., Schuster, S., Seddah, D., Seeker, W., Seraji, M., Shakurova, L., Shen, M., Sichinava, D., Silveira, N., Simi, M., Simionescu, R., Simkó, K., Šimková, M., Simov, K., Smith, A., Suhr, A., Sulubacak, U., Szántó, Z., Taji, D., Tanaka, T., Tsarfaty, R., Tyers, F., Uematsu, S., Uria, L., van Noord, G., Varga, V., Vincze, V., Washington, J. N., v Zabokrtský, Z., Zeldes, A., Zeman, D., and Zhu, H. (2017). Universal dependencies 2.0. Technical report, LINDAT/CLARIN digital library at the Institute of Formal and Applied Linguistics (ÚFAL), Faculty of Mathematics and Physics, Charles University. find
  24. Prasad, R., Webber, B., Lee, A., and Joshi, A. (2019). LDC2019T05. Penn Discourse Treebank version 3.0. find
  25. Qi, P., Dozat, T., Zhang, Y., and Manning, C. D. (2018). Universal Dependency parsing from scratch. In Proceedings of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pages 160–170, Brussels, Belgium. find our card
  26. Rehm, G., Santini, M., Mehler, A., Braslavski, P., Gleim, R., Stubbe, A., Symonenko, S., Tavosanis, M., and Vidulin, V. (2008). Towards a reference corpus of web genres for the evaluation of genre identification systems. In LREC. find
  27. Schäfer, R. (2015). Processing and querying large web corpora with the COW14 architecture. In Proceedings of Challenges in the Management of Large Corpora (CMLC-3), pages 28–34, Lancaster, UK. find
  28. Schulz, S. and Ketschik, N. (2019). From 0 to 10 million annotated words: Part-of-speech tagging for Middle High German. LREC, pages 1–27. find
  29. Shibuya, T. and Hovy, E. (2019). Nested Named Entity Recognition via Second-best Sequence Learning and Decoding. arXiv:1909.02250 [cs], September. arXiv: 1909.02250. ↗
  30. Toutanova, K. N. (2005). Effective Statistical Models for Syntactic and Semantic Disambiguation. Ph.D. thesis, Stanford University. find
  31. Weischedel, R., Pradhan, S., Ramshaw, L., Kaufman, J., Franchini, M., El-Bachouti, M., Xue, N., Palmer, M., Hwang, J. D., Bonial, C., Choi, J., Mansouri, A., Foster, M., aati Hawwary, A., Marcus, M., Taylor, A., Greenberg, C., Hovy, E., Belvin, R., and Houston, A. (2012). Ontonotes release 5.0. Technical report, Linguistic Data Consortium, Philadelphia. find
  32. Wolpert, D. H. (1992). Stacked generalization. Neural Networks, 5(2):241–259. find
  33. Yu, Y., Zhu, Y., Liu, Y., Liu, Y., Peng, S., Gong, M., and Zeldes, A. (2019). GumDrop at the DISRPT2019 shared task: A model stacking approach to discourse unit segmentation and connective detection. In Proceedings of Discourse Relation Treebanking and Parsing (DISRPT 2019), pages 133–143, Minneapolis, MN. find our card
  34. Zeldes, A. and Simonson, D. (2016). Different flavors of GUM: Evaluating genre and sentence type effects on multilayer corpus annotation quality. In Proceedings of LAW X – The 10th Linguistic Annotation Workshop, pages 68–78, Berlin. find our card
  35. Zeldes, A. and Zhang, S. (2016). When annotation schemes change rules help: A configurable approach to coreference resolution beyond OntoNotes. In Proceedings of the NAACL2016 Workshop on Coreference Resolution Beyond OntoNotes (CORBON), pages 92–101. find
  36. Zeldes, A. (2017). The GUM corpus: Creating multilayer resources in the classroom. LREC, 51(3):581–612. find
  37. Zeman, D., Hajič, J., Popel, M., Potthast, M., Straka, M., Ginter, F., Nivre, J., and Petrov, S. (2018). CoNLL 2018 shared task: Multilingual parsing from raw text to Universal Dependencies. In Proceedings of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pages 1–21, Brussels, Belgium. find our card
  38. Zhou, Z.-H. and Li, M. (2005). Tri-training: Exploiting unlabeled data using three classifiers. IEEE Transactions on Knowledge & Data Engineering, 17(11):1529–1541. 7. Appendix: Sample Analyses find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/lingbulk-2020-gessler-amalgum-free-balanced-multilayer-english/