Where New Words Are Born: Distributional Semantic Analysis of Neologisms and Their Semantic Neighborhoods

What it is about

The paper asks where new words appear in semantic space. Using word vectors trained on two periods of American English (COHA, 1800 to 1989, 385M words, and COCA, 1990 to 2012), the authors take neologisms and compare their semantic neighbourhoods with those of stable control words. Two factors are tested: semantic sparsity (how few historical words lie close to the new word) and the frequency growth rate of those neighbours, measured as the Spearman correlation of frequency with decade. Neologisms sit in sparser neighbourhoods, and the neighbours of neologisms grow faster in frequency. The comparison uses 720 neologism and control pairs, and the growth factor gives the stronger signal, although sparsity also helps.

Ideas

• I1. en/src/lexgen.rs and en/src/dict.rs: for every token in an unseen modern text that is not in the en lexicon (110,308 lemmas), count the number of known lemmas within a small co-occurrence distance, as a sparsity score. Measure whether the score separates the unknown words that are later accepted from those that are not, on a hand-labelled list.
• I2. Growth-rate feature for tools/gutenberg: for a set of words, compute the per-book frequency trend against publication date across the public-domain books, and check the sign against the known history of a few words (for example "hast" and "doth" should fall). Use this as a test of the trend function, not as a new lexicon rule.
• I3. Gap report: run the en lemmatiser (en/src/morph.rs) over the school reading texts and list the forms it cannot analyse. Measure the share that a later dictionary build would cover, as a check on the lexicon, before any neural step.

For Mova

The touch points are en/src/lexgen.rs, en/src/dict.rs and en/src/morph.rs, which handle unknown forms. The chapter is 19. The lexicon: vocabulary, multiword expressions and register, with 09. Early Modern and archaic forms for change over time. COHA and COCA are licensed corpora and are not used in the build; any measurement on them is test-only. Measure first the share of unknown forms in the school texts that the en lexicon cannot analyse.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.7275/1jra-8m83 · arXiv 2001.07740

@inproceedings{ryskina2020where,
  title = {{Where New Words Are Born: Distributional Semantic Analysis of Neologisms and Their Semantic Neighborhoods}},
  author = {Ryskina, Maria and Rabinovich, Ella and Berg-Kirkpatrick, Taylor},
  booktitle = {SCiL 2020 (Society for Computation in Linguistics)},
  year = {2020},
  doi = {10.7275/1jra-8m83},
  eprint = {2001.07740},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.7275/1jra-8m83}
}

References

34 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Jean Aitchison. 2001. Language Change: Progress Or Decay? Cambridge University Press. find
2. René Appel and Pieter Muysken. 2006. Language contact and bilingualism. Amsterdam University Press. find
3. Robert Bamler and Stephan Mandt. 2017. Dynamic word embeddings. In International Conference on Machine Learning, pages 380–389. find
4. Lyle Campbell. 2013. Historical Linguistics: an Introduction. MIT Press, Cambridge, MA. find
5. Paul Cook. 2012. Using social media to find English lexical blends. In Proceedings of the 15th EU-RALEX International Congress (EURALEX 2012), pages 846–854, Oslo, Norway. find
6. Frank E. Daulton. 2012. Lexical borrowing. In The Encyclopedia of Applied Linguistics. American Cancer Society. find
7. Mark Davies. 2002. The Corpus of Historical American English (COHA): 400 million words, 1810-2009. Brigham Young University. find
8. Mark Davies. 2008. The corpus of contemporary American English. BYE, Brigham Young University. find
9. Marco Del Tredici and Raquel Fernández. 2018. The road to success: Assessing the fate of linguistic innovations in online communities. In Proceedings of the 27th International Conference on Computational Linguistics, pages 1591–1603. find
10. Georgiana Dinu, Angeliki Lazaridou, and Marco Baroni. 2014. Improving zero-shot learning by mitigating the hubness problem. arXiv preprint arXiv:1412.6568. — [arXiv:1412.6568](https://arxiv.org/abs/1412.6568) find
11. Haim Dubossarsky, Daphna Weinshall, and Eitan Grossman. 2017. Outta control: Laws of semantic change and inherent biases in word representation models. In Proceedings of the 2017 conference on empirical methods in natural language processing, pages 1136–1145. find
12. Jacob Eisenstein. 2017. Identifying regional dialects in on-line social media. The Handbook of Dialectology, pages 368–383. find
13. Jacob Eisenstein, Brendan O’Connor, Noah A Smith, and Eric P Xing. 2014. Diffusion of lexical change in social media. PloS one, 9(11):e113114. find
14. Manaal Faruqui, Yulia Tsvetkov, Pushpendre Rastogi, and Chris Dyer. 2016. Problems with evaluation of word embeddings using word similarity tasks. In Proceedings of the 1st Workshop on Evaluating Vector-Space Representations for NLP, pages 30– 35. find
15. Nikhil Garg, Londa Schiebinger, Dan Jurafsky, and James Zou. 2018. Word embeddings quantify 100 years of gender and ethnic stereotypes. Proceedings of the National Academy of Sciences, 115(16):E3635–E3644. find
16. William L Hamilton, Jure Leskovec, and Dan Jurafsky. 2016. Diachronic word embeddings reveal statistical laws of semantic change. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 1489–1501. find
17. Kokil Jaidka, Niyati Chhaya, and Lyle Ungar. 2018. Diachronic degradation of language models: Insights from social media. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), volume 2, pages 195–200. find
18. Daniel Kershaw, Matthew Rowe, and Patrick Stacey. 2016. Towards modelling language innovation acceptance in online social networks. In Proceedings of the Ninth ACM International Conference on Web Search and Data Mining, pages 553–562. ACM. find
19. Vivek Kulkarni, Rami Al-Rfou, Bryan Perozzi, and Steven Skiena. 2015. Statistically significant detection of linguistic change. In Proceedings of the 24th International Conference on World Wide Web, pages 625–635. International World Wide Web Conferences Steering Committee. find
20. Andrey Kutuzov, Lilja Øvrelid, Terrence Szymanski, and Erik Velldal. 2018. Diachronic word embeddings and semantic shifts: a survey. In Proceedings of the 27th International Conference on Computational Linguistics, pages 1384–1397. find
21. Alessandro Lenci. 2018. Distributional models of word meaning. Annual review of Linguistics, 4:151–171. find
22. Rochelle Lieber. 2017. Derivational morphology. find
23. Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of machine learning research, 9(Nov):2579–2605. find
24. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems, pages 3111–3119. find
25. Michael Proffitt, editor. 2018. OED Online. Oxford University Press. http://www.oed.com/. ↗
26. Miloš Radovanović, Alexandros Nanopoulos, and Mirjana Ivanović. 2010. Hubs in space: Popular nearest neighbors in high-dimensional data. Journal of Machine Learning Research, 11(Sep):2487–2531. find
27. Samuel Reese, Gemma Boleda, Montse Cuadros, Lluı́s Padró, and German Rigau. 2010. Wikicorpus: A word-sense disambiguated multilingual Wikipedia corpus. In Proceedings of the Seventh conference on International Language Resources and Evaluation (LREC’10). find
28. Vincent Renner, Franois Maniez, and Pierre Arnaud, editors. 2012. Cross-disciplinary perspectives on lexical blending. De Gruyter Mouton, Berlin. find
29. Tobias Schnabel, Igor Labutov, David Mimno, and Thorsten Joachims. 2015. Evaluation methods for unsupervised word embeddings. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 298–307. find
30. Ian Stewart and Jacob Eisenstein. 2018. Making” fetch” happen: The influence of social and linguistic context on nonstandard word growth and decline. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4360–4370. find
31. Nina Tahmasebi, Lars Borin, and Adam Jatowt. 2018. Survey of computational approaches to diachronic conceptual change. arXiv preprint arXiv:1811.06278. — [arXiv:1811.06278](https://arxiv.org/abs/1811.06278) find
32. John R Taylor and Anthony P. Grant. 2014. Lexical Borrowing. Oxford University Press, Oxford. find
33. Uriel Weinreich. 2010. Languages in contact: Findings and problems. Walter de Gruyter, The Hague. find
34. Yang Xu and Charles Kemp. 2015. A computational evaluation of two laws of semantic change. In CogSci. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/lex-2020-ryskina-neologisms-distributional/