Mova0.5 pre-release⌕
☰
home / docs / papers / English / Modeling the Evolution of English Noun Compounds with Feature-Rich Diachronic Compositionality Prediction

Modeling the Evolution of English Noun Compounds with Feature-Rich Diachronic Compositionality Prediction

What it is about

Noun compounds such as "snowman" can lose their compositional meaning over time. The paper represents each English compound as a vector of time-specific values, computed from its uses in the Corpus of Historical American English. It builds a wide set of features (frequency, productivity, context and transformer-based), and classifies 120 gold-standard compounds for present-day compositionality. The best results come from BERT-derived features, and the feature sets are complementary. The authors find that the rise of low-compositional meanings goes together with a drop in compositionality and with sustained semantic change. Compositionality estimates need less context in the transformer than semantic-change estimates. Evaluation uses repeated 5-fold cross-validation with 10 repetitions. The accuracy is highest for compound-level features, then modifier, then head.

Ideas

For Mova

The chapter is 18. Compounding, conversion and the minor processes, with 19. The lexicon: vocabulary, multiword expressions and register for the lexicon side. The code is en/src/dict.rs (the compound entries) and en/src/morph.rs (compound analysis). Measure: accuracy on the gold compounds under repeated 5-fold cross-validation, run twice, with a shuffled-label control.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.acl-long.984

@inproceedings{miletic2025modeling,
  title = {{Modeling the Evolution of English Noun Compounds with Feature-Rich Diachronic Compositionality Prediction}},
  author = {Miletić, Filip and Walde, Sabine Schulte im},
  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.acl-long.984},
  url = {https://doi.org/10.18653/v1/2025.acl-long.984}
}

References

45 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Reem Alatrash, Dominik Schlechtweg, Jonas Kuhn, and Sabine Schulte im Walde. 2020. CCOHA: Clean corpus of historical American English. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 6958–6966, Marseille, France. European Language Resources Association. find our card
  2. Pegah Alipoor and Sabine Schulte im Walde. 2020. Variants of vector space reductions for predicting the compositionality of English noun compounds. In Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 4379– 4387, Marseille, France. European Language Resources Association. find
  3. Timothy Baldwin and Su Nam Kim. 2010. Multiword expressions. In Nitin Indurkhya and Fred J. Damerau, editors, Handbook of Natural Language Processing, pages 267–292. CRC Press, Boca Raton, USA. find
  4. Lars Buijtelaar and Sandro Pezzelle. 2023. A psycholinguistic analysis of BERT’s representations of compounds. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2230–2241, Dubrovnik, Croatia. Association for Computational Linguistics. find
  5. Joan Bybee. 2015. Language Change. Cambridge Textbooks in Linguistics. Cambridge University Press, Cambridge, UK. find
  6. Pierluigi Cassotti, Lucia Siciliani, Marco DeGemmis, Giovanni Semeraro, and Pierpaolo Basile. 2023. XL-LEXEME: WiC pretrained model for cross-lingual LEXical sEMantic changE. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 1577–1585, Toronto, Canada. Association for Computational Linguistics. find
  7. Andy Coenen, Emily Reif, Ann Yuan, Been Kim, Adam Pearce, Fernanda Viégas, and Martin Wattenberg. 2019. Visualizing and measuring the geometry of BERT. In Advances in Neural Information Processing Systems, volume 32. Curan Associates. find
  8. Silvio Cordeiro, Aline Villavicencio, Marco Idiart, and Carlos Ramisch. 2019. Unsupervised compositionality prediction of nominal compounds. Computational Linguistics, 45(1):1–57. find
  9. Verna Dankers, Christopher Lucas, and Ivan Titov. 2022. Can transformer be too compositional? Analysing idiom processing in neural machine translation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3608–3626, Dublin, Ireland. Association for Computational Linguistics. find
  10. Mark Davies. 2012. Expanding horizons in historical linguistics with the 400-million word Corpus of Historical American English. Corpora, 7(2):121–157. find
  11. Nivja H. de Jong, Laurie B. Feldman, Robert Schreuder, Matthew Pastizzo, and R.Harald Baayen. 2002. The processing and representation of Dutch and English compounds: Peripheral morphological and central orthographic effects. Brain and Language, 81(1):555– 567. find
  12. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. ↗
  13. Prajit Dhar, Janis Pagel, and Lonneke van der Plas. 2019. Measuring the compositionality of noun-noun compounds over time. In Proceedings of the 1st International Workshop on Computational Approaches to Historical Language Change, pages 234–239, Florence, Italy. Association for Computational Linguistics. find
  14. Prajit Dhar and Lonneke van der Plas. 2019. Learning to predict novel noun-noun compounds. In Proceedings of the Joint Workshop on Multiword Expressions and WordNet (MWE-WN 2019), pages 30–39, Florence, Italy. Association for Computational Linguistics. find
  15. Haim Dubossarsky, Daphna Weinshall, and Eitan Grossman. 2017. Outta control: Laws of semantic change and inherent biases in word representation models. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1136–1145, Copenhagen, Denmark. Association for Computational Linguistics. find
  16. Marcos Garcia, Tiago Kramer Vieira, Carolina Scarton, Marco Idiart, and Aline Villavicencio. 2021a. Assessing the representations of idiomaticity in vector models with a noun compound dataset labeled at type and token levels. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 2730–2741, Online. Association for Computational Linguistics. find
  17. Marcos Garcia, Tiago Kramer Vieira, Carolina Scarton, Marco Idiart, and Aline Villavicencio. 2021b. Probing for idiomaticity in vector space models. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 3551–3564, Online. Association for Computational Linguistics. find
  18. Mario Giulianelli, Marco Del Tredici, and Raquel Fernández. 2020. Analysing lexical semantic change with contextualised word representations. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3960– 3973, Online. Association for Computational Linguistics. find
  19. Hila Gonen, Ganesh Jawahar, Djamé Seddah, and Yoav Goldberg. 2020. Simple, interpretable and stable method for detecting words with usage change across corpora. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 538–555, Online. Association for Computational Linguistics. find
  20. Kristina Gulordava and Marco Baroni. 2011. A distributional similarity approach to the detection of semantic change in the Google Books Ngram corpus. In Proceedings of the GEMS 2011 Workshop on GEometrical Models of Natural Language Semantics, pages 67–71, Edinburgh, UK. Association for Computational Linguistics. find
  21. William L. Hamilton, Jure Leskovec, and Dan Jurafsky. 2016a. Cultural shift or linguistic drift? Comparing two computational measures of semantic change. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2116–2121, Austin, Texas. Association for Computational Linguistics. find
  22. William L. Hamilton, Jure Leskovec, and Dan Jurafsky. 2016b. Diachronic word embeddings reveal statistical laws of semantic change. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1489–1501, Berlin, Germany. Association for Computational Linguistics. find
  23. Kasra Hosseini, Kaspar Beelen, Giovanni Colavizza, and Mariona Coll Ardanuy. 2021. Neural language models for nineteenth-century English. Journal of Open Humanities Data, 7:22. find
  24. Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S. Weld, Luke Zettlemoyer, and Omer Levy. 2020. Span-BERT: Improving pre-training by representing and predicting spans. Transactions of the Association for Computational Linguistics, 8:64–77. find
  25. Urban Knupleš, Diego Frassinelli, and Sabine Schulte im Walde. 2023. Investigating the nature of disagreements on mid-scale ratings: A case study on the abstractness-concreteness continuum. In Proceedings of the 27th Conference on Computational Natural Language Learning (CoNLL), pages 70–86, Singapore. Association for Computational Linguistics. find
  26. Severin Laicher, Sinan Kurtyigit, Dominik Schlechtweg, Jonas Kuhn, and Sabine Schulte im Walde. 2021. Explaining and improving BERT performance on lexical semantic change detection. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Student Research Workshop, pages 192–202, Online. Association for Computational Linguistics. find
  27. Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2020. ALBERT: A lite BERT for self-supervised learning of language representations. In Proceedings of the Eighth International Conference on Learning Representations (ICLR 2020). find
  28. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized BERT pretraining approach. arXiv:1907.11692. ↗
  29. Samin Mahdizadeh Sani, Malak Rassem, Chris W. Jenkins, Filip Miletić, and Sabine Schulte im Walde. 2024. What can diachronic contexts and topics tell us about the present-day compositionality of English noun compounds? In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 17449–17458, Torino, Italia. ELRA and ICCL. find
  30. Enrique Manjavacas and Lauren Fonteyn. 2022. Adapting vs. pre-training language models for historical languages. Journal of Data Mining & Digital Humanities. find
  31. Maximilian Maurer, Chris Jenkins, Filip Miletic, and Sabine Schulte im Walde. 2023. Classifying noun compounds for present-day compositionality: Contributions of diachronic frequency and productivity patterns. In Proceedings of the 19th Conference on Natural Language Processing, Ingolstadt, Germany. find
  32. Tomás Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. In 1st International Conference on Learning Representations, ICLR 2013, Workshop Track Proceedings, Scottsdale, Arizona, USA. find
  33. Filip Miletić and Sabine Schulte im Walde. 2024. Semantics of multiword expressions in transformer-based models: A survey. Transactions of the Association for Computational Linguistics, 11:593–612. find
  34. Filip Miletić and Sabine Schulte im Walde. 2023. A systematic search for compound semantics in pretrained BERT architectures. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 1499– 1512, Dubrovnik, Croatia. Association for Computational Linguistics. find
  35. F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830. find
  36. Tiago P. Peixoto. 2019. Bayesian stochastic blockmodeling. In Patrick Doreian, Vladimir Batagelj, and Anuska Ferligoj, editors, Advances in Network Clustering and Blockmodeling, chapter 11. Wiley Online Library. find
  37. Lewis Pollock. 2018. Statistical and methodological problems with concreteness and other semantic variables: A list memory experiment case study. Behavior Research Methods, 50(3):1198–1216. find
  38. Siva Reddy, Diana McCarthy, and Suresh Manandhar. 2011. An empirical study on compositionality in compound nouns. In Proceedings of 5th International Joint Conference on Natural Language Processing, pages 210–218, Chiang Mai, Thailand. Asian Federation of Natural Language Processing. find
  39. Radim Řehůřek and Petr Sojka. 2010. Software framework for topic modelling with large corpora. In Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks, pages 45–50, Valletta, Malta. ELRA. find
  40. Guy D. Rosin, Ido Guy, and Kira Radinsky. 2022. Time masking for temporal language models. In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining, WSDM ’22, page 833–841, New York, NY, USA. Association for Computing Machinery. find
  41. Bahar Salehi, Paul Cook, and Timothy Baldwin. 2014. Using distributional similarity of multi-way translations to predict multiword expression compositionality. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, pages 472–481, Gothenburg, Sweden. Association for Computational Linguistics. find
  42. Bahar Salehi, Paul Cook, and Timothy Baldwin. 2015. A word embedding approach to predicting the compositionality of multiword expressions. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 977–983, Denver, Colorado. Association for Computational Linguistics. find
  43. Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter. In 5th Workshop on Energy Efficient Machine Learning and Cognitive Computing – NeurIPS 2019. find
  44. Dominik Schlechtweg, Anna Hätty, Marco Del Tredici, and Sabine Schulte im Walde. 2019. A wind of change: Detecting and evaluating lexical semantic change across times and domains. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 732–746, Florence, Italy. Association for Computational Linguistics. find
  45. Sabine Schulte im Walde, Anna Hätty, and Stefan Bott. 2016. The role of modifier and head properties in predicting the compositionality of English and German noun-noun compounds: A vector-space perspective. In Proceedings of the Fifth Joint Conference on Lexical and Computational Semantics, pages 148–158, Berlin, Germany. Association for Computational Linguistics. find our card

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/miletic-2025-modeling-evolution-english-noun-compounds/