Word-Embeddings Distinguish Denominal and Root-Derived Verbs in Semitic

What it is about

Distributed Morphology posits two levels of word formation: a root plus a pattern gives unpredictable meanings (the same root in different binyanim can drift far apart), while a verb built on an existing word stays close to that word's meaning. Modern Hebrew offers a formal diagnostic (Arad): a denominal verb keeps a consonant that belongs to the noun's pattern, not to the root (מ in mixšev "computerize" from maxšev "computer"; the -on of xešbon in hitxašben). The authors build triples — noun, its denominal verb, and other verbs from the same root — from the Knesset corpus and test four embedding models (fastText, GloVe, Word2Vec, AlephBERT): the denominal is closer to the noun than the root-derived verbs are, as predicted.

Ideas

• I1. In the Torah dictionary, mark verbs that BDB labels "denom." and, where visible, the extra pattern consonant (e.g. a verb built on a מ- or ת- noun); show the source noun in the entry rather than only the root.
• I2. A cheap, explainable check of the paper's prediction on our data: gloss overlap. For each root, compare the English glosses of the noun with those of the denominal verb and of the root's other verbs; denominals should share more gloss words with their noun.

For Mova

The paper is on Modern Hebrew and uses embeddings, which Mova does not use at answer time. I1 and I2 go into macula dict and The lexicon (roots and word formation); a note on the two levels belongs to 07-weak-verbs.md/06-verb.md only as background on binyan meaning.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.4204/eptcs.366.6 · arXiv 2208.05721

@inproceedings{benbaji2022word,
  title = {{Word-Embeddings Distinguish Denominal and Root-Derived Verbs in Semitic}},
  author = {Benbaji, Ido and Doron, Omri and Henot-Mortier, Adele},
  booktitle = {EPTCS 366, 2022, pp. 35-49},
  year = {2022},
  doi = {10.4204/eptcs.366.6},
  eprint = {2208.05721},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.4204/eptcs.366.6}
}

References

40 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Carl Allen & Timothy M. Hospedales (2019): Analogies Explained: Towards Understanding Word Embeddings. CoRR abs/1901.09813, doi:10.48550/arXiv.1901.09813. arXiv:1901.09813. ↗
2. Maya Arad (2003): Locality Constraints on the Interpretation of Roots: The Case of Hebrew Denominal Verbs. Natural Language and Linguistic Theory 21(4), pp. 737–778, doi:10.1023/a:1025533719905. ↗
3. Mark Aronoff (1976): Word Formation in Generative Grammar. Linguistic Inquiry monographs, MIT press. find
4. Sanjeev Arora, Yuanzhi Li, Yingyu Liang, Tengyu Ma & Andrej Risteski (2016): A Latent Variable Model Approach to PMI-based Word Embeddings. Transactions of the Association for Computational Linguistics 4, pp. 385–399, doi:10.1162/tacl_a_00106. Available at https://aclanthology.org/Q16-1028. ↗
5. Outi Bat-El (1994): Stem Modification and Cluster Transfer in Modern Hebrew. Natural Language and Linguistic Theory 12(4), pp. 571–596, doi:10.1007/BF00992928. Available at http://www.jstor.org/ stable/4047868. ↗
6. J.D. Bobaljik (2012): Universals in Comparative Morphology: Suppletion, Superlatives, and the Structure of Words. Current Studies in Linguistics, MIT Press, doi:10.7551/mitpress/9069.001.0001. ↗
7. Piotr Bojanowski, Edouard Grave, Armand Joulin & Tomas Mikolov (2016): Enriching Word Vectors with Subword Information. arXiv preprint arXiv:1607.04606, doi:10.48550/arXiv.1607.04606. Available at https://arxiv.org/abs/1607.04606. ↗
8. Henry Brice (2016): The root and word distinction: an experimental study of Hebrew denominal verbs. Morphology 27(2), pp. 159–177, doi:10.1007/s11525-016-9297-0. ↗
9. Noam Chomsky (1970): Remarks on Nominalization. In R. Jacobs & P. S. Rosenbaum, editors: Reading in English Transformational Grammar, Ginn, Waltham, pp. 184–221. find
10. Noam Chomsky (1973): Conditions on Transformations. In S. R. Anderson & R. P. V. Kiparsky, editors: A Festschrift for Morris Halle, Holt, Rinehart & Winston, New York, pp. 232–286. find
11. Norman Cliff (1993): Dominance statistics: Ordinal analyses to answer ordinal questions. Psychological Bulletin 114(3), pp. 494–509, doi:10.1037/0033-2909.114.3.494. ↗
12. Jacob Devlin, Ming-Wei Chang, Kenton Lee & Kristina Toutanova (2018): BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. CoRR abs/1810.04805, doi:10.48550/arXiv.1810.04805. arXiv:1810.04805. ↗
13. David Embick (2010): Localism versus globalism in morphology and phonology. MIT Press, Cambridge, doi:10.7551/mitpress/9780262014229.001.0001. ↗
14. Kawin Ethayarajh, David Duvenaud & Graeme Hirst (2018): Towards Understanding Linear Word Analogies, doi:10.48550/ARXIV.1810.04882. Available at https://arxiv.org/abs/1810.04882. ↗
15. Ram Frost, Avital Deutsch & Kenneth I. Forster (2000): Decomposing morphologically complex words in a nonlinear morphology. Journal of Experimental Psychology: Learning, Memory, and Cognition 26(3), pp. 751–765, doi:10.1037/0278-7393.26.3.751. ↗
16. Ram Frost, Kenneth I. Forster & Avital Deutsch (1997): What can we learn from the morphology of Hebrew? A masked-priming investigation of morphological representation. Journal of Experimental Psychology: Learning, Memory, and Cognition 23(4), pp. 829–856, doi:10.1037/0278-7393.23.4.829. ↗
17. Ram Frost, Tamar Kugler, Avital Deutsch & Kenneth I. Forster (2005): Orthographic Structure Versus Morphological Structure: Principles of Lexical Organization in a Given Language. Journal of Experimental Psychology: Learning, Memory, and Cognition 31(6), pp. 1293–1326, doi:10.1037/0278-7393.31.6.1293. ↗
18. Alex Gittens, Dimitris Achlioptas & Michael W. Mahoney (2017): Skip-Gram - Zipf + Uniform = Vector Additivity. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Association for Computational Linguistics, Vancouver, Canada, pp. 69–76, doi:10.18653/v1/P17-1007. Available at https://aclanthology.org/P17-1007. ↗
19. Edouard Grave, Piotr Bojanowski, Prakhar Gupta, Armand Joulin & Tomás Mikolov (2018): Learning Word Vectors for 157 Languages. CoRR abs/1802.06893, doi:10.48550/arXiv.1802.06893. arXiv:1802.06893. ↗
20. Louis Guttman (1954): Some necessary conditions for common-factor analysis. Psychometrika 19(2), pp. 149–161, doi:10.1007/bf02289162. ↗
21. Morris Halle & Alec Marantz (1993): Distributed Morphology and the Pieces of Inflection. In: The View from Building 20, MIT Press, Cambridge, MA, pp. 111–176. find
22. Heidi Harley (2014): On the identity of roots. Theoretical Linguistics 40(3-4), pp. 225–276, doi:10.1515/tl-2014-0010. ↗
23. Sándor G. J. Hervey & Jan W. F. Mulder (1973): Pseudo-Composites and Pseudo-Words: Sufficient and Necessary Criteria for Morphological Analysis. La Linguistique 9(1), pp. 41–70. Available at http://www.jstor.org/stable/30248840. ↗
24. D. Jurafsky & J.H. Martin (2000): Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition. Prentice Hall series in artificial intelligence, Pearson Prentice Hall. find
25. Qi Liu, Matt J. Kusner & Phil Blunsom (2020): A Survey on Contextual Embeddings. CoRR abs/2003.07278, doi:10.48550/arXiv.2003.07278. arXiv:2003.07278. ↗
26. Alec Marantz (2000): Roots: the universality of root and pattern morphology. In: conference on Afro-Asiatic languages, University of Paris VII, 3, p. 14. find
27. Tomás Mikolov, Kai Chen, Greg Corrado & Jeffrey Dean (2013): Efficient Estimation of Word Representations in Vector Space. In Yoshua Bengio & Yann LeCun, editors: 1st International Conference on Learning Representations, ICLR 2013, Scottsdale, Arizona, USA, May 2-4, 2013, Workshop Track Proceedings, doi:10.48550/arXiv.1301.3781. Available at http://arxiv.org/abs/1301.3781. ↗
28. Sathvik Nair, Mahesh Srinivasan & Stephan C. Meylan (2020): Contextualized Word Embeddings Encode Aspects of Human-Like Word Sense Knowledge. CoRR abs/2010.13057, doi:10.48550/arXiv.2010.13057. arXiv:2010.13057. ↗
29. James H Neely (2012): Semantic priming effects in visual word recognition: A selective review of current findings and theories. Basic processes in reading, pp. 272–344. find
30. Karl Pearson (1901): LIII. On lines and planes of closest fit to systems of points in space. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science 2(11), pp. 559–572, doi:10.1080/14786440109462720. ↗
31. Jeffrey Pennington, Richard Socher & Christopher Manning (2014): GloVe: Global Vectors for Word Representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Association for Computational Linguistics, Doha, Qatar, pp. 1532–1543, doi:10.3115/v1/D14-1162. Available at https://aclanthology.org/D14-1162. ↗
32. Manuel Perea & Eva Rosa (2002): The effects of associative and semantic priming in the lexical decision task. Psychological Research 66(3), pp. 180–194, doi:10.1007/s00426-002-0086-5. ↗
33. Paul Martin Postal (1969): Anaphoric Islands. In: Chicago Linguistic Society 5, pp. 205–239. find
34. Jean-François Prunet, Renée Béland & Ali Idrissi (2000): The Mental Representation of Semitic Words. Linguistic Inquiry 31(4), pp. 609–648, doi:10.1162/002438900554497. Available at http://www.jstor.org/stable/4179126. ↗
35. Amit Seker, Elron Bandel, Dan Bareket, Idan Brusilovsky, Refael Shaked Greenfeld & Reut Tsarfaty (2021): AlephBERT: A Hebrew Large Pre-Trained Language Model to Start-off your Hebrew NLP Application With. CoRR abs/2104.04052, doi:10.48550/arXiv.2104.04052. arXiv:2104.04052. ↗ our card
36. Yishai Tobin (2004): Hebrew (Semitic). In Geert Booij, Christian Lehmann, Joachim Mugdan, & Stavros Skopeteas, editors: Morphologie/Morphology, de Gruyter, Berlin and New York, pp. 1343–58, doi:10.1515/9783110172782.2.16.1343. ↗
37. Eran Tomer (2012): Automatic Hebrew Text Vocalization. Master’s thesis, Ben Gurion University of the Negev. find
38. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser & Illia Polosukhin (2017): Attention Is All You Need. CoRR abs/1706.03762, doi:10.48550/arXiv.1706.03762. arXiv:1706.03762. ↗
39. Hadas Velan & Ram Frost (2007): Cambridge University versus Hebrew University: The impact of letter transposition on reading English and Hebrew. Psychonomic Bulletin & Review 14(5), pp. 913–918, doi:10.3758/bf03194121. ↗
40. Hadas Velan & Ram Frost (2011): Words with and without internal structure: What determines the nature of orthographic and morphological processing? Cognition 118(2), pp. 141–156, doi:10.1016/j.cognition.2010.11.013. ↗

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/hbo-2022-benbaji-word-embeddings-distinguish-denominal-root/