Mova0.5 pre-release⌕
☰
home / docs / papers / English / Slaapte or Sliep? Extending Neural-Network Simulations of English Past Tense Learning to Dutch and German

Slaapte or Sliep? Extending Neural-Network Simulations of English Past Tense Learning to Dutch and German

What it is about

The paper repeats Kirov and Cotterell's (2018) study of past-tense learning with a BiLSTM encoder-decoder, on English, Dutch and German. The authors build a new dataset of about 6,100 verb forms per language from UniMorph, split by lemma into 80/10/10 so that no lemma appears in two splits. The model reproduces over-regularisation and the micro U-shape of single verbs. On the English irregulars, its test accuracy is 35.3%, against 28.6% in the original paper; the authors say that this gap comes from about 20 irregular verbs. When the model is trained on spelling instead of phonemes, English irregular test accuracy falls from 40.5% to 28.1%, while Dutch and German change little, because their spelling is more transparent.

Ideas

For Mova

The English chapters are 13. Verbs and 14-irregular-verbs.md, with the morphology code in en/src/morph.rs. The paper's findings for English are the irregular-verb accuracy and the spelling effect, both measured on lemma-held-out splits. Dutch and German results are for comparison only. The dataset is built from UniMorph; check its licence before using it in a release.

Cite

The paper itself, not this card. BibTeX from the library's record of the work.

@inproceedings{yang2023slaapte,
  title = {{Slaapte or Sliep? Extending Neural-Network Simulations of English Past Tense Learning to Dutch and German}},
  author = {Yang, Xiulin and Chen, Jingyan and Eerden, Arjan van and Samin, Ahnaf and Bisazza, Arianna},
  booktitle = {Proceedings of the 24th Nordic Conference on Computational Linguistics (NoDaLiDa)},
  year = {2023},
  url = {https://aclanthology.org/2023.nodalida-1.11/}
}

References

31 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Adam Albright and Bruce Hayes. 2003. Rules vs. analogy in english past tenses: A computational/experimental study. Cognition, 90(2):119– 161. find
  2. R Mirjam Ernestus and Harald Baayen. 2004. Analogical effects in regular past tense production in dutch. find
  3. Harald Baayen, Richard Piepenbrock, and H Van Rijn. 1993. The celex lexical database (cd-rom). linguistic data consortium. Philadelphia, PA: University of Pennsylvania. find
  4. Omer Goldman, David Guriel, and Reut Tsarfaty. 2022. (un)solving morphological inflection: Lemma overlap artificially inflates models’ performance. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 864–870, Dublin, Ireland. Association for Computational Linguistics. find
  5. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473. ↗
  6. Emily Goodwin, Koustuv Sinha, and Timothy J. O’Donnell. 2020. Probing linguistic systematicity. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1958–1969, Online. Association for Computational Linguistics. find
  7. Ryan P Blything, Ben Ambridge, and Elena VM Lieven. 2018. Children’s acquisition of the english past-tense: Evidence for a single-route account from novel verb production data. Cognitive Science, 42:621–639. find
  8. A Van den Bosch, Alain Content, W Daelemans, and Béatrice De Gelder. 1994. Analysing orthographic depth of different languages using data-oriented algorithms: Qualico94. In Proceedings of the 2d International Conference on Quantitative Linguistics, pages 26–31. find
  9. Ulrike Hahn and Ramin Charles Nakisa. 2000. German inflection: Single route or dual route? Cognitive Psychology, 41(4):313–360. find
  10. Joan Bybee. 1995. Regular morphology and the lexicon. Language and cognitive processes, 10(5):425– 455. find
  11. Dieuwke Hupkes, Sara Veldhoen, and Willem Zuidema. 2018. Visualisation and’diagnostic classifiers’ reveal how recurrent and recursive neural networks process hierarchical structure. Journal of Artificial Intelligence Research, 61:907–926. find
  12. Joan Bybee and Sandra Thompson. 1997. Three frequency effects in syntax. In Annual Meeting of the Berkeley Linguistics Society, volume 23, pages 378– 388. find
  13. Akos Kádár, Grzegorz Chrupała, and Afra Alishahi. 2017. Representation of linguistic form and function in recurrent neural networks. Computational Linguistics, 43(4):761–780. find
  14. Cotterell, Mans Hulden, and David Yarowsky. 2020. UniMorph 3.0: Universal Morphology. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 3922–3931, Marseille, France. European Language Resources Association. find
  15. Akhilesh Kakolu Ramarao, Yulia Zinova, Kevin Tang, and Ruben van de Vijver. 2022. HeiMorph at SIG-MORPHON 2022 shared task on morphological acquisition trajectories. In Proceedings of the 19th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology, pages 236–239, Seattle, Washington. Association for Computational Linguistics. find
  16. Kate McCurdy, Sharon Goldwater, and Adam Lopez. 2020. Inflecting when there’s no majority: Limitations of encoder-decoder neural networks as cognitive models for German plurals. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1745–1756, Online. Association for Computational Linguistics. find
  17. Katharina Kann and Hinrich Schütze. 2016. Med: The lmu system for the sigmorphon 2016 shared task on morphological reinflection. In Proceedings of the 14th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology, pages 62–70. find
  18. Joe Pater. 2019. Generative linguistics and neural networks at 60: Foundation, friction, and fusion. Language, 95(1):e41–e74. find
  19. Christo Kirov and Ryan Cotterell. 2018. Recurrent neural networks in linguistic theory: Revisiting pinker and prince (1988) and the past tense debate. Transactions of the Association for Computational Linguistics, 6:651–665. find
  20. Tiago Pimentel, Maria Ryskina, Sabrina J. Mielke, Shijie Wu, Eleanor Chodroff, Brian Leonard, Garrett Nicolai, Yustinus Ghanggo Ate, Salam Khalifa, Nizar Habash, Charbel El-Khaissi, Omer Goldman, Michael Gasser, William Lane, Matt Coler, Arturo Oncevay, Jaime Rafael Montoya Samame, Gema Celeste Silva Villegas, Adam Ek, Jean-Philippe Bernardy, Andrey Shcherbakov, Aziyana Bayyr-ool, Karina Sheifer, Sofya Ganieva, Matvey Plugaryov, Elena Klyachko, Ali Salehi, Andrew Krizhanovsky, Natalia Krizhanovsky, Clara Vania, Sardana Ivanova, Aelita Salchak, Christopher Straughn, Zoey Liu, Jonathan North Washington, Duygu Ataman, Witold Kieraś, Marcin Woliński, Totok Suhardijanto, Niklas Stoehr, Zahroh Nuriah, Shyam Ratan, Francis M. Tyers, Edoardo M. Ponti, Grant Aiton, Richard J. Hatcher, Emily Prud’hommeaux, Ritesh Kumar, Mans Hulden, Botond Barta, Dorina Lakatos, Gábor Szolnok, Judit Ács, Mohit Raj, David Yarowsky, Ryan Cotterell, Ben Ambridge, and Ekaterina Vylomova. 2021. SIGMORPHON 2021 shared task on morphological reinflection: Generalization across languages. In Proceedings of the 18th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology, pages 229–259, Online. Association for Computational Linguistics. find
  21. Guillaume Klein, Yoon Kim, Yuntian Deng, Jean Senellart, and Alexander M Rush. 2017. Opennmt: Open-source toolkit for neural machine translation. arXiv preprint arXiv:1701.02810. ↗
  22. Jordan Kodner and Salam Khalifa. 2022. SIGMORPHON–UniMorph 2022 shared task 0: Modeling inflection in language acquisition. In Proceedings of the 19th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology, pages 157–175, Seattle, Washington. Association for Computational Linguistics. find
  23. Stan A Kuczaj II. 1977. The acquisition of regular and irregular past tense forms. Journal of verbal learning and verbal behavior, 16(5):589–600. find
  24. Gary F Marcus, Ursula Brinkmann, Harald Clahsen, Richard Wiese, and Steven Pinker. 1995. German inflection: The exception that proves the rule. Cognitive psychology, 29(3):189–256. find
  25. Gary F Marcus, Steven Pinker, Michael Ullman, Michelle Hollander, T John Rosen, Fei Xu, and Harald Clahsen. 1992. Overregularization in language acquisition. Monographs of the society for research in child development, pages i–178. find
  26. Steven Pinker and Alan Prince. 1988. On language and connectionism: Analysis of a parallel distributed processing model of language acquisition. Cognition, 28(1-2):73–193. find
  27. Xavier Marjou. 2021. OTEANN: Estimating the transparency of orthographies with an artificial neural network. In Proceedings of the Third Workshop on Computational Typology and Multilingual NLP, pages 1–9, Online. Association for Computational Linguistics. find
  28. Steven Pinker and Michael T Ullman. 2002. The past and future of the past tense. Trends in cognitive sciences, 6(11):456–463. find
  29. David C Plaut and Laura M Gonnerman. 2000. Are non-semantic morphological effects incompatible with a distributed connectionist approach to lexical processing? Language and Cognitive Processes, 15(4-5):445–485. find
  30. Arya D. McCarthy, Christo Kirov, Matteo Grella, Amrit Nidhi, Patrick Xia, Kyle Gorman, Ekaterina Vylomova, Sabrina J. Mielke, Garrett Nicolai, Miikka Silfverberg, Timofey Arkhangelskiy, Nataly Krizhanovsky, Andrew Krizhanovsky, Elena Klyachko, Alexey Sorokin, John Mansfield, Valts Ernštreits, Yuval Pinter, Cassandra L. Jacobs, Ryan Kim Plunkett and Patrick Juola. 1999. A connectionist model of english past tense and plural morphology. Cognitive Science, 23(4):463–490. find
  31. A Kim Plunkett and Virginia Marchman. 1991. U-shaped learning and frequency effects in a multi-layered perception: Implications for child language acquisition. Cognition, 38(1):43–102. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/yang-2023-slaapte-or-sliep-extending-neural/