Becoming Linguistically Mature: Modeling English and German Children's Writing Development Across School Grades

What it is about

The paper classifies the school grade of children's essays from their "complexity contours". A sliding window of sentences is moved through each text, and a complexity measure (57 for English, 13 for German) is computed per window, giving a series instead of one average. Recurrent (GRU) classifiers read these series. The English data are the GiG corpus (2,898 texts from 983 children, grades 2 to 11) and the German data are FD-LEX (5,628 texts, grades 5 and 9). On English, the contour model reaches 82% accuracy against 76% for the model that sees only text averages (baseline 28%); on German, 74% against 70% (baseline 51%). Sensitivity-based pruning ranks the features; the top 14 English measures are lexical sophistication, word length and register n-grams.

Ideas

• I1. A deterministic contour in Rust: for each text in a school set, compute per-sentence measures from en parses (mean length of sentence and clause, share of rare words) over a sliding window, then summarise each contour by maximum, trough and slope. Classify grade with a decision list. Measure against the average-only baseline; the shuffled-sentence control must drop.
• I2. Feature audit: the paper's pruning idea (remove one feature, retrain, rank) is a clean way to explain which measure drives a grade decision. Implement it with BTreeMap order and report the ranking in world as a table.
• I3. Lexical sophistication for 19. The lexicon: vocabulary, multiword expressions and register: build the word-level list from open corpora only (not from restricted lists) and count, per window, the share of words above a frequency cut.

For Mova

The English chapters are 19. The lexicon: vocabulary, multiword expressions and register and 37-register-and-variation.md. The code would be an en contour module feeding world (school courses). The paper's GiG and FD-LEX data are restricted to research use, so the measure must use Mova's own school texts or open text, and the paper's numbers serve only as a comparison point.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.bea-1.6

@inproceedings{kerz2020becoming,
  title = {{Becoming Linguistically Mature: Modeling English and German Children's Writing Development Across School Grades}},
  author = {Kerz, Elma and Qiao, Yu and Wiechmann, Daniel and Ströbel, Marcus},
  booktitle = {Proceedings of the Fifteenth Workshop on Innovative Use of NLP for Building Educational Applications},
  year = {2020},
  doi = {10.18653/v1/2020.bea-1.6},
  url = {https://doi.org/10.18653/v1/2020.bea-1.6}
}

References

35 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Karin Berendes, Sowmya Vajjala, Detmar Meurers, Doreen Bryant, Wolfgang Wagner, Maria Chinkina, and Ulrich Trautwein. 2018. Reading demands in secondary school: Does the linguistic complexity of textbooks increase with grade level and the academic orientation of the school track? Journal of Educational Psychology, 110(4):518. find
2. Katharina Ehret and Benedikt Szmrecsanyi. 2019. Compressing learner language: An information-theoretic measure of complexity in sla production data. Second Language Research, 35(1):23–45. find
3. Nick C Ellis. 2019. Essentials of a theory of language cognition. The Modern Language Journal, 103:39– 60. find
4. Ruth A Berman. 2007. Developing linguistic knowledge and language use across adolescence. find
5. Xiaobin Chen and Detmar Meurers. 2018. Word frequency and readability: Predicting the text-level readability with a lexical-level attribute. Journal of Research in Reading, 41(3):486–510. find
6. Thomas François and Cédrick Fairon. 2012. An ai readability formula for french as a foreign language. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 466–477. Association for Computational Linguistics. find
7. Jianlin Cheng, Zheng Wang, and Gianluca Pollastri. 2008. A neural network approach to ordinal regression. In 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence), pages 1279–1284. IEEE. find
8. John A Hawkins and Luna Filipović. 2012. Criterial features in L2 English: Specifying the reference levels of the Common European Framework, volume 1. Cambridge University Press. find
9. KyungHyun Cho, Bart van Merrienboer, Dzmitry Bahdanau, and Yoshua Bengio. 2014. On the properties of neural machine translation: Encoder-decoder approaches. CoRR, abs/1409.1259. ↗
10. Dan Klein and Christopher D Manning. 2003. Accurate unlexicalized parsing. In Proceedings of the 41st Annual Meeting on Association for Computational Linguistics-Volume 1, pages 423–430. Association for Computational Linguistics. find our card
11. European Commission. 2019. Proposal for a council recommendation on a comprehensive approach to the teaching and learning of languages. find
12. Pat Langley. 1994. Selection of relevant features in machine learning. In Proceedings of the AAAI Fall symposium on relevance, pages 1–5. find
13. Koby Crammer and Yoram Singer. 2002. Pranking with ranking. In Advances in neural information processing systems, pages 641–647. find
14. Francisco Lorenzo and Oliver Meyer. 2017. Special issue: Languages of schooling: explorations into disciplinary literacies. find
15. Scott A Crossley, Rod Roscoe, and Danielle S McNamara. 2014. What is successful writing? an investigation into the multiple ways writers can write successful essays. Written Communication, 31(2):184– 214. find
16. Xiaofei Lu. 2011. A corpus-based evaluation of syntactic complexity measures as indices of college-level esl writers’ language development. Tesol Quarterly, 45(1):36–62. find
17. Scott A Crossley, Jennifer L Weston, Susan T McLain Sullivan, and Danielle S McNamara. 2011. The development of writing proficiency as a function of grade level: A linguistic analysis. Written Communication, 28(3):282–311. find
18. Xiaofei Lu. 2012. The relationship of lexical richness to the quality of esl learners’ oral narratives. The Modern Language Journal, 96(2):190–208. find
19. Xiaofei Lu. 2017. Automated measurement of syntactic complexity in corpus-based l2 writing research and implications for writing assessment. Language Testing, 34(4):493–511. find
20. Marcus Ströbel, Elma Kerz, and Daniel Wiechmann. 2020. The relationship between first and second language writing: Investigating the effects of first language complexity on second language complexity in advanced stages of learning. Language Learning, n/a(n/a). find
21. Christopher Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven Bethard, and David McClosky. 2014. The stanford corenlp natural language processing toolkit. In Proceedings of 52nd annual meeting of the association for computational linguistics: system demonstrations, pages 55–60. find
22. Joachim Utans and John Moody. 1991. Selecting neural network architectures via the prediction risk: Application to corporate bond rating prediction. In Proceedings First International Conference on Artificial Intelligence Applications on Wall Street, pages 35– 41. IEEE. find
23. Dominic Masters and Carlo Luschi. 2018. Revisiting small batch training for deep neural networks. arXiv preprint arXiv:1804.07612. ↗
24. Zarah Weiss and Detmar Meurers. 2019. Analyzing linguistic complexity and accuracy in academic language development of german across elementary and secondary school. In Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 380–393. find
25. Detmar Meurers. 2020. Natural language processing and language learning. In Carol A. Chapelle, editor, The Concise Encyclopedia of Applied Linguistics. Wiley. find
26. Menglin Xia, Ekaterina Kochmar, and Ted Briscoe. 2019. Text readability assessment for second language learners. arXiv preprint arXiv:1906.07580. ↗
27. John Moody. 1994. Prediction risk and architecture selection for neural networks. In From statistics to neural networks, pages 147–165. Springer. find
28. Ildikó Pilán, David Alfter, and Elena Volodina. 2016. Coursebook texts as a helping hand for classifying linguistic complexity in language learners’ writings. In Proceedings of the Workshop on Computational Linguistics for Linguistic Complexity (CL4LC), pages 120–126. find
29. Pytorch. 2019. Pytorch: Tensors and dynamic neural networks in python with strong gpu acceleration. https://github.com/pytorch/pytorch . ↗
30. Anna N Rafferty and Christopher D Manning. 2008. Parsing three german treebanks: Lexicalized and unlexicalized baselines. In Proceedings of the Workshop on Parsing German, pages 40–46. find
31. Patrick Emanuel Rebuschat, Meurers Detmar, and Tony McEnery. 2017. Language learning research at the intersection of experimental, computational and corpus-based approaches. Language Learning, 67(S1):6–13. find
32. Catherine E Snow. 2010. Academic language and the challenge of reading for learning about science. science, 328(5977):450–452. find
33. Catherine E Snow and Paola Uccelli. 2009. The challenge of academic language. The Cambridge handbook of literacy, pages 112–133. find
34. Marcus Ströbel. 2014. Tracking complexity of l2 academic texts: A sliding-window approach. Master thesis. RWTH Aachen University. find
35. Marcus Ströbel, Elma Kerz, Daniel Wiechmann, and Yu Qiao. 2018. Text genre classification based on linguistic complexity contours using a recurrent neural network. In MRC@ IJCAI, pages 56–63. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/kerz-2020-becoming-linguistically-mature-modeling-english/