Detecting Syntactic Change Using a Neural Part-of-Speech Tagger

What it is about

The paper trains a diachronic LSTM part-of-speech tagger on American English from the 19th, 20th and 21st centuries (COHA), with the year of each text as an input. The learned year embeddings are analysed with PCA. Their first principal component correlates strongly and linearly with time, so the tagger learns a chronological order without being told one. The model can use this to predict the year of composition of new sentences fairly well. A feedforward baseline with the same data shows a weaker time signal. The authors take this as evidence that the change the tagger learns is syntactic and not only lexical. They also note the puzzle of a continuous change in a discrete grammar, and suggest that individual constructions be tested for continuity.

Ideas

• I1. Dating sentences from construction counts: for each sentence, count the marked forms (do-support, -est/-eth forms through morph::archaic, the thee/thou set). Predict the period from those counts, a deterministic and explainable version of the year-prediction test, and measure it on 14. Sample texts, read word by word.
• I2. The paper's baseline design as a control: a word-only model against a model that also sees the tags. The structure-aware version must win, or the claim about syntax is not supported.
• I3. A count-based PCA of the tag distributions per decade, computed from Mova's tagger output. It needs no neural model and shows the same kind of time axis, if the data show one.

For Mova

The chapters are 4. Verb endings: -est, -st, -eth, -th and 12. How Mova reads this English today: pipeline, fixes, failures. The code is en/src/morph.rs (morph::archaic), en/src/emode.rs and en/src/ptag.rs. Measure: the period-prediction accuracy from counts, with shuffled-year labels as the negative control, run twice.

Topics: Part-of-speech tagging

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/w19-4721

@inproceedings{merrill2019detecting,
  title = {{Detecting Syntactic Change Using a Neural Part-of-Speech Tagger}},
  author = {Merrill, William and Stark, Gigi and Frank, Robert},
  booktitle = {Proceedings of the 1st International Workshop on Computational Approaches to Historical Language Change},
  year = {2019},
  doi = {10.18653/v1/w19-4721},
  url = {https://doi.org/10.18653/v1/w19-4721}
}

References

7 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Enoch Olad Aboh. 2015. The Emergence of Hybrid Grammars: Language Contact and Change. Cambridge Approaches to Language Contact. Cambridge University Press. find
2. Mark Davies. 2010-. The corpus of historical American English: 400 million words, 1810-2009. find
3. Haim Dubossarsky, Daphna Weinshall, and Eitan Grossman. 2017. Outta control: Laws of semantic change and inherent biases in word representation models. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1136–1145, Copenhagen, Denmark. Association for Computational Linguistics. find
4. William L. Hamilton, Jure Leskovec, and Dan Jurafsky. 2016. Diachronic word embeddings reveal statistical laws of semantic change. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1489–1501, Berlin, Germany. Association for Computational Linguistics. find
5. Eun Seo Jo, Dai Shen, and Michael Xing. 2017. Backprop to the future: A neural network approach to linguistic change over time. Manuscript, Stanford University. find
6. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. CoRR, abs/1301.3781. ↗
7. Partha Niyogi and Robert C. Berwick. 1995. The logical problem of language change. Technical report. AI Memo 1516, Artificial Intelligence Laboratory, MIT. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/merrill-2019-detecting-syntactic-change-neural-part/