A Neural Model for Part-of-Speech Tagging in Historical Texts

What it is about

Historical texts differ from modern ones in spelling, morphology and syntax, so taggers trained on modern text fail on them. The usual fix is a spelling-normalisation stage that maps old spellings to modern ones before tagging. The paper trains a character-level bidirectional GRU tagger that reads historical text directly. Its training loss adds a term that keeps the representation of an original spelling close to that of its normalised form. Tests cover Swedish verb identification (Gender and Work corpus, F-score 0.8529 on test with the POS weight 0.8, against 0.8601 for the SMT-normalisation pipeline) and German POS tagging on GerManC (early modern German, 1650-1800). On German the neural tagger is almost level with the normalisation pipeline: the difference is five tokens out of 4,845. The error analysis finds that the Swedish problems come from morphology and syntax, not only spelling.

Ideas

I1. en/src/emode.rs and 2. Spelling, print conventions and tokens: measure tagging accuracy on Early Modern English sentences with and without the closed form table applied before en/src/ptag.rs. The paper's result says normalisation is a strong baseline, so this is the comparison to beat.
I2. The paper's Swedish errors include the first-person-plural ending -om, which a spelling rule confuses with other endings. Write the analogous closed list for EModE verb endings (-eth, -est, -st) into 4. Verb endings: -est, -st, -eth, -th and count how often each ending is confused with a noun or an adjective.
I3. The German homily shows that infinitives and first-person plurals are homonymous. Build a count table of English homonym pairs in EModE (for example do as base and as plural) and use the tree context to choose, with the choice logged so each decision can be explained.

For Mova

The chapters are 2. Spelling, print conventions and tokens, 04-verb-endings.md and 12-mova-pipeline.md. The code is en/src/emode.rs (closed table), en/src/ptag.rs (tagger) and tools/gutenberg (texts). The measure: tag accuracy on a held-out set of EModE lessons, with and without the table, run twice. The neural model is not for Mova; the normalisation-versus-direct comparison and the error categories are.

Topics: Part-of-speech tagging

Cite

The paper itself, not this card. BibTeX from the library's record of the work.

@inproceedings{hardmeier2016neural,
  title = {{A Neural Model for Part-of-Speech Tagging in Historical Texts}},
  author = {Hardmeier, Christian},
  booktitle = {Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers},
  year = {2016},
  url = {https://aclanthology.org/C16-1088/}
}

References

15 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

Stanley F. Chen and Joshua Goodman. 1998. An empirical study of smoothing techniques for language modeling. Technical report, Computer Science Group, Harvard University, Cambridge (Mass.). find
Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder–decoder for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1724–1734, Doha (Qatar), October. Association for Computational Linguistics. find
Rosemarie Fiebranz, Erik Lindberg, Jonas Lindström, and Maria Ågren. 2011. Making verbs count: The research project ‘Gender and Work’ and its methodology. Scandinavian Economic History Review, 59(3):273–293. find
Sofia Gustafson-Capková and Britt Hartmann. 2006. Manual of the Stockholm Umeå Corpus version 2.0. https://spraakbanken.gu.se/parole/Docs/SUC2.0-manual.pdf. ↗
Péter Halácsy, András Kornai, and Csaba Oravecz. 2007. HunPos – an open source trigram tagger. In Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics. Companion Volume: Proceedings of the Demo and Poster Sessions, pages 209–212, Prague (Czech Republic), June. Association for Computational Linguistics. find
Philipp Koehn, Hieu Hoang, Alexandra Birch, et al. 2007. Moses: Open source toolkit for statistical machine translation. In Annual Meeting of the Association for Computational Linguistics: Demonstration session, pages 177–180, Prague (Czech Republic). find
Wang Ling, Tiago Luı́s, Luı́s Marujo, Ramón Fernandez Astudillo, Silvio Amir, Chris Dyer, Alan W. Black, and Isabel Trancoso. 2015. Finding function in form: Compositional character models for open vocabulary word representation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1520–1530, Lisbon (Portugal), September. Association for Computational Linguistics. find
Franz Josef Och. 2003. Minimum error rate training in statistical machine translation. In Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics, pages 160–167, Sapporo (Japan). find
Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio. 2013. On the difficulty of training recurrent neural networks. In Proceedings of the 30th International Conference on Machine Learning, pages 1310–1318, Atlanta (Georgia, USA). find
Eva Pettersson and Joakim Nivre. 2011. Automatic verb extraction from historical Swedish texts. In Proceedings of the 5th ACL-HLT Workshop on Language Technology for Cultural Heritage, Social Sciences, and Humanities, pages 87–95, Portland (Oregon, USA), June. Association for Computational Linguistics. find
Eva Pettersson, Beáta Megyesi, and Jörg Tiedemann. 2013. An SMT approach to automatic annotation of historical text. In Proceedings of the Workshop on Computational Historical Linguistics at NODALIDA, pages 54–69, Oslo (Norway). find
Eva Pettersson. 2016. Spelling Normalisation and Linguistic Analysis of Historical Text for Information Extraction, volume 17 of Studia Linguistica Upsaliensia. Acta Universitatis Upsaliensis, Uppsala. find
Silke Scheible, Richard J. Whitt, Martin Durrell, and Paul Bennett. 2011. A gold standard corpus of early modern German. In Proceedings of the 5th Linguistic Annotation Workshop, pages 124–128, Portland (Oregon, USA), June. Association for Computational Linguistics. find our card
Mike Schuster and Kuldip K. Paliwal. 1997. Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing, 45(11):2673–2681. find
Wojciech Skut, Brigitte Krenn, Thorsten Brants, and Hans Uszkoreit. 1997. An annotation scheme for free word order languages. In Proceedings of the Fifth Conference on Applied Natural Language Processing, pages 88–95, Washington (District of Columbia, USA), March. Association for Computational Linguistics. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/hardmeier-2016-neural-model-part-speech-tagging/