A Good Part-of-Speech Tagger in about 200 Lines of Python
Summary
This article is a practical walkthrough of how to build a fast, accurate part-of-speech tagger with very little code. It argues that a greedy, left-to-right averaged perceptron with a well-chosen set of features gets close to state-of-the-art accuracy, while being much simpler and faster than CRFs or beam-search taggers. Each word is tagged once, using features of the current word, its neighbours, its suffixes and prefixes, and the tags already assigned to the previous words. To work around the train/test mismatch for previous-tag features, training uses the model's own predictions rather than gold tags. The article also stresses small engineering choices that matter in practice, such as a tag dictionary for frequent unambiguous words, word normalisation, and weight averaging. The overall message is that a simple, well-understood linear model is often a better engineering choice than a more complex one.
How Mova uses it
en/src/ptag.rs— the Penn Treebank tagger follows this recipe: an averaged perceptron (Collins 2002) with a greedy left-to-right pass, as described in the article.- Features are integers (hashed feature parts) with per-tag weights; neighbouring words, prefixes/suffixes, word shape and a "previous tag + word" conjunction are used, close to the article's feature set.
- Adapted: instead of a separate tag dictionary shortcut, the built-in lexicon's set of possible tags for a word form is itself a feature (a tag mask), so the model learns how far to trust the lexicon.
- Extended beyond the article: for unknown words a TnT-style prior P(tag | suffix, lexicon) from
en/src/tag.rsis added to the perceptron score, and a beam of 8 replaces the purely greedy pass in the final configuration.
Effectiveness in Mova
Measured on UD English EWT test (from en/src/ptag.rs and the en README):
- TnT baseline: 93.12%;
- greedy averaged perceptron (this recipe): 93.74%;
- plus TnT prior for unknown words: 94.13%;
- plus neighbour word shape and beam 8: 94.35% (PUD 94.04%, GUM 93.82%, vs TnT 93.12 / 92.84 on those). The dependency parser trained on these tags improved from LAS 77.27 to 79.00. Two variants did not help: a separate feature per lexicon-mask tag (unknown words dropped from 71% to 65%) and training rare words without word-identity features (±0.1).
👨🔬💥
More ideas (a second reading)
- I1. Greedy against beam in
en/src/ptag.rs: runen tag-evalwith beam 1 and beam 8 on EWT dev and on the two-fold check (en folds), and report accuracy and tokens per second; if the gain is near the 0.1% the post reports, greedy decoding is the cheaper default for long documents. - I2. A tag dictionary shortcut: words seen at least N times in train with one tag in, say, 97% of cases are tagged from the in-code dictionary without scoring; measure the share of tokens covered, the accuracy on those tokens and the speed-up on EWT dev, with N and the threshold chosen on the folds only.
- I3. Number and year classes: map four-digit numbers 1800–2100 to one year class and other digit strings to one digit class in the tagger's features, and measure accuracy on tokens with digits (EWT and ATIS dev), where unknown numbers otherwise fall to the suffix model.
- I4. A title-case frequency feature: count how often each word is capitalised mid-sentence in a large raw collection (book lessons, the school corpus) and use the binned ratio as a feature instead of raw case; metric: PROPN/NNP accuracy and accuracy on the first word of sentences in EWT dev.
Cite
@inproceedings{honnibal2013good,
title = {{A Good Part-of-Speech Tagger in about 200 Lines of Python}},
author = {Honnibal, Matthew},
booktitle = {Blog post (2013-09-18), explosion.ai blog},
year = {2013},
url = {https://explosion.ai/blog/part-of-speech-pos-tagger-in-python}
}This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/honnibal-2013-good-part-of-speech-tagger/