TnT – A Statistical Part-of-Speech Tagger

Summary

TnT ("Trigrams'n'Tags") shows that a carefully engineered second-order hidden Markov model is as accurate as more complex taggers of its time. Tag trigram probabilities are smoothed by linear interpolation of unigram, bigram and trigram estimates, with the interpolation weights set by deleted interpolation rather than tuned by hand. Unknown words are handled by a suffix model that estimates tag distributions from word endings of rare training words, with separate statistics for capitalised and lowercase words. Decoding is exact Viterbi search, with beam pruning for speed. The tagger is fast, language-independent and reaches state-of-the-art accuracy on English and German treebanks.

How Mova uses it

• en/src/tag.rs — a direct reimplementation: trigram Markov model over tag classes with a dense transition table, deleted-interpolation smoothing, a word-to-tags lexicon for known words, and suffix models (up to 10 characters, separately for upper- and lowercase words, trained on words seen at most 10 times). Exact Viterbi over tag pairs.
• en/src/ptag.rs — the main tagger is now an averaged perceptron with a beam; TnT's lexicon and suffix model still supply prior tag classes for unknown words.
• TnT remains a baseline and a fast mode (e.g. greedy parsing with TnT tags, faster generation).

Effectiveness in Mova

Measured in the development notes of en: TnT alone scores 93,12% (EWT), 93,12% (PUD), 92,84% (GUM) tagging accuracy; the perceptron tagger that uses TnT priors for unknown words reaches 94,35 / 94,04 / 93,82%. Downstream, greedy parsing with TnT tags gives EWT UAS/LAS 82,3/77,3 versus 83,4/79,0 with the perceptron tags. TnT-based generation is about 4 times faster (~10 500 vs ~2 600 sentences/s) at slightly lower quality (54,6% vs 56,6% exact, chrF 92,4 vs 92,8).

👨‍🔬💥

Topics: Part-of-speech tagging

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.3115/974147.974178 · arXiv cs/0003055

@inproceedings{brants2000tnt,
  title = {{TnT – A Statistical Part-of-Speech Tagger}},
  author = {Brants, Thorsten},
  booktitle = {Sixth Applied Natural Language Processing Conference},
  year = {2000},
  doi = {10.3115/974147.974178},
  eprint = {cs/0003055},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.3115/974147.974178}
}

References

14 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Thorsten Brants, Wojciech Skut, and Hans Uszkoreit. 1999. Syntactic annotation of a German newspaper corpus. In Proceedings of the ATALA Treebank Workshop, pages 69-76, Paris, France. find
2. Eric Brill. 1993. A Corpus-Based Approach to Language Learning. Ph.D. Dissertation, Department of Computer and Information Science, University of Pennsylvania. find
3. Eugene Charniak, Curtis Hendrickson, Neil Jacobson, and Mike Perkowitz. 1993. Equations for part-of-speech tagging. In Proceedings of the Eleventh National Con[erence on Artificial Intelligence, pages 784-789, Menlo Park: AAAI Press/MIT Press. find
4. Doug Cutting, Julian Kupiec, Jan Pedersen, and Penelope Sibun. 1992. A practical part-of-speech tagger. In Proceedings of the 3rd Conference on Applied Natural Language Processing (ACL), pages 133-140. find
5. Walter Daelemans, Jakub Zavrel, Peter Berck, and Steven Gillis. 1996. Mbt: A memory-based part of speech tagger-generator. In Proceedings of the Workshop on Very Large Corpora, Copenhagen, Denmark. find
6. Mitchell Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of English: The Penn Treebank. Computational Linguistics, 19(2):313-330. find our card
7. Lawrence R. Rabiner. 1989. A tutorial on Hidden Markov Models and selected applications in speech recognition. In Proceedings o] the IEEE, volume 77(2), pages 257-285. find
8. Adwait Ratnaparkhi. 1996. A maximum entropy model for part-of-speech tagging. In Proceedings o] the Conference on Empirical Methods in Natural Language Processing EMNLP-96, Philadelphia, PA. find
9. Christer Samuelsson. 1993. Morphological tagging based entirely on Bayesian inference. In 9th Nordic Conference on Computational Linguistics NODALIDA-93, Stockholm University, Stockholm, Sweden. find
10. Helmut Schmid. 1995. Improvements in part-of-speech tagging with an application to German. In Helmut Feldweg and Erhard Hinrichts, editors, Lexikon und Text. Niemeyer, Tfibingen. find
11. Wojciech Skut, Brigitte Krenn, Thorsten Brants, and Hans Uszkoreit. 1997. An annotation scheme for free word order languages. In Proceedings of the Fifth Conference on Applied Natural Language Processing ANLP-97, Washington, DC. find
12. Hans van Halteren, Jakub Zavrel, and Walter Daelemans. 1998. Improving data driven wordclass tagging by system combination. In Proceedings of the International Conference on Computational Linguistics COLING-98, pages 491-497, Montreal, Canada. find
13. Martin Volk and Gerold Schneider. 1998. Comparing a statistical and a rule-based tagger for german. In Proceedings of KONVENS-98, pages 125-137, Bonn. find
14. Jakub Zavrel and Walter Daelemans. 1999. Eval-uatie van part-of-speech taggers voor bet corpus gesproken nederlands. CGN technical report, Katholieke Universiteit Brabant, Tilburg. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/brants-2000-tnt-statistical-part-speech-tagger/