A global model for joint lemmatization and part-of-speech prediction

What it is about

The authors predict lemmas and part-of-speech tags together, because the two depend on each other. Their model combines a tag-set model and a lemmatization model, and adds joint features over a dependency structure of the words, so the decision for one word can use its neighbours. It uses only morphological lexicons and unlabeled data, so the tagger is partly supervised. The joint model is tested on English, Bulgarian, Slovene and Czech. It reduces lemmatization error by up to 19%, tag error by up to 26%, and the error on the full morphological analysis by up to 22.6%, compared with the separate models.

Ideas

• I1. en/src/morph.rs with en/src/ptag.rs: for each form with several lemma candidates (for example "found" as verb or noun), score the candidate lemma with the tag decided by the tagger, and keep the best pair. Measure lemma accuracy on EWT test with and without the tag.
• I2. Use the dependency structure already built by en/src/parse.rs: the lemma of a word gets a feature from its head's tag. Measure the lemma error reduction on EWT test, and the change in tag accuracy, so the two gains are reported separately.
• I3. Check the paper's claim of joint gain on English alone: count the lemma and tag errors of the separate models on EWT test, and report the number of errors that a joint score would fix before building it.

For Mova

The code is en/src/morph.rs, en/src/ptag.rs and en/src/parse.rs, with en morph-eval, en tag-eval and en parse-eval. The chapter is 10. Word classes. The paper's models use lexicons and a log-linear model, which fit Mova's rules. Measure first on EWT test: lemma and tag accuracy with the joint score, twice.

Topics: Part-of-speech tagging

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.3115/1687878.1687947

@inproceedings{toutanova2009global,
  title = {{A global model for joint lemmatization and part-of-speech prediction}},
  author = {Toutanova, Kristina and Cherry, Colin},
  booktitle = {Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP},
  year = {2009},
  doi = {10.3115/1687878.1687947},
  url = {https://doi.org/10.3115/1687878.1687947}
}

References

23 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Meni Adler, Yoav Goldberg, and Michael Elhadad. 2008. Unsupervised lexicon-based resolution of unknown words for full morpholological analysis. In Proceedings of ACL-08: HLT. find
2. Galen Andrew, Trond Grenager, and Christopher Manning. 2004. Verb sense and subcategorization: Using joint inference to improve performance on complementary tasks. In EMNLP. find
3. Erwin Marsi Antal van den Bosch and Abdelhadi Soudi. 2007. Memory-based morphological analysis and part-of-speech tagging of arabic. In Abdelhadi Soudi, Antal van den Bosch, and Gunter Neumann, editors, Arabic Computational Morphology Knowledge-based and Empirical Methods. Springer. find
4. R. H. Baayen, R. Piepenbrock, and L. Gulikers. 1995. The CELEX lexical database. find
5. Antal Van Den Bosch and Walter Daelemans. Memory-based morphological analysis. In Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics. find
6. Alexander Clark. 2002. Memory-based learning of morphology with stochastic transducers. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (ACL), pages 513–520. find
7. Shay B. Cohen and Noah A. Smith. 2007. Joint morphological and syntactic disambiguation. In EMNLP. find
8. Michael Collins. 2000. Discriminative reranking for natural language parsing. In ICML. find our card
9. M. Collins. 2002. Discriminative training methods for hidden markov models: Theory and experiments with perceptron algorithms. In EMNLP. find our card
10. S. Cucerzan and D. Yarowsky. 2000. Language independent minimally supervised induction of lexical probabilities. In Proceedings of ACL 2000. find
11. Markus Dreyer, Jason R. Smith, and Jason Eisner. 2008. Latent-variable modeling of string transductions with finite-state methods. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1080–1089, Honolulu, October. find
12. Tomaž Erjavec and Saǎo Džeroski. 2004. Machine learning of morphosyntactic structure: lemmatizing unknown Slovene words. Applied Artificial Intelligence, 18:17— 41. find
13. Tomaž Erjavec. 2004. Multext-east version 3: Multilingual morphosyntactic specifications, lexicons and corpora. In Proceedings of LREC-04. find
14. Jenny Rose Finkel, Christopher D. Manning, and Andrew Y. Ng. 2006. Solving the problem of cascading errors: Approximate bayesian inference for linguistic annotation pipelines. In EMNLP. find
15. Nizar Habash and Owen Rambow. 2005. Arabic tokenization, part-of-speech tagging and morphological disambiguation in one fell swoop. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics. find our card
16. Sittichai Jiampojamarn, Colin Cherry, and Grzegorz Kondrak. 2008. Joint processing and discriminative training for letter-to-phoneme conversion. In Proceedings of ACL-08: HLT, pages 905–913, Columbus, Ohio, June. find
17. M. Marcus, B. Santorini, and Marcinkiewicz. 1993. Building a large annotated coprus of english: the penn treebank. Computational Linguistics, 19. find
18. Raymond J. Mooney and Mary Elaine Califf. 1995. Induction of first-order decision lists: Results on learning the past tense of english verbs. Journal of Artificial Intelligence Research, 3:1—24. find
19. Eric Sven Ristad and Peter N. Yianilos. 1998. Learning string-edit distance. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(5):522–532. find
20. Sunita Sarawagi and William Cohen. 2004. Semimarkov conditional random fields for information extraction. In ICML. 1999. find
21. Kristina Toutanova and Mark Johnson. 2008. A bayesian LDA-based model for semi-supervised part-of-speech tagging. In nips08. find
22. Richard Wicentowski. 2002. Modeling and Learning Multilingual Inflectional Morphology in a Minimally Supervised Framework. Ph.D. thesis, Johns-Hopkins University. find
23. R. Zens and H. Ney. 2004. Improvements in phrase-based statistical machine translation. In HLT-NAACL, pages 257–264, Boston, USA, May. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2009-toutanova-global-model-joint-lemmatization-part/