Evaluation of Finite State Morphological Analyzers Based on Paradigm Extraction from Wiktionary

What it is about

The paper tests a method that turns inflection tables into general paradigms, which can then be compiled into weighted or unweighted finite-state transducers for analysis and generation. The tables come from the English edition of Wiktionary for 55 languages in 19 language groups and 10 scripts, all tagged with universal morphological tags. The task is lemmatization of unseen word forms, with lemma-plus-POS and lemma-plus-tags as harder tests. Results vary widely: lemmatization ranges from 0% (Basque) to 96.3% (Armenian) and 97.5% (Hindi). Hindi yields 33 paradigms from 767 training tables, with recall above 95% on all three tests. Where the data are limited or less representative, the recall is very low.

Ideas

• I1. en/src/morph.rs: write the English inflection paradigms as a small set of rules, count how many paradigms each rule set produces, and report the number that have only one example, following the paper's Table 2 layout. Use it as a diagnostic for the lexicon, not as a training source.
• I2. Out-of-lexicon test: hold out 10% of EWT test forms that are missing from data/forms.tsv, then measure lemmatization of those forms with the rule-based path against the current dictionary path. The paper's split is 90% train and 10% held-out, and this is the English analogue.
• I3. Report three tests in en morph-eval: lemma, lemma plus POS, and lemma plus tags. The paper shows that these diverge most for data-poor languages, so the English figures should be reported the same way.

For Mova

The code is en/src/morph.rs and en/src/lexgen.rs, with the check in en morph-eval. The chapters are 13. Verbs, 14-irregular-verbs.md and 11-nouns.md. The paper's English-edition tables are Wiktionary-derived (CC BY-SA), so they are for tests and measurements only; never carry them into the build. Measure first: lemma accuracy on held-out English forms outside the dictionary, with two identical runs before any claim.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/w17-4004

@inproceedings{liu2017evaluation,
  title = {{Evaluation of Finite State Morphological Analyzers Based on Paradigm Extraction from Wiktionary}},
  author = {Liu, Ling and Hulden, Mans},
  booktitle = {SCLeM 2017 (EMNLP workshop, Copenhagen)},
  year = {2017},
  doi = {10.18653/v1/w17-4004},
  url = {https://doi.org/10.18653/v1/w17-4004}
}

References

15 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. John Sylak-Glassman, Christo Kirov, Matt Post, Roger Que, and David Yarowsky. 2015. A universal feature schema for rich morphological annotation and fine-grained cross-lingual part-of-speech tagging. In Cerstin Mahlow and Michael Piotrowski, editors, Proceedings of the 4th Workshop on Systems and Fr… find
2. Malin Ahlberg, Markus Forsberg, and Mans Hulden. 2014. Semi-supervised learning of morphological paradigms and lexicons. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, pages 569– 578, Gothenburg, Sweden. Association for Computational L… find our card
3. Huihsin Tseng, Daniel Jurafsky, and Christopher Manning. 2005. Morphological features help POS tagging of unknown words across language varieties. In Proceedings of the fourth SIGHAN workshop on Chinese language processing, pages 32–39. find
4. Malin Ahlberg, Markus Forsberg, and Mans Hulden. 2015. Paradigm classification in supervised learning of morphology. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1024–1029, Denver, Colorado, M… find
5. Eleftherios Avramidis and Philipp Koehn. 2008. Enriching morphologically poor languages for statistical machine translation. In ACL, pages 763–770. find
6. Ryan Cotterell, Christo Kirov, John Sylak-Glassman, David Yarowsky, Jason Eisner, and Mans Hulden. 2016. find
7. The SIGMORPHON 2016 shared task— morphological reinflection. find
8. Kevin Duh and Katrin Kirchhoff. 2004. Automatic learning of language model structure. In Proceedings of the 20th international conference on Computational Linguistics, page 148. Association for Computational Linguistics. find
9. Greg Durrett and John DeNero. 2013. Supervised learning of complete morphological paradigms. In HLT-NAACL, pages 1185–1195. find our card
10. Markus Forsberg and Mans Hulden. 2016. Learning transducer models for morphological analysis from example inflections. find
11. Mans Hulden and Jerid Francom. 2012. Boosting statistical tagger accuracy with simple rule-based grammars. In LREC, pages 2114–2117. find
12. Mans Hulden. 2014. Generalizing inflection tables into paradigms with finite state operations. In Proceedings of the 2014 Joint Meeting of SIGMOR-PHON and SIGFSM, pages 29–36. find
13. Katharina Kann and Hinrich Schütze. 2016. Single-model encoder-decoder with explicit morphological representation for reinflection. arXiv preprint arXiv:1606.00589. find
14. Aleksandr E. Kibrik. 1998. Archi. In Andrew Spencer and Arnold M. Zwicky, editors, The Handbook of Morphology, pages 455–476. Oxford: Blackwell Publishers. find
15. Christo Kirov, John Sylak-Glassman, Roger Que, and David Yarowsky. 2016. Very-large scale parsing and normalization of wiktionary morphological paradigms. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016), pages 3121–3126. find our card

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/lex-2017-liu-fst-paradigm-extraction-wiktionary/