Apertium-fin-eng–Rule-based Shallow Machine Translation for WMT 2019 Shared Task

What it is about

A rule-based Finnish-English system built for the WMT 2019 shared task, with its dictionary and rules developed during the task. The baseline came from FinnWordNet translations, omorfi for Finnish morphology and apertium-eng, with the dictionary cut from about 260,000 WordNet-style items to about 150,000 usable translations. The paper argues for shared-task data as a test-driven development loop: each change to the lexicon is checked against the task data, and unknown words are collected from the test text, guessed by the tools, and accepted by a human. Lexical selection rules were learned automatically from Europarl, bigram and unigram contexts, which raised the rule count from about 18,000 to about 30,000 for Finnish-English. Grammar work was done by hand, reading the output for systematic errors. A classification of the lexical errors in the submissions is given, and the paper notes that neural systems were more tolerant of noise such as broken words. The system is far behind the best neural entries on BLEU and CharacTER, and the paper says so.

Ideas

I1. Use the dev set as a regression gate for the lexicon. A merge or delta is accepted only if the dev score does not fall, and the log records which failing token each change was for. The translator's residual-driven lexicon (open problem 2) can be checked the same way.
I2. Count the candidates per token at each layer: source ambiguity, lexical-selection ambiguity and target ambiguity, as the paper does in its table, and multiply them. This gives the number of hypotheses the encoder must decide, a measure of where the translator is undecided.
I3. Classify the translator's errors into lexical, out-of-vocabulary, order and agreement classes, and count them per author delta. The existing divergence classes of roundtrip can be extended with the paper's categories so that a fix is attributed to the right layer.
I4. Collect out-of-vocabulary words from each side of an author's text, propose a concept for each, and have a human accept or reject them. For the Hebrew reader, the unknown Strong's numbers are the natural queue.

For Mova

The code is tools/translate (the lexicon, delta and measure code) and tools/roundtrip (the divergence classes). The dev-set gate and the out-of-vocabulary queue are the most useful parts: they match the rules on determinism and on measuring before claiming. The n-gram lexical-selection rules are a count-based model that Mova already has in its encoder, so the paper gives a comparison, not a new component.

Topics: Machine translation

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/w19-5336

@inproceedings{pirinen2019apertium,
  title = {{Apertium-fin-eng–Rule-based Shallow Machine Translation for WMT 2019 Shared Task}},
  author = {Pirinen, Tommi A},
  booktitle = {Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers, Day 1)},
  year = {2019},
  doi = {10.18653/v1/w19-5336},
  url = {https://doi.org/10.18653/v1/w19-5336}
}

References

13 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

Kenneth R Beesley and Lauri Karttunen. 2003. Finite State Morphology. CSLI publications. find
Ondřej Bojar, Christian Federmann, Mark Fishel, Yvette Graham, Barry Haddow, Matthias Huck, Philipp +Koehn, Christof Monz, Mathias Müller, and Matt Post. 2019. Findings of the 2019 conference on machine translation (wmt19). In Proceedings of the Fourth Conference on Machine Translation, Volume 2: Shared Task Papers, Florence, Italy. Association for Computational Linguistics. find
Mikel L Forcada, Mireia Ginestı́-Rosell, Jacob Nordfalk, Jim ORegan, Sergio Ortiz-Rojas, Juan Antonio Pérez-Ortiz, Felipe Sánchez-Martı́nez, Gema Ramı́rez-Sánchez, and Francis M Tyers. 2011. Apertium: a free/open-source platform for rule-based machine translation. Machine translation, 25(2):127–144. find
Arvi Hurskainen and Jörg Tiedemann. 2017. Rule-based machine translation from english to finnish. In Proceedings of the Second Conference on Machine Translation, pages 323–329. 14 http://matrix.statmt.org/matrix/ output/1903?score_id=39757 ↗
Fred Karlsson. 1990. Constraint grammar as a framework for parsing unrestricted text. In Proceedings of the 13th International Conference of Computational Linguistics, volume 3, pages 168–173, Helsinki. find
Philipp Koehn. 2005. Europarl: A parallel corpus for statistical machine translation. In MT summit, volume 5, pages 79–86. find
Prasanth Kolachina and Aarne Ranta. 2015. Gf wide-coverage english-finnish mt system for wmt 2015. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 141–144. find
Krister Lindn, Erik Axelson, Sam Hardwick, Tommi A Pirinen, and Miikka Silfverberg. 2011. Hfstframe-work for compiling and applying morphologies. Systems and Frameworks for Computational Morphology, pages 67–85. find
Joss Moorkens, Antonio Toral, Sheila Castilho, and Andy Way. 2018. Translators perceptions of literary post-editing using statistical and neural machine translation. Translation Spaces, 7(2):240–262. find
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting on association for computational linguistics, pages 311–318. Association for Computational Linguistics. find our card
Tommi A Pirinen. 2015. Development and use of computational morphology of finnish in the open source and open science era: Notes on experiences with omorfi development. SKY Journal of Linguistics, 28. find
Francis M Tyers, Felipe Sánchez-Martı́nez, Mikel L Forcada, et al. 2012. Flexible finite-state lexical selection for rule-based machine translation. find our card
Weiyue Wang, Jan-Thorsten Peter, Hendrik Rosendahl, and Hermann Ney. 2016. Character: Translation edit rate on character level. In Proceedings of the First Conference on Machine Translation: Volume 2, Shared Task Papers, volume 2, pages 505–510. find our card

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/pirinen-2019-apertium-fin-eng-rule-based/