High quality machine translation using a machine-learned sentence realization component

What it is about

A full English-to-French and English-to-German machine translation system, in which the target side is the machine-learned Amalgam realiser of the paper above. Transfer patterns are learned from sentence-aligned manuals (about 1.8 million pairs for French, 900,000 for German) and produce semantic graphs in the target language. Amalgam then realises the sentence. The German and French realisers, rated 1 to 4 by about five raters on 545 sentences, score 3.25 and 2.92 when the realiser regenerates the sentences from their own analyses. In a June 2003 blind test, the translation systems scored 2.43 for both pairs, against 2.26 (French) and 2.14 (German) for a commercial system with domain dictionaries. The BLEU scores are 0.382 for French and 0.323 for German. The authors note that about 15% of sentences are lost to analysis errors before realisation, that word order and missing aggregation are the main realiser faults, and that a classifier choosing the next daughter in ordering beat a language-model ordering in French, English and German.

Ideas

• I1. Try the "choose the next daughter" order on the translator's linearisation. At each step, pick the next constituent among those left, scored by counts over the class of the previous constituent, the candidate and the parent, all keyed by enum. Compare exact order match on dev with the Markov chains in en/src/lin.rs, run twice.
• I2. Make reader noise a separate number. The paper's caveat that analysis errors cap the output is the same as Mova's problem 1. Run the encoder on gold meta-worlds for a sample of verses; its residual then shows the encoder's share alone.
• I3. Add a small human check to normal mode. The paper's rating (five raters, 1 to 4, against a reference) is cheap for a sample of 50 verses. Report it apart from the automatic scores and state the rater count.
• I4. Lexicon from aligned pairs with a residual check. The transfer database comes from aligned pairs, as the meta-world lexicon does from Bible pairs. The paper's results support problem 2: keep a concept merge only if it lowers the reader and encoder bits.

For Mova

The relevant code is en/src/lin.rs (ordering), tools/translate/src/meta.rs (lexicon merges) and the report in tools/translate/src/measure.rs. The paper's gains are for commercial-grade manuals; Mova's Bible setting is different, so only the ordering method and the evaluation design should be borrowed.

Topics: Machine translation

Cite

The paper itself, not this card. BibTeX from the library's record of the work.

@inproceedings{smets2003high,
  title = {{High quality machine translation using a machine-learned sentence realization component}},
  author = {Smets, Martine and Gamon, Michael and Pinkham, Jessie and Reutter, Tom and Pettenaro, Martine},
  booktitle = {Proceedings of Machine Translation Summit IX: Papers},
  year = {2003},
  url = {https://aclanthology.org/2003.mtsummit-papers.48/}
}

References

15 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Bangalore S. and Rambow O. (2000) “Exploiting a probabilistic hierarchical model for generation”. In Proceedings of COLING 2000, Saarbrücken, Germany, pp. 42-48. find
2. Chickering D. M. (2002) The WinMine Toolkit. Microsoft Technical Report MSR-TR-2002-103. find
3. Corston-Oliver S., Gamon M., Ringger E. and Moore R. (2002) “An overview of Amalgam: a machine-learned generation module”. In Proceedings of INLG 2002, New York, pp.33-40. find
4. Corston-Oliver S. and Gamon M. (2003) “Combining decision trees and transformation-based learning to correct transferred linguistic representations”. To appear in Proceedings of MT Summit IX. find
5. Gamon M., Ringger E., Corston-Oliver S., Moore R. (2002a) “Machine-learned contexts for linguistic operations in German sentence realization”. In Proceedings of ACL 2002, pp. 25-32. find our card
6. Gamon M., Ringger E. and Corston-Oliver S. (2002b) Amalgam: A machine-learned generation module. Microsoft Research Technical Report MSR-TR-2002-57. find
7. Gamon M., Ringger E., Zhang Z., Moore R. and S. Corston-Oliver (2002c) “Extraposition: A case study in German sentence realization”. In: Proceedings of COLING 2002, pp. 301-307. find
8. Heidorn G. E. (2000) “Intelligent Writing Assistance”. In R. Dale, H. Moisl, and H. Somers (eds.), A Handbook of Natural Language Processing: Techniques and Applications for the Processing of Language as Text, Marcel Dekker, New York. find
9. Langkilde I. (2000) “Forest-Based Statistical Sentence generation”. In Proceedings of NAACL 2000, pp. 170-177. find
10. Langkilde-Geary I. (2002) “An Empirical Verification of Coverage and Correctness for a General-Purpose Sentence Generator”. In Proceedings of INLG 2002, New York, pp.17-24. find
11. Langkilde I. and Knight K. (1998a) “The practical value of n-grams in generation”. In Proceedings of the 9th International Workshop on Natural Language Generation, Niagara-on-the-Lake, Canada pp. 248-255. find
12. Langkilde I. and Knight K. (1998b) “Generation that exploits corpus-based statistical knowledge”. In Proceedings of the 36th ACL and 17th COLING. Montreal, Canada pp. 704-710. find
13. Menezes, A. and Richardson S. 2001. A Best-First Alignment Algorithm for Automatic Extraction of Transfer Mappings from Bilingual Corpora. In Proceedings of the Data-Driven MT workshop, ACL 2001. find
14. Papineni K., Roukos S., Ward T. and Zhu W.-J. (2001). “Bleu: a Method for Automatic Evaluation of Machine Translation”. IBM Research Report RC 22176. find our card
16. Smets M., Gamon M., Corston-Oliver S. and Ringger E. (2003) “The adaptation of a machine-learned sentence realization system to French”, Proceedings of the 10 th conference of the European Chapter of the Association for Computational Linguistics, 2003. Budapest, Hungary pp 323-330. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/smets-2003-high-quality-machine-translation-machine/