Rapid prototyping of a transfer-based Hebrew-to-English machine translation system

What it is about

A two-month build of a first broad-domain Hebrew-to-English system, about four person-months of work, on top of existing public resources. Hebrew's morphological and orthographic ambiguity is the main difficulty. The system has two parts: a transfer engine that produces a lattice of candidate translation segments, and a decoder that picks the best path with an English language model. The transfer grammar is small and written by hand. On 62 unseen newspaper sentences, the word-to-word version scored BLEU 0.0606, the version with an automatically learned grammar 0.0775, and the hand-written grammar 0.1013 (NIST 3.42, 3.54 and 3.79). The development set was 113 sentences, and the paper calls the results encouraging.

Ideas

• I1. A lattice of chunk choices, scored by the target standard: the encoder's candidate chunks for each meta token form a lattice, and a decoder picks the best path with counts from the target standard in profile.rs. Ties are broken by fixed BTreeMap order so the path is the same on each run.
• I2. Keep the three-row control: for each author pair, report the word-to-word baseline, the learned delta and the full system in measure.rs, so that the value of structure is shown by a difference and not assumed.
• I3. Ambiguity as a lattice of readings: the Hebrew reader in tools/translate/src/hebrew.rs could keep several MACULA analyses per word with counts, rather than one reading. This goes at open problem 7 (the Hebrew reader is the weakest part).

For Mova

This is the closest paper to the Hebrew reader. The lattice with a target count model is a deterministic alternative to picking one chunk per meta token, and the three-row baseline is the control that the translator's measures already need. Its BLEU values are small and from another setting; they are not used as a benchmark here.

Topics: Machine translation

Cite

The paper itself, not this card. BibTeX from the library's record of the work.

@inproceedings{lavie2004rapid,
  title = {{Rapid prototyping of a transfer-based Hebrew-to-English machine translation system}},
  author = {Lavie, Alon and Peterson, Erik and Probst, Katharina and Wintner, Shuly and Eytani, Yaniv},
  booktitle = {Proceedings of the 10th Conference on Theoretical and Methodological Issues in Machine Translation of Natural Languages},
  year = {2004},
  url = {https://aclanthology.org/2004.tmi-1.1/}
}

References

11 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Carbonell, Jaime, Katharina Probst, Erik Peterson, Christian Monson, Alon Lavie, Ralf Brown, and Lori Levin. 2002. Automatic Rule Learning for Resource-Limited MT. In Proceedings of the 5th Biennial Conference of the Association for Machine Translation in the Americas (AMTA-02), October. find
2. Dahan, Hiya. 1997. Hebrew–English English–Hebrew Dictionary. Academon, Jerusalem. find
3. Doddington, George. 2002. Automatic Evaluation of Machine Translation Quality Using N-gram Co-Occurrence Statistics. In Proceedings of the Second Conference on Human Language Technology (HLT-2002). find
4. Efron, Bradley and Robert Tibshirani. 1986. Bootstrap Methods for Standard Errors, Confidence Intervals and Other Measures of Statistical Accuracy. Statistical Science, 1:54–77. find
5. Lavie, Alon, Stephan Vogel, Lori Levin, Erik Peterson, Katharina Probst, Ariadna Font Llitjos, Rachel Reynolds, Jaime Carbonell, and Richard Cohen. 2003. Experiments with a Hindi-to-English Transfer-based MT System under a Miserly Data Scenario. Transactions on Asian Language Information Processing (TALIP), 2(2), June. find
6. Papineni, Kishore, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: a Method for Automatic Evaluation of Machine Translation. In Proceedings of 40th Annual Meeting of the Association for Computational Linguistics (ACL), pages 311–318, Philadelphia, PA, July. find our card
7. Peterson, Erik. 2002. Adapting a Transfer Engine for Rapid Machine Translation Development. Master’s thesis, Georgetown University. find
8. Probst, Katharina, Lori Levin, Erik Peterson, Alon Lavie, and Jaime Carbonell. 2002. MT for Resource-Poor Languages Using Elicitation-Based Learning of Syntactic Transfer Rules. Machine Translation, 17(4). find
9. Segal, Erel. 1999. Hebrew Morphological Analyzer for Hebrew Undotted Texts. Master’s thesis, Technion, Israel Institute of Technology, Haifa, October. In Hebrew. find
10. Wintner, Shuly. 2004. Hebrew Computational Linguistics: Past and Future. Artificial Intelligence Review, 21(2):113–138. find
11. Wintner, Shuly and Shlomo Yona. 2003. Resources for Processing Hebrew. In Proceedings of the MT-Summit IX workshop on Machine Translation for Semitic Languages, New Orleans, September. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/lavie-2004-rapid-prototyping-transfer-based-hebrew/