An Unsupervised Morpheme-Based HMM for Hebrew Morphological Disambiguation

What it is about

Adler and Elhadad disambiguate Hebrew morphology with an unsupervised HMM. The only resource is a morphological analyser, with no tagged corpus. Because the morphemes of a word cannot be seen, only the words they form, each morpheme slot in the sentence is a lattice node that records its possible predecessors and successors. They adapt Baum-Welch and Viterbi to this lattice, so segmentation and tagging are learned in one step. The data are about 6M words of newspaper text; the average word has 2.4 analyses, and the full tag set has 1,934 tags against about 200 morpheme tags. The test set is about 30K hand-annotated words. The morpheme model with a context-free initialisation reaches 88.5% for full morphological disambiguation, against 85.75% for the word model and 78.2% for the baseline. For segmentation plus POS it reaches 92.32%. Unknown words (7.5% of the test set) and the noun, verb and adjective confusions of participles account for most errors.

Ideas

• I1. Keep all segmentations of a token as alternatives in a lattice, and let a Viterbi pass choose the path, instead of committing to the segmenter's best answer first. For tools/hbo, this would sit between the segmenter and gparse; the lattice is deterministic and can be checked by hand.
• I2. Initial probabilities from a context-free approximation: similar-word statistics (the paper's Levinger-style prior) lift the word model from 82.01 to 84.08 with a first-order HMM. Measure this prior for the Hebrew lexicon before any model is trained on it.
• I3. The paper's confusion table is a checklist of error classes: proper name against noun (17.9%), noun against verb (15.3%), adjective against noun (5.4%), all from the participle. Write these as test cases in 05. The verb: system, binyanim, tense and use and Nouns, adjectives, pronouns and numerals.
• I4. A fallback for unknown tokens: enumerate all segmentations with affix tags, give the unanalysed base form a single tag UK, and set its tag from the corpus distribution of similar forms. This is a rule to try in tools/hbo for tokens the lexicon misses.

For Mova

I1 goes into tools/hbo, between the segmenter and gparse. I2 is a measured prior for the he lexicon, recorded in 11. Modern Hebrew in Universal Dependencies. I3 is a set of test cases for 05. The verb: system, binyanim, tense and use and Nouns, adjectives, pronouns and numerals. I4 is a fallback rule for tools/hbo. The model is not used; the lattice and the error classes are what carry over.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.3115/1220175.1220259

@inproceedings{adler2006unsupervised,
  title = {{An Unsupervised Morpheme-Based HMM for Hebrew Morphological Disambiguation}},
  author = {Adler, Meni and Elhadad, Michael},
  booktitle = {Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics},
  year = {2006},
  doi = {10.3115/1220175.1220259},
  url = {https://doi.org/10.3115/1220175.1220259}
}

References

23 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Emmanuel Allon. 1995. Unvocalized Hebrew Writing. Ben Gurion University Press. (in Hebrew). find
2. Roy Bar-Haim, Khalil Sima’an, and Yoad Winter. 2005. Choosing an optimal architecture for segmentation and pos-tagging of modern Hebrew. In Proceedings of ACL-05 Workshop on Computational Approaches to Semitic Languages. find our card
3. Leonard E. Baum. 1972. An inequality and associated maximization technique in statistical estimation for probabilistic functions of a Markov process. Inequalities, 3:1–8. find
4. Na’ama Ben-Mordechai. 2005. Named entities recognition in Hebrew. Master’s thesis, Ben Gurion University of the Negev, Beer Sheva, Israel. (in Hebrew). find
5. Eric Brill. 1995. Transformation-based error-driven learning and natural languge processing: A case study in part-of-speech tagging. Computational Linguistics, 21:543–565. find
6. David Carmel and Yoelle S. Maarek. 1999. Morphological disambiguation for Hebrew search systems. In Proceeding of NGITS-99. find
7. Stanley F. Chen. 1996. Building Probabilistic Models for Natural Language. Ph.D. thesis, Harvard University, Cambridge, MA. find
8. Mona Diab, Kadri Hacioglu, and Daniel Jurafsky. 2004. Automatic tagging of Arabic text: From raw text to base phrase chunks. In Proceeding of HLT-NAACL-04. find
9. Kevin Duh and Katrin Kirchhoff. 2005. Pos tagging of dialectal Arabic: A minimally supervised approach. In Proceedings of ACL-05 Workshop on Computational Approaches to Semitic Languages. find
10. Michael Elhadad, Yael Netzer, David Gabay, and Meni Adler. 2005. Hebrew morphological tagging guidelines. Technical report, Ben Gurion University, Dept. of Computer Science. find
11. David Elworthy. 1994. Does Baum-Welch re-estimation help taggers? In Proceeding of ANLP-94. find
12. Nizar Habash and Owen Rambow. 2005. Arabic tokenization, part-of-speech tagging and morphological disambiguation in one fell swoop. In Proceeding of ACL-05. find our card
13. Nadav Har’el and Dan Kenigsberg. 2004. HSpell - the free Hebrew spell checker and morphological analyzer. Israeli Seminar on Computational Linguistics, December 2004. find
14. Daniel Jurafsky and James H. Martin. 2000. Speech and language processing. Prentice-Hall. find
15. Gennady Lembersky. 2001. Named entities recognition; compounds: approaches and recognitions methods. Master’s thesis, Ben Gurion University of the Negev, Beer Sheva, Israel. (in Hebrew). find
16. Moshe Levinger, Uzi Ornan, and Alon Itai. 1995. Learning morpholexical probabilities from an untagged corpus with an application to Hebrew. Computational Linguistics, 21:383–404. find
17. Moshe Levinger. 1992. Morhphological disambiguation in hebrew. Master’s thesis, Technion, Haifa, Israel. (in Hebrew). find
18. Christopher D. Manning and Hinrich Schutze. 1999. Foundation of Statistical Language Processing. MIT Press. find
19. Bernard Merialdo. 1994. Tagging English text with probabilistic model. Computatinal Linguistics, 20:155–171. find
20. Uzi Ornan. 2002. Hebrew in latin script. Lĕšonénu, LXIV:137–151. (in Hebrew). find
21. Erel Segal. 2000. Hebrew morphological analyzer for Hebrew undotted texts. Master’s thesis, Technion, Haifa, Israel. (in Hebrew). find
22. Scott M. Thede and Mary P. Harper. 1999. A second-order hidden Markov model for part-of-speech tagging. In Proceeding of ACL-99. find
23. Shlomo Yona and Shuly Wintner. 2005. A finite-state morphological grammar of Hebrew. In Proceedings of ACL-05 Workshop on Computational Approaches to Semitic Languages. find our card

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2006-adler-unsupervised-morpheme-based-hmm-hebrew/