Choosing an Optimal Architecture for Segmentation and POS-Tagging of Modern Hebrew

What it is about

Bar-Haim et al. build an HMM segmenter and POS tagger for Modern Hebrew from a morphological analyser and a very small tagged corpus (the Hebrew Treebank, from Ha'aretz, with under 30K words per training fold), plus about 340K untagged words. They compare two tokenisations: a word-level model, where each tag is a whole segmentation pattern such as IN-H-NN, and a morpheme-level model, where the tags are plain POS and the input is the segments. The morpheme model tags better (90.21 against 89.42 word-level, 5-fold cross-validation) but segments no better. Error analysis shows that the covert definite article h causes many errors: b-bit ("in a house") against b-h-bit ("in the house"). Their best model, M+h, makes the definiteness marker a feature of the stem's tag and reaches 90.51 tagging and 96.74 segmentation accuracy. A lexical model interpolated with an unsupervised estimate (weight 0.85 on the annotated counts) adds 1.5 points to tagging. The analyser misses 3.78% of test words, which caps accuracy at 96.22%.

Ideas

• I1. Definiteness as a stem feature: when b, k or l precedes a noun, decide between the bare and the h-marked reading from lexical counts of definite and indefinite uses. The paper's example lmmflh (25 occurrences, only 2 indefinite) shows the signal. This is a count table for Prepositions, prefixes, conjunctions and particles, with the spelling side in 2. Script and spelling.
• I2. Interpolate the lexical model: relative frequency from the annotated data with weight 0.85, plus an unsupervised estimate from a large untagged corpus. The same interpolation can be measured on the lex table of tools/hbo, on held-out text.
• I3. A minimal-pair test: bbit hgdwl ("in the big house") against bbit gdwl ("in a big house"). The adjective's definiteness decides the segmentation. Use as a test case for Modern Hebrew segmentation in 11. Modern Hebrew in Universal Dependencies.

For Mova

I1 goes into Prepositions, prefixes, conjunctions and particles and 2. Script and spelling. I2 is an idea for the lex table in tools/hbo and for the Modern Hebrew notes in 11. Modern Hebrew in Universal Dependencies, to be measured. I3 is a test case for 11. Modern Hebrew in Universal Dependencies. The paper's tagger is an HMM; Mova would take the tokenisation choice and the lexical smoothing, not the tagger.

Topics: Part-of-speech tagging

Cite

The paper itself, not this card. BibTeX from the library's record of the work.

@inproceedings{barhaim2005choosing,
  title = {{Choosing an Optimal Architecture for Segmentation and POS-Tagging of Modern Hebrew}},
  author = {Bar-Haim, Roy and Sima'an, Khalil and Winter, Yoad},
  booktitle = {Proceedings of the ACL Workshop on Computational Approaches to Semitic Languages},
  year = {2005},
  url = {https://aclanthology.org/W05-0706/}
}

References

14 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Meni Adler. 2001. Hidden Markov Model for Hebrew part-of-speech tagging. Master’s thesis, Ben Gurion University, Israel. In Hebrew. find
2. Leonard Baum. 1972. An inequality and associated maximization technique in statistical estimation for probabilistic functions of a Markov process. In Inequalities III:Proceedings of the Third Symposium on Inequalities, University of California, Los Angeles, pp.1-8. find
3. Eric Brill. 1995. Transformation-based error-driven learning and natural language processing: A case study in part-of-speech tagging. Computational Linguistic, 21:784–789. find
4. David Carmel and Yoelle Maarek. 1999. Morphological disambiguation for Hebrew search systems. In Proceedings of the 4th international workshop,NGITS-99. find
5. Eugene Charniak, Curtis Hendrickson, Neil Jacobson, and Mike Perkowitz. 1993. Equations for part-of-speech tagging. In National Conference on Artificial Intelligence, pages 784–789. find
6. K. W. Church. 1988. A stochastic parts program and noun phrase parser for unrestricted text. In Proc. of the Second Conference on Applied Natural Language Processing, pages 136–143, Austin, TX. find
7. Mona Diab, Kadri Hacioglu, and Daniel Jurafsky. 2004. Automatic tagging of Arabic text: From raw text to base phrase chunks. In HLT-NAACL 2004: Short Papers, pages 149–152. find
8. S.M. Katz. 1987. Estimation of probabilities from sparse data from the language model component of a speech recognizer. IEEE Transactions of Acoustics, Speech and Signal Processing, 35(3):400–401. find
9. Young-Suk Lee, Kishore Papineni, Salim Roukos, Ossama Emam, and Hany Hassan. 2003. Language model based Arabic word segmentation. In ACL, pages 399–406. find
10. M. Levinger, U. Ornan, and A. Itai. 1995. Morphological disambiguation in Hebrew using a priori probabilities. Computational Linguistics, 21:383–404. find
11. Moshe Levinger. 1992. Morphological disambiguation in Hebrew. Master’s thesis, Computer Science Department, Technion, Haifa, Israel. In Hebrew. find
12. Erel Segal. 2000. Hebrew morphological analyzer for Hebrew undotted texts. Master’s thesis, Computer Science Department, Technion, Haifa, Israel. http://www.cs.technion.ac.il/-∼erelsgl/bxi/hmntx/teud.html. ↗
13. K. Sima’an, A. Itai, Y. Winter, A. Altman, and N. Nativ. 2001. Building a tree-bank of Modern Hebrew text. Traitment Automatique des Langues, 42:347–380. find
14. Andreas Stolcke. 2002. SRILM - an extensible language modeling toolkit. In ICSLP, pages 901–904, Denver, Colorado, September. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2005-haim-choosing-optimal-architecture-segmentation-pos/