A Comprehensive NLP System for Modern Standard Arabic and Modern Hebrew

What it is about

The paper describes Morfix, a commercial Hebrew system from Melingo with morphological analysis, context-sensitive lemmatization, vocalization, text-to-phoneme conversion and a syntax-based prosody model, and its extension to Modern Standard Arabic. The authors argue that the two languages share one architecture: the same lemma database, rule databases and a single notion of AGREEMENT, with language-specific rules only where needed. A lemma is the least marked inflectional form, not the root; roots group lemmas into "word families" on top. About 70 million inflected forms are generated from lemma records and rules, with lexicographer exceptions. The ambiguity comes from homographic forms, clitic combinations and unvocalized spelling. The figures are the vendor's own, without a public test set: for Hebrew, above 98% of tokens get the top-scored lemma right, 1–2% of web words stay unanalysed, and the system runs at about 10,000 words per second on an 800 MHz machine.

Ideas

• I1. A lemma policy for the lexicon: lemma = least marked form, a different part of speech means a different lemma, and the root is an optional grouping key. Use it in 8. Lexicon and register; the root grouping could also fill the gap that macula dict has no roots yet.
• I2. Verb-derived nominals are ambiguous with copular clauses (the paper's axi shomer, "my brother is a guard" or "my brother guards"). Build a minimal-pair test for 05. The verb: system, binyanim, tense and use and 05b. Verb paradigms: the strong verb in seven binyanim and the weak roots, where the cue is the presence of an auxiliary or a tense marker in the clause.
• I3. Spelling alternatives (Ktiv Haser against Ktiv Male) as rules that generate variants from the lexical form, not as stored strings. The paper reports that the semi-vocalized system adds about 25% to the token count. The rule table belongs in 2. Script and spelling.
• I4. Report two numbers for any Hebrew lemma stage: top-score lemma accuracy and the rate of unanalysed tokens, both on a held-out corpus. These are the gates we would use.

For Mova

I1 goes into 8. Lexicon and register. I2 is a test for 05. The verb: system, binyanim, tense and use and 05b. Verb paradigms: the strong verb in seven binyanim and the weak roots. I3 goes into 2. Script and spelling. I4 is a measurement rule for 12. Mova and Modern Hebrew: where it stands, what it would need.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.3115/1118637.1118646

@inproceedings{kamir2002comprehensive,
  title = {{A Comprehensive NLP System for Modern Standard Arabic and Modern Hebrew}},
  author = {Kamir, Dror and Soreq, Naama and Neeman, Yoni},
  booktitle = {Proceedings of the ACL-02 Workshop on Computational Approaches to Semitic Languages},
  year = {2002},
  doi = {10.3115/1118637.1118646},
  url = {https://doi.org/10.3115/1118637.1118646}
}

References

10 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Alcalay R. (1990). The Complete Hebrew-English Dictionary, Massada and Yediot Aharonot Pub., Tel-Aviv. find
2. Al-Munjid fi l-Lugha wa-l-I’lam (1992), Dar El-Mashriq Pub., Beirut. find
3. Ayalon D. and P. Shinar (1947). Arabic-Hebrew Dictionary for the Modern Arabic Language, Hebrew University, Jerusalem. find
4. Beesley, K.R. (2001). “Finite-State Morphological Analysis and Generation of Arabic at XeroxRe-search: Status and Plans in 2001”, in Arabic Language Processing: Status and Prospects - 39 th Annual Meeting of the Association for Computational Linguistics, pp. 1-8. find
5. Dichy, J. (2001), “On Lemmatization of the Arabic Entries of Multilingual Lexical Databases”, in Arabic Language Processing: Status and Prospects - 39 th Annual Meeting of the Association for Computational Linguistics, pp. 23-30. find
6. Even-Shoshan, A. (1992). Ha-Milon He-Ħadash, Kiryat Sefer, Jerusalem. find
7. Glinert, L. (1989). The Grammar of Modern Hebrew, Cambridge University Press, Cambridge. find
8. Holes, C. (1995). Modern Arabic: Structures, Functions and Varieties, Longman Linguistics Library, London. find
9. Wehr, H. (1976). A Dictionary of Modern Written Arabic, Ithaca, NY. find
10. Wright, W. (1896). A Grammar of the Arabic Language, Cambridge: University Press. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2002-kamir-comprehensive-nlp-system-modern-standard/