Moshe Koppel

Ideas Mova took (1)

From the ideas ledger: an idea tried, kept or queued, or measured with no gain, and what it gave.

• measured, no gain — Constrain prefix segmentation by a table of legal proclitic sets keyed on the word's initial letters. (DictaBERT: A State-of-the-Art BERT Suite for Modern Hebrew · legal prefix-chain table as a hard filter (chain:N, learned chains): split right dev 97.38 / 94.21 unchanged, cv +-0.01 (pointed, unpointed); the learned…)

Papers

1. Dicta-LM 3.0: Advancing The Frontier of Hebrew Sovereign LLMs (2026)
   Dicta-LM 3.0 is an open-weight family of Hebrew-and-English LLMs at 24B, 12B and 1.7B parameters. Each was made by continued pre-training of an existing base model (Mistral-Small-3.1, Nemotron Nano V2, Qwen3-1.7B) on about 100B Hebrew tokens mixed with 30B English tokens, then post-trained with SFT, DPO and GRPO for…
2. NeoDictaBERT: Pushing the Frontier of BERT models for Hebrew (2025)
   The authors move the NeoBERT encoder design (pre-norm RMSNorm, RoPE, SwiGLU, 28 layers at width 768, 4,096-token context) to Hebrew.
3. MRL Parsing Without Tears: The Case of Hebrew (2024)
   Parsers for morphologically rich languages either run a pipeline (segment, tag, parse), which passes errors forward, or a joint lattice over all segmentations, which is slow and needs a lexicon.
4. MsBERT: A New Model for the Reconstruction of Lacunae in Hebrew Manuscripts (2024)
   The authors pretrain a BERT-base model, MsBERT, on Hebrew manuscript transcriptions. The paper argues that existing Hebrew BERTs, including BEREL trained on printed rabbinic texts, break on manuscript orthography: extra matres lectionis, acronyms written with apostrophes, truncated words, and the proclitic לש. The…
5. Adapting LLMs to Hebrew: Unveiling DictaLM 2.0 with Enhanced Vocabulary and Instruction Capabilities (2024)
   The paper adapts Mistral-7B to Hebrew by continued pre-training on about 200B tokens of half Hebrew and half English.
6. Do Pretrained Contextual Language Models Distinguish between Hebrew Homograph Analyses? (2023)
   Unpointed Hebrew is full of homographs whose analyses differ in segmentation (the prefix or suffix is or is not a word), in morphosyntax, or only in sense.
7. DictaBERT: A State-of-the-Art BERT Suite for Modern Hebrew (2023)
   The paper presents DictaBERT, a BERT-architecture model for Modern Hebrew, pre-trained with masked language modelling only (next-sentence prediction removed) on about 3.8B tokens: 70% cleaned HeDC4 web text and 30% news, blogs, subtitles and novels.
8. Introducing DictaLM -- A Large Generative Language Model for Modern Hebrew (2023)
   The authors release DictaLM, a 7B-parameter generative model for Modern Hebrew, trained on about 7.5 billion tokens.
9. Introducing BEREL: BERT Embeddings for Rabbinic-Encoded Language (2022)
   A short paper releasing a BERT-base model trained on about 220 million words of Rabbinic Hebrew (the Sefaria and Dicta libraries).
10. Large Pre-Trained Models with Extra-Large Vocabularies: A Contrastive Analysis of Hebrew BERT Models and a New One to Outperform Them All (2022)
    The authors train AlephBERTGimmel, a Modern Hebrew BERT with a 128K word-piece vocabulary (earlier models had 2.4K–52K), on the same ~2 billion words as AlephBERT.
11. Nakdan: Professional Hebrew Diacritizer (2020)
    Nakdan adds vowel points to undiacritized Hebrew, for modern, Rabbinic and poetic texts, through a web interface that lets an editor choose among the alternatives.
12. A Novel Challenge Set for Hebrew Morphological Disambiguation and Diacritics Restoration (2020)
    Hebrew homographs are often unbalanced: one analysis is 30 to 129 times more frequent than the other. A tagger can then score well on the majority reading and fail on the minority one, which is the reading that matters in rare cases. The authors build a challenge set of 21 homographs (12 unbalanced, 9 balanced) with…
13. Identification of Parallel Passages Across a Large Hebrew/Aramaic Corpus (2016)
    A fast method for finding all approximate parallel passages (20+ words) in a large Hebrew/Aramaic corpus, where spelling (plene vs defective, final letters, א/י), particles and synonyms vary.
14. Unsupervised Decomposition of a Document into Authorial Components (2011)
    The question is whether the authorial layers of a composite text can be separated without any labelled example.

Co-authors (18)

• Avi Shmidman ×13
• Shaltiel Shmidman ×10
• Reut Tsarfaty ×4
• Cheyn Shmuel Shmidman ×3
• Joshua Guedalia ×3
• Amir DN Cohen ×2
• Dan Bareket ×2
• Amir David Nissan Cohen
• Amit Seker
• Eli Handel
• Ely Porat
• Eylon Gueta
• Hillel Gershuni
• Idan Dershowitz
• Nachum Dershowitz
• Navot Akiva
• Ometz Shmidman
• Yoav Goldberg

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/authors/moshe-koppel/