Elron Bandel

2 paper cards in Mova's library · 2021–2022

Elron Bandel wrote 2 of the papers Mova reads, in 2021–2022, most of them on Named entity recognition; often with Amit Seker, Dan Bareket, Idan Brusilovsky.

The profile is the library's match of the name; two people with one name are not always told apart.

Ideas Mova took (1)

From the ideas ledger: an idea tried, kept or queued, or measured with no gain, and what it gave.

kept — Turn the morpheme head into a deterministic check: a segmenter's output for a token must concatenate back to the surface string. The paper's character-with-boundary view makes this a free invariant to test on every token. (AlephBERT: Language Model Pre-training and Evaluation from… · consonant check on every candidate (Model::spells): 0 failures on dev, test and the two-fold cv of train; kept as a gate over cut(); commit 5c24bc409)

AlephBERT: Language Model Pre-training and Evaluation from Sub-Word to Sentence Level (2022) 1 idea kept
Seker et al. pre-train AlephBERT, a BERT-style model for Modern Hebrew, on about 1.9 billion words: the deduplicated Hebrew part of OSCAR, all of Hebrew Wikipedia, and Hebrew tweets, with a 52K wordpiece vocabulary. The base model trained for 10 epochs and took 8 days on eight V100 GPUs. The paper's main point is…
AlephBERT: A Hebrew Large Pre-Trained Language Model to Start-off your Hebrew NLP Application With (2021)
The authors build AlephBERT, a BERT-base style language model for Modern Hebrew, pre-trained on about 1.9 billion words: deduplicated OSCAR, a large set of Hebrew tweets (72% of sentences), and Hebrew Wikipedia.