Style transfer and classification in hebrew news items

What it is about

A short student-style report with two experiments on Hebrew news. The first fine-tunes HeBERT's sentiment head into a classifier that assigns an article title to one of 11 news sections (military, law, health and education, economy, politics, soccer, and others). The data is over 1 GB of articles scraped from Israeli news portals since 2000. The classifier reaches 0.83 accuracy on a 10% validation split after three epochs; the author plots the final-layer representations of titles in 2D with UMAP. The second experiment trains a 12-layer character-level BERT-like generator that gets the news section and the release time as extra vectors concatenated to each character embedding. The generator is judged by loss and perplexity and by a few examples; it overfits after about 20 epochs. The report notes that HeBERT's YAP-based subword tokenizer cannot be used for generation, which is why a character model was trained.

Ideas

I1. Build a rule or bag-of-words baseline for the title-to-section task on the same split, and report it next to the 0.83 figure. Until a baseline is measured, the transformer number does not say what the task needs. The section labels make a small, checkable test set.
I2. Treat the 11 news sections as a register and topic tag set for Hebrew texts. Mova can use it as a label for headlines and short texts, and check tag accuracy on held-out data before any claim.
I3. The generator note is a design lesson: a tokenizer whose output is not a readable left-to-right sequence cannot generate text. Mova's segmenter keeps its output as readable segments, which keeps the trees inspectable.

For Mova

I1 is a measurement for 8. Lexicon and register (headline register) and the plan in 12. Mova and Modern Hebrew: where it stands, what it would need. I2 is a tag set for the same chapter. I3 belongs in the design notes of tools/hbo. The generator itself is not a Mova component: Mova answers from its world, not from generated text.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. arXiv 2212.03019

@misc{weingarten2022style,
  title = {{Style transfer and classification in hebrew news items}},
  author = {Weingarten, Nir},
  year = {2022},
  eprint = {2212.03019},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2212.03019}
}

References

9 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

Tsarfaty R, Seddah D, Goldberg Y, K¨ubler S, Versley Y, Candito M, Foster J, Rehbein I, Tounsi L (2010) Statistical parsing of morphologically rich languages (spmrl) what, how and whither. Proceedings of the NAACL HLT 2010 First Workshop on Statistical Parsing of Morphologically-Rich Languages, 1–12. find
Jacob Devlin. 2018. Multilingual bert readme document Reut Tsarfaty, Amit Seker, Shoval Sadde, Stav Klein. 2019. What is wrong with Hebrew NLP? And how to make it right Amir More and Reut Tsarfaty. 2016. Data-driven morphological analysis and disambiguation for morphologically rich languages and universal dependencies. In Proceedings of COLING, pages 337–348. find
The COLING 2016 Organizing Committee. find
Yue Zhang and Stephen Clark. 2011. Syntactic processing using the generalized perceptron and beam search. Computational Linguistics, 37(1):105–151. find
Avihay Chirqui and Inbal Yahav. 2020. HeBERT & HebEMO: a Hebrew BERT model and a tool for polarity analysis and emotion recognition Yue Zhang and Stephen Clark. 2011. Syntactic processing using the generalized perceptron and beam search. Computational Linguistics, 37(1):105–151. find our card
Yue Zhang and Joakim Nivre. 2011. Transition-based dependency parsing with rich non-local features. In Proceedings of the ACL, HLT ’11, pages 188–193, Stroudsburg, PA, USA. ACL. find
Vaswani, Shazeer, Parmar, Uszkoreit, Jones, Gomez, Kaiser, Polosukhin. 2017. Attention is All you Need. find
Devlin, Chang, Lee, Toutanova. 2018. BERT: Pre-training of Deep Bidirectional transformers for Language Understanding. find
McInnes, L., Healy, Melville J. 2018. UMAP: uniform manifold approximation and projection for dimension reduction find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/langstream-2022-weingarten-style-transfer-classification-hebrew-news/