Building Patient Journeys in Hebrew: A Language Model for Clinical Timeline Extraction

What it is about

The authors continue the pre-training of DictaBERT 2.0 on about 5 million de-identified Hebrew hospital records (about 0.55 billion words). Two ways of extending the vocabulary are tested: a simple merge with a 10,000-token tokenizer (2,011 new tokens) and AdaLM (3,000 new tokens). The model is fine-tuned for pairwise temporal relation classification with four labels (before, after, equal, vague). The two new datasets are Med-TRC (4,262 valid pairs; events are verbs found by a POS tagger) and Onc-TRC (2,377 pairs; events come from about 300 oncology terms). Because "before" dominates, the before class is clipped to the size of the second-largest class. Continued pre-training helps most when training data is small, and the vocabulary variants differ little. De-identification by replacement costs little in performance. The corpus and the two datasets are not public; the model is released for research use only, by formal medical institutions under an IRB protocol.

Ideas

• I1. Take the events from a parse, not from a word list. Med-TRC marks events as verbs and adds an "invalid" label for pairs where the event detection failed, so that event errors are not counted as relation errors. For world, events can be the verb nodes of the UD trees, and a pair with a wrong tree gets the same invalid label instead of a wrong order.
• I2. Add a "vague" relation for pairs where no order can be derived, instead of forcing a guess, and score with a relaxed F1 that does not punish a before or after answer when the gold is vague. The event chain of world could use the same four-way output for the wayyiqtol and narrative cases.
• I3. Measure the compression of the Hebrew segmenter on a corpus: tokens per word, before and after a change of the prefix or lexicon tables. The paper uses this as a cheap check before any downstream test. For tools/hbo, it is a fast regression test for the segmenter.

For Mova

I1 and I2 belong to world (event order) and to its test sets for English and Hebrew; the Hebrew ones follow the plan in 12. Mova and Modern Hebrew: where it stands, what it would need. I1 also needs 11. Modern Hebrew in Universal Dependencies for the verb-node definition. I3 is a test for tools/hbo. Take from the paper only the method. Its datasets and model are not for Mova builds, because the model is restricted to research use.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. arXiv 2512.11502

@misc{hashiloni2025building,
  title = {{Building Patient Journeys in Hebrew: A Language Model for Clinical Timeline Extraction}},
  author = {Hashiloni, Kai Golan and Nokai, Brenda Kasabe and Shevach, Michal and Shemesh, Esthy and Bartin, Ronit and Bergrin, Anna and Harel, Liran and Dershowitz, Nachum and Arad, Liat Nadai and Bar, Kfir},
  year = {2025},
  eprint = {2512.11502},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2512.11502}
}

References

20 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Iz Beltagy, Kyle Lo, and Arman Cohan. SciBERT: A pretrained language model for scientific text. In Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan, editors, Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3615–3620, Hong Kong, China, November 2019. Association for Computational Linguistics. ↗
2. Ilias Chalkidis, Manos Fergadiotis, Prodromos Malakasiotis, Nikolaos Aletras, and Ion Androutsopoulos. LEGAL-BERT: The Muppets straight out of law school, 2020. find
3. Zeming Chen, Alejandro Hernández Cano, Angelika Romanou, Antoine Bonnet, Kyle Matoba, Francesco Salvi, Matteo Pagliardini, Simin Fan, Andreas Köpf, Amirkeivan Mohtashami, Alexandre Sallinen, Alireza Sakhaeirad, Vinitra Swamy, Igor Krawczuk, Deniz Bayazit, Axel Marmet, Syrielle Montariol, Mary-Anne Hartley, Martin Jaggi, and Antoine Bosselut. Meditron-70b: Scaling medical pretraining for large language models, 2023. ↗
4. Avihay Chriqui and Inbal Yahav. HeBERT and HebEMO: A Hebrew model and a tool for polarity analysis and emotion recognition. INFORMS Journal on Data Science, 1(1), 2022. find
5. Zoltan Csaki, Pian Pawakapan, Urmish Thakker, and Qiantong Xu. Efficiently adapting pretrained language models to new languages, 2023. ↗
6. Yiming Cui, Ziqing Yang, and Xin Yao. Efficient and effective text encoding for Chinese LLaMA and Alpaca, 2024. ↗
7. Alon Eirew, Kfir Bar, and Ido Dagan. Beyond pairwise: Global zero-shot temporal graph generation, 2025. find
8. Aryo Gema, Pasquale Minervini, Luke Daines, Tom Hope, and Beatrice Alex. Parameter-efficient fine-tuning of LLaMA for the clinical domain. In Tristan Naumann, Asma Ben Abacha, Steven Bethard, Kirk Roberts, and Danielle Bitterman, editors, Proceedings of the 6th Clinical Natural Language Processing Workshop, pages 91–104, Mexico City, Mexico, June 2024. Association for Computational Linguistics. find
9. Omer Goldman, Avi Caciularu, Matan Eyal, Kris Cao, Idan Szpektor, and Reut Tsarfaty. Unpacking tokenization: Evaluating text compression and its correlation with model performance, 2024. find
10. Eylon Gueta, Avi Shmidman, Shaltiel Shmidman, Cheyn Shmuel Shmidman, Joshua Guedalia, Moshe Koppel, Dan Bareket, Amit Seker, and Reut Tsarfaty. Large pre-trained models with extra-large vocabularies: A contrastive analysis of Hebrew models and a new one to outperform them all, 2023. find
11. Xu Guo and Han Yu. On the domain adaptation and generalization of pretrained language models: A survey, 2022. ↗
12. Rujun Han, I-Hung Hsu, Mu Yang, Aram Galstyan, Ralph Weischedel, and Nanyun Peng. Deep structured neural network for event temporal relation extraction. In Mohit Bansal and Aline Villavicencio, editors, Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL), pages 666– 106, Hong Kong, China, November 2019. Association for Computational Linguistics. [Han et al., 2019b] Rujun Han, Qiang Ning, and Nanyun Peng. Joint event and temporal relation extraction with shared representations and structured prediction. In Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan, editors, Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 434–444, Hong Kong, China, November 2019. Association for Computational Linguistics. [Han et al., 2021] Rujun Han, Xiang Ren, and Nanyun Peng. Econet: Effective continual pretraining of language models for event temporal reasoning, 2021. [Hofmann et al., 2022] Valentin Hofmann, Hinrich Schuetze, and Janet Pierrehumbert. An embarrassingly simple method to mitigate undesirable properties of pretrained language model tokenizers. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio, editors, Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 385–393, Dublin, Ireland, May 2022. Association for Computational Linguistics. [Hong et al., 2021] Jimin Hong, TaeHee Kim, Hyesu Lim, and Jaegul Choo. AVocaDo: Strategy for adapting vocabulary to downstream domain. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih, editors, Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 4692–4700, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. [Mathur et al., 2021] Puneet Mathur, Rajiv Jain, Franck Dernoncourt, Vlad Morariu, Quan Hung Tran, and Dinesh Manocha. TIMERS: Document-level temporal relation extraction. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli, editors, Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 524–533, Online, August 2021. Association for Computational Linguistics. [Nag et al., 2023] Arijit Nag, Bidisha Samanta, Animesh Mukherjee, Niloy Ganguly, and Soumen Chakrabarti. Entropy-guided vocabulary augmentation of multilingual language models for low-resource tasks. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki, editors, Findings of the Association for Computational Linguistics: ACL 2023, pages 8619–8629, Toronto, Canada, July 2023. Association for Computational Linguistics. find
13. Qiang Ning, Hao Wu, and Dan Roth. A multi-axis annotation scheme for event temporal relations. In Iryna Gurevych and Yusuke Miyao, editors, Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1318–1328, Melbourne, Australia, July 2018. Association for Computational Linguistics. [Ning et al., 2019] Qiang Ning, Sanjay Subramanian, and Dan Roth. An improved neural baseline for temporal relation extraction. In Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan, editors, Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 6203–6209, Hong Kong, China, November 2019. Association for Computational Linguistics. [Rasmy et al., 2020] Laila Rasmy, Yang Xiang, Ziqian Xie, Cui Tao, and Degui Zhi. Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease prediction, 2020. [Roccabruna et al., 2024] Gabriel Roccabruna, Massimo Rizzoli, and Giuseppe Riccardi. Will LLMs replace the encoder-only models in temporal relation classification? In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 20402–20415, Miami, Florida, USA, November 2024. Association for Computational Linguistics. [Schmidt et al., 2024] Craig W. Schmidt, Varshini Reddy, Haoran Zhang, Alec Alameddine, Omri Uzan, Yuval Pinter, and Chris Tanner. Tokenization is more than compression, 2024. [Seker et al., 2022] Amit Seker, Elron Bandel, Dan Bareket, Idan Brusilovsky, Refael Greenfeld, and Reut Tsarfaty. AlephBERT: Language model pre-training and evaluation from sub-word to sentence level. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio, editors, Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 46–56, Dublin, Ireland, May 2022. Association for Computational Linguistics. [Shmidman et al., 2024] Shaltiel Shmidman, Avi Shmidman, Amir D. N. Cohen, and Moshe Koppel. Adapting s to Hebrew: Unveiling DictaLM 2.0 with enhanced vocabulary and instruction capabilities, 2024. [Tai et al., 2020] Wen Tai, H. T. Kung, Xin Dong, Marcus Comiter, and Chang-Fu Kuo. exBERT: Extending pre-trained models with domain-specific vocabulary under constrained training resources. In Trevor Cohn, Yulan He, and Yang Liu, editors, Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1433– 1439, Online, November 2020. Association for Computational Linguistics. [Touvron et al., 2023] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. LLaMA: Open and efficient foundation language models, 2023. find our card
14. Haoyu Wang, Hongming Zhang, Yuqian Deng, Jacob Gardner, Dan Roth, and Muhao Chen. Extracting or guessing? improving faithfulness of event temporal relation extraction. In Andreas Vlachos and Isabelle Augenstein, editors, Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 541–553, Dubrovnik, Croatia, May 2023. Association for Computational Linguistics. find
15. Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Jeff Klingner, Apurva Shah, Melvin Johnson, Xiaobing Liu, Łukasz Kaiser, Stephan Gouws, Yoshikiyo Kato, Taku Kudo, Hideto Kazawa, Keith Stevens, George Kurian, Nishant Patil, Wei Wang, Cliff Young, Jason Smith, Jason Riesa, Alex Rudnick, Oriol Vinyals, Greg Corrado, Macduff Hughes, and Jeffrey Dean. Google’s neural machine translation system: Bridging the gap between human and machine translation, 2016. ↗
16. Guy Yanko, Shahaf Pariente, and Kfir Bar. Temporal relation classification in Hebrew. In Jong C. Park, Yuki Arase, Baotian Hu, Wei Lu, Derry Wijaya, Ayu Purwarianti, and Adila Alfa Krisnadhi, editors, Findings of the Association for Computational Linguistics IJCNLP-AACL 2023, pages 261–267, Nusa Dua, Bali, November 2023. Association for Computational Linguistics. find our card
17. Yunzhi Yao, Shaohan Huang, Wenhui Wang, Li Dong, and Furu Wei. Adapt-and-distill: Developing small, fast and effective pretrained language models for domains, 2021. find
18. Shuaicheng Zhang, Qiang Ning, and Lifu Huang. Extracting temporal event relation with syntax-guided graph transformer. In Marine Carpuat, Marie-Catherine de Marneffe, and Ivan Vladimir Meza Ruiz, editors, Findings of the Association for Computational Linguistics: NAACL 2022, pages 379–390, Seattle, United States, July 2022. Association for Computational Linguistics. find
19. Yichao Zhou, Yu Yan, Rujun Han, J. Harry Caufield, Kai-Wei Chang, Yizhou Sun, Peipei Ping, and Wei Wang. Clinical temporal relation extraction with probabilistic soft logic regularization and global inference, 2020. find
20. Jie Zhou, Shenpo Dong, Hongkui Tu, Xiaodong Wang, and Yong Dou. RSGT: Relational structure guided temporal relation extraction. In Nicoletta Calzolari, Chu-Ren Huang, Hansaem Kim, James Pustejovsky, Leo Wanner, Key-Sun Choi, Pum-Mo Ryu, Hsin-Hsi Chen, Lucia Donatelli, Heng Ji, Sadao Kurohashi, Patrizia Paggio, Nianwen Xue, Seokhwan Kim, Younggyun Hahm, Zhong He, Tony Kyungil Lee, Enrico Santus, Francis Bond, and Seung-Hoon Na, editors, Proceedings of the 29th International Conference on Computational Linguistics, pages 2001–2010, Gyeongju, Republic of Korea, October 2022. International Committee on Computational Linguistics. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/langstream-2025-hashiloni-building-patient-journeys-hebrew-language/