Mova0.5 pre-release⌕
☰
home / docs / papers / Ancient Hebrew / MiqraBERT: Regression-Based Sentence-BERT Finetuning for Biblical Hebrew Parallel Detection

MiqraBERT: Regression-Based Sentence-BERT Finetuning for Biblical Hebrew Parallel Detection

What it is about

A follow-up to the author's 2025 benchmark: AlephBERT (Modern Hebrew) is finetuned as a Sentence-BERT model with cosine-similarity regression on 1,650 verse and half-verse pairs, 825 true parallels (Chronicles synoptic material and poetic parallels from classic studies, the T'OMIM set) and 825 random negatives, over ten seeds. The paper explains why pretrained encoders fail: every Biblical Hebrew verse lands in one narrow region of the space, so unrelated verses get high cosine (E5 rates 2 Sam 5:1 and Job 38:4 at 0.87). After finetuning, the separation between parallel and non-parallel scores grows 2.7-fold and the overlap region shrinks from about 24% to 6%; narrative parallels reach recall@10 of 87.1%, but poetic parallels stay below 9%. The related-work review is useful in itself: Lobbezoo (2015) found string-distance features on lexemes reach AUC 0.937 on cross-reference pairs, while 162 ETCBC grammatical features together reach only 0.646.

Ideas

For Mova

I1 is the same tool as in the 2025 card (hbo-2025-smiley-intertextual-parallel-detection-biblical-hebrew I1). I2 belongs in 2. Cantillation: the accents (te'amim) and 08-syntax.md (poetry: the verse's dichotomy and parallel cola) and uses macula dict glosses; the atnaḥ gives the split point. I3 is a measurement habit for chapter 13. The model itself is not usable at test time (a neural encoder), only as a teacher or a comparison.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. arXiv 2606.19638

@misc{smiley2026miqrabert,
  title = {{MiqraBERT: Regression-Based Sentence-BERT Finetuning for Biblical Hebrew Parallel Detection}},
  author = {Smiley, David M.},
  year = {2026},
  eprint = {2606.19638},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2606.19638}
}

References

74 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Alter, Robert. The Art of Biblical Poetry. 2nd ed. New York: Basic Books, 2011. find
  2. Berlin, Adele. The Dynamics of Biblical Parallelism. Revised and Expanded Edition. Grand Rapids: Eerdmans, 2008. find
  3. Biś, Daniel, Maksim Podkorytov, and Xiuwen Liu. ”Too Much in Common: Shifting of Embeddings in Transformer Language Models and its Implications.” Pages 5117–30 in Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Online: Association for Computational Linguistics, 2021. find
  4. Bothwell, Stephen, Justin DeBenedetto, Theresa Crnkovich, Hildegund Muller, and David Chiang. ”Introducing Rhetorical Parallelism Detection: A New Task with Datasets, Metrics, and Baselines.” Pages 5007–39 in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. Singapore: Association for Computational Linguistics, 2023. find
  5. Brill, Oran, Moshe Koppel, and Avi Shmidman. ”FAST: Fast and Accurate Synoptic Texts.” Digital Scholarship in the Humanities 35/2 (2020): 254–264. find
  6. Burns, Patrick J., James A. Brofos, Kyle Li, Pramit Chaudhuri, and Joseph P. Dexter. ”Profiling of Intertextuality in Latin Literature Using Word Embeddings.” Pages 4900–4907 in Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, 2021. find
  7. Dodge, Jesse, Gabriel Ilharco, Roy Schwartz, Ali Farhadi, Hannaneh Hajishirzi, and Noah Smith. ”Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping.” arXiv:2002.06305, 2020. https://doi.org/10.48550/arXiv.2002.06305 ↗
  8. Doron, Edit. ”The Biblical Sources of Modern Hebrew Syntax.” Pages 222–56 in Language Contact, Continuity and Change in the Genesis of Modern Hebrew. Amsterdam: John Benjamins Publishing Company, 2019. find
  9. Efron, Bradley, and Robert J. Tibshirani. An Introduction to the Bootstrap. Monographs on Statistics and Applied Probability 57. New York: Chapman & Hall, 1993. find
  10. Endres, John Carol, Corrine L. Patton, William R. Millar, John Barclay Burns, Corrine L. Carvalho, Pauline A. Viviano, and Jim Fitzgerald, eds. Chronicles and Its Synoptic Parallels in Samuel, Kings, and Related Biblical Texts. Collegeville, MN: Liturgical Press, 1998. find
  11. Ethayarajh, Kawin. ”How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings.” Pages 55–65 in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Hong Kong, China: Association for Computational Linguistics, 2019. find
  12. Evans, Robert. ”Shitposting, Inspirational Terrorism, and the Christchurch Mosque Massacre.” Bellingcat, March 15, 2019. Accessed December 8, 2025. https://www.bellingcat.com/news/rest-of-w orld/2019/03/15/shitposting-inspirational-terrorism-and-the-christchurch-m osque-massacre/ . ↗
  13. Even-Shoshan, Avraham. The New Dictionary [in Hebrew]. Jerusalem: Kirjat Sepher, 1970. find
  14. Fawcett, Tom. ”An Introduction to ROC Analysis.” Pattern Recognition Letters 27/8 (2006): 861–74. https://doi.org/10.1016/j.patrec.2005.10.010 ↗
  15. Fokkelman, Joannes Petrus. Reading Biblical Poetry: An Introductory Guide. Translated by Ineke Smit. Louisville, KY: Westminster John Knox Press, 2001. find
  16. Fono, Niv, Harel Moshayof, Eldar Karol, Itai Assraf, and Mark Last. ”Embible: Reconstruction of Ancient Hebrew and Aramaic Texts Using Transformers.” Pages 846–52 in Findings of the Association for Computational Linguistics: EACL 2024. St. Julian’s, Malta: Association for Computational Linguistics, 2024. find our card
  17. Gao, Jun, Di He, Xu Tan, Tao Qin, Liwei Wang, and Tieyan Liu. ”Representation Degeneration Problem in Training Natural Language Generation Models.” Pages 1–14 in International Conference on Learning Representations. New Orleans, LA: OpenReview, 2019. find
  18. Gao, Tianyu, Xingcheng Yao, and Danqi Chen. ”SimCSE: Simple Contrastive Learning of Sentence Embeddings.” Pages 6894–6910 in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. Punta Cana, Dominican Republic: Association for Computational Linguistics, 2021. find
  19. Godey, Nathan, Éric de la Clergerie, and Benoît Sagot. ”Anisotropy Is Inherent to Self-Attention in Transformers.” Pages 35–48 in Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics, Volume 1: Long Papers. St. Julian’s, Malta: Association for Computational Linguistics, 2024. find
  20. Gosset, William S. [Student]. ”The Probable Error of a Mean.” Biometrika 6 (1908): 1–25. find
  21. HaCohen-Kerner, Yaakov, Hananya Beck, Elchai Yehudai, and Dror Mughaz. ”Identifying Historical Period and Ethnic Origin of Documents Using Stylistic Feature Sets.” Pages 102–13 in Discovery Science. Springer Berlin Heidelberg, 2006. find
  22. HaCohen-Kerner, Yaakov, Hananya Beck, Elchai Yehudai, and Dror Mughaz. ”Stylistic Feature Sets As Classifiers of Documents According to Their Historical Period and Ethnic Origin.” Applied Artificial Intelligence 24/9 (2010): 847–62. find
  23. HaCohen-Kerner, Yaakov, Nadav Schweitzer, and Dror Mughaz. ”Automatically Identifying Citations in Hebrew-Aramaic Documents.” Cybernetics and Systems 42/3 (2011): 180–97. find
  24. Jurafsky, Daniel, and James H. Martin. Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition, with Language Models. 3rd ed. Online manuscript released January 6, 2026. find
  25. Kalimi, Isaac. The Reshaping of Ancient Israelite History in Chronicles. University Park, PA: Penn State University Press, 2012. find
  26. Kaplan, Jared, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. ”Scaling Laws for Neural Language Models.” arXiv:2001.08361, 2020. https://doi.org/10.48550/arXiv.2001.08361 ↗
  27. Knoppers, Gary N. I Chronicles 10–29: A New Translation with Introduction and Commentary. Anchor Bible 12A. New York: Doubleday, 2004. find
  28. Kugel, James L. The Idea of Biblical Poetry: Parallelism and Its History. New Haven: Yale University Press, 1981. find
  29. Levenshtein, Vladimir I. ”Binary Codes Capable of Correcting Deletions, Insertions, and Reversals.” Soviet Physics Doklady 10 (1965): 707–710. find
  30. Li, Bohan, Hao Zhou, Junxian He, Mingxuan Wang, Yiming Yang, and Lei Li. ”On the Sentence Embeddings from Pre-trained Language Models.” Pages 9119–30 in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Online: Association for Computational Linguistics, 2020. find
  31. Li, Qian, Hao Peng, Jianxin Li, Congying Xia, Renyu Yang, Lichao Sun, Philip S. Yu, and Lifang He. ”A Survey on Text Classification: From Traditional to Deep Learning.” ACM Transactions on Intelligent Systems and Technology 13/2 (2022): 1–41. find
  32. Liebeskind, Chaya, and Shmuel Liebeskind. ”Deep Learning for Period Classification of Historical Hebrew Texts.” Journal of Data Mining & Digital Humanities (2020): 5864. find
  33. Liebeskind, Chaya, Shmuel Liebeskind, and Dan Bouhnik. ”Machine Translation for Historical Research: A Case Study of Aramaic-Ancient Hebrew Translations.” Journal on Computing and Digital Heritage 17/2 (2024): 1–23. find
  34. Lobbezoo, Bert. ”Computer-Based Recognition of Intertextuality within the Hebrew Bible.” Master’s Thesis. Delft University of Technology, 2015. find
  35. McGovern, Hope, Hale Sirin, Tom Lippincott, and Andrew Caines. ”Detecting Narrative Patterns in Biblical Hebrew and Greek.” Pages 269–79 in Proceedings of the 1st Workshop on Machine Learning for Ancient Languages (ML4AL 2024). Association for Computational Linguistics, 2024. find our card
  36. Miller, George A. ”WordNet: A Lexical Database for English.” Communications of the ACM 38/11 (1995): 39–41. find our card
  37. Miller, Hadar, Tsvi Kuflik, and Moshe Lavee. ”Text Alignment in the Service of Text Reuse Detection.” Applied Sciences 15/6 (2025): 3395. find
  38. Mosbach, Marius, Maksym Andriushchenko, and Dietrich Klakow. ”On the Stability of Fine-Tuning BERT: Misconceptions, Explanations, and Strong Baselines.” 9th International Conference on Learning Representations (ICLR) 2021. find
  39. Mrkšić, Nikola, Diarmuid Ó Séaghdha, Blaise Thomson, Milica Gašić, Lina Rojas-Barahona, Pei-Hao Su, David Vandyke, Tsung-Hsien Wen, and Steve Young. ”Counter-fitting Word Vectors to Linguistic Constraints.” Pages 142–48in Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego, CA: Association for Computational Linguistics, 2016. find
  40. Mughaz, Dror, Yaakov HaCohen-Kerner, and Dov Gabbay. ”Extracting and Tagging Unstructured Citation of a Hebrew Religious Document.” Pages 461–73 in Proceedings of the Information Science and Information Technology Education Conference, 2019. find
  41. Naaijer, Martijn, and Dirk Roorda. ”Parallel Texts in the Hebrew Bible, New Methods and Visualizations.” (2016). https://hal.science/hal-01283051v1. ↗ our card
  42. Naaijer, Martijn, Constantijn Sikkel, Mathias Coeckelbergs, Jisk Attema, and Willem Van Peursen. ”A Transformer-Based Parser for Syriac Morphology.” Pages 23–29 in Proceedings of the Ancient Language Processing Workshop Associated with the RANLP-2023. Association for Computational Linguistics, 2023. find our card
  43. Okuda, Nozomu. ”Deep Learning for Intertext Discovery in Latin Epic: Latin SBERT and the Lucan-Vergil Benchmark.” ProQuest Dissertations and Theses. Ph.D., State University of New York at Buffalo, 2022. find
  44. Ramdas, Aaditya, Nicolás Trillos, and Marco Cuturi. ”On Wasserstein Two-Sample Testing and Related Families of Nonparametric Tests.” Entropy 19/2 (2017): 47. find
  45. Reimers, Nils, and Iryna Gurevych. ”Reporting Score Distributions Makes a Difference: Performance Study of LSTM-Networks for Sequence Tagging.” Pages 338–48 in Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Copenhagen: Association for Computational Linguistics, 2017. . ”Sentence-BERT: Sentence Embeddings Using Siamese BERT-Networks.” Pages 3980–90 in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Association for Computational Linguistics, 2019. find
  46. Reshef, Yael. Historical Continuity in the Emergence of Modern Hebrew. Lanham, MD: Lexington Books, 2019. find
  47. Rodrigue Schwarzwald, Ora. ”Lexicon: Modern Hebrew.” Encyclopedia of Hebrew Language and Linguistics Online: Brill, 2013. . ”Morphology: Modern Hebrew.” Encyclopedia of Hebrew Language and Linguistics Online: Brill, 2013. . ”The Components of the Hebrew Lexicon: The Influence of Hebrew Classical Sources, Jewish Languages and Other Foreign Languages on Modern Hebrew.” Hebrew Linguistics 39 (1995): 79–90. find
  48. Roorda, Dirk, Gino Kalkman, Martijn Naaijer, and Andreas van Cranenburgh. ”LAF-Fabric: A Data Analysis Tool for Linguistic Annotation Framework with an Application to the Hebrew Bible.” arXiv:1410.0286, 1 October 2014. https://doi.org/10.48550/arXiv.1410.0286 ↗ our card
  49. Rosensweig, Elisha, Benjamin Resnick, Hillel Gershuni, Joshua Guedalia, Nachum Dershowitz, and Avi Shmidman. ”Automatic Text Segmentation of Ancient and Historic Hebrew.” Pages 1–11 in Proceedings of the Second Workshop on Ancient Language Processing. Association for Computational Linguistics, 2025. find our card
  50. Schmid, Friedrich, and Axel Schmidt. ”Nonparametric Estimation of the Coefficient of Overlapping— Theory and Empirical Application.” Computational Statistics & Data Analysis 50 (2006): 1583–1596. find
  51. Seker, Amit, Elron Bandel, Dan Bareket, Idan Brusilovsky, Refael Shaked Greenfeld, and Reut Tsarfaty. ”AlephBERT: A Hebrew Large Pre-Trained Language Model to Start-off Your Hebrew NLP Application With.” arXiv:2104.04052, 8 April 2021. https://doi.org/10.48550/arXiv.2104.04052 ↗ our card
  52. Shmidman, Avi, and Moshe Koppel. ”Torah Study and the Digital Revolution: A Glimpse of the Future.” The Lehrhaus, 28 January 2020. https://thelehrhaus.com/commentary/torah-study-and -the-digital-revolution-a-glimpse-of-the-future/ . ↗
  53. Shmidman, Avi, Joshua Guedalia, Shaltiel Shmidman, Cheyn Shmuel Shmidman, Eli Handel, and Moshe Koppel. ”Introducing BEREL: BERT Embeddings for Rabbinic-Encoded Language.” arXiv:2208.01875, 2022. https://doi.org/10.48550/arXiv.2208.01875 ↗ our card
  54. Shmidman, Avi, Moshe Koppel, and Ely Porat. ”Identification of Parallel Passages Across a Large Hebrew/Aramaic Corpus.” Journal of Data Mining & Digital Humanities Special Issue on Computer-Aided Processing of Intertextuality in Ancient Languages (2018): 1388. find our card
  55. Shmidman, Avi, Ometz Shmidman, Hillel Gershuni, and Moshe Koppel. ”MsBERT: A New Model for the Reconstruction of Lacunae in Hebrew Manuscripts.” Pages 13–18 in Proceedings of the 1st Workshop on Machine Learning for Ancient Languages (ML4AL 2024). Association for Computational Linguistics, 2024. find our card
  56. Shmidman, Avi, Shaltiel Shmidman, Moshe Koppel, and Yoav Goldberg. ”Nakdan: Professional Hebrew Diacritizer.” Pages 197–203 in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations. Association for Computational Linguistics, 2020. find our card
  57. Shmidman, Avi. ”Automatic Identification of Biblical Citations and Allusions in Hebrew Texts.” Pages 335–48 in Jewish Studies in the Digital Age. De Gruyter, 2022. find
  58. Shmidman, Shaltiel, Avi Shmidman, and Moshe Koppel. ”DictaBERT: A State-of-the-Art BERT Suite for Modern Hebrew.” arXiv:2308.16687, 13 October 2023. https://doi.org/10.48550/arXiv.2308. ↗ our card
  59. Shmidman, Shaltiel, Avi Shmidman, Amir DN Cohen, and Moshe Koppel. ”Adapting LLMs to Hebrew: Unveiling DictaLM 2.0 with Enhanced Vocabulary and Instruction Capabilities.” arXiv:2407.07080, 9 July 2024. https://doi.org/10.48550/arXiv.2407.07080 ↗ our card
  60. Shmidman, Shaltiel, Avi Shmidman, Moshe Koppel, and Reut Tsarfaty. ”MRL Parsing Without Tears: The Case of Hebrew.” Pages 4537–50 in Findings of the Association for Computational Linguistics ACL 2024. Association for Computational Linguistics, 2024. find our card
  61. Sivan, Reuven. The Revival of the Hebrew Language. Jerusalem: Rubinstein, 1980. find
  62. Smiley, David M. ”Intertextual Parallel Detection in Biblical Hebrew: A Transformer-Based Benchmark.” arXiv:2506.24117, 30 June 2025. https://doi.org/10.48550/arXiv.2506.24117 . “T’OMIM: Tanakh Observable Matches of Intertextual Mimesis”. Zenodo, March 22, 2026. https://doi.org/10.5281/zenodo.19135731 ↗ our card
  63. Tan, Qitao, Xiaoying Song, Guanghui Ye, and Chuan Wu. ”An Effective Negative Sampling Approach for Contrastive Learning of Sentence Embedding.” Machine Learning 112 (2023): 4837–61. find
  64. Thakur, Nandan, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. ”BEIR: A Heterogeneous Benchmark for Zero-shot Evaluation of Information Retrieval Models.” Pages 1–16 in Proceedings of the 2021 Conference on Neural Information Processing Systems: Datasets and Benchmarks Track. find
  65. Tschuggnall, Michael, and Günther Specht. ”From Plagiarism Detection to Bible Analysis: The Potential of Machine Learning for Grammar-Based Text Analysis.” Pages 245–48 in Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Cham: Springer International Publishing, 2016. find
  66. Tsumura, David Toshio. Vertical Grammar of Parallelism in Biblical Hebrew. Ancient Israel and Its Literature 47. Atlanta: SBL Press, 2023. van Peursen, Willem, and Eep Talstra. ”Computer-Assisted Analysis of Parallel Texts in the Bible. The Case of 2 Kings Xviii-Xix and Its Parallels in Isaiah and Chronicles.” Vetus Testamentum 57/1 (2007): 45–72. van Peursen, Willem, Constantijn Sikkel, and Dirk Roorda. ”Hebrew Text Database ETCBC4b.” Eep Talstra Centre for Bible and Computing, VU University Amsterdam, and Data Archiving and Networked Services, Royal Netherlands Academy of Arts and Sciences, 2015. find
  67. Vaswani, Ashish, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. ”Attention Is All You Need.” Pages 6000–6010 in Proceedings of the 31st Conference on Neural Information Processing Systems (NIPS). Long Beach, CA: Curran Associates, 2017. find
  68. Wang, Hao, and Yong Dou. ”SNCSE: Contrastive Learning for Unsupervised Sentence Embedding with Soft Negative Samples.” Pages 419–31 in Advanced Intelligent Computing Technology and Applications. Singapore: Springer Nature Singapore, 2023. find
  69. Wang, Jiapeng, and Yihong Dong. ”Measurement of Text Similarity: A Survey.” Information 11/9 (2020): 421. find
  70. Woods, Peter J. ”Shitposting as Public Pedagogy.” Curriculum Inquiry 53/4 (2023): 359–80. find
  71. Xu, Lanling, Jianxun Lian, Wayne Xin Zhao, Ming Gong, Linjun Shou, Daxin Jiang, Xing Xie, and Ji-Rong Wen. ”Negative Sampling for Contrastive Representation Learning: A Review.” arXiv:2206.00212, 2022. https://doi.org/10.48550/arXiv.2206.00212 ↗
  72. Xu, Lingling, Haoran Xie, Zongxi Li, Fu Lee Wang, Weiming Wang, and Qing Li. ”Contrastive Learning Models for Sentence Representations.” ACM Transactions on Intelligent Systems and Technology 14/4 (2023): 1–34. find
  73. Yamini, Bat-Zion. ”The Revival of Ancient Hebrew Words With the Revival of Israel.” Sociology Study 9/4 (2019): 156–168. find
  74. Yang, Zhen, Ming Ding, Tinglin Huang, Yukuo Cen, Junshuai Song, Bin Xu, Yuxiao Dong, and Jie Tang. ”Does Negative Sampling Matter? A Review with Insights into Its Theory and Applications.” IEEE Transactions on Pattern Analysis and Machine Intelligence 46/8 (2024): 5692–711. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/hbo-2026-smiley-miqrabert-regression-based-sentence-bert/