Intertextual Parallel Detection in Biblical Hebrew: A Transformer-Based Benchmark

What it is about

The paper tests whether off-the-shelf sentence embedders can find the verses of Chronicles that reuse Samuel and Kings (558 known parallel verses, BHSA text). Each verse is embedded with four pretrained models (multilingual E5, AlephBERT trained on Modern Hebrew, MPNet, LaBSE); the score is cosine similarity to the known parallel against the mean over all non-parallels, plus the Wasserstein distance between the two distributions and nearest-neighbour precision/recall. E5 gives the highest similarity to true parallels (mean 0.966) but also scores non-parallels high (0.882); AlephBERT separates them better (0.914 vs 0.638). As nearest-neighbour retrieval, E5 reaches F1 0.88 and AlephBERT 0.87, MPNet and LaBSE 0.78. The author admits the embeddings cannot say why two verses match, and that only narrative was tested.

Ideas

• I1. A transparent baseline for the same benchmark: verse similarity by weighted overlap of MACULA lemmas (or roots), plus the overlap of (head lemma, relation, dependent lemma) triples from our trees. Report the same nearest-neighbour precision/recall against the paper's 0.88, with each match explained by the shared lemmas and triples.
• I2. Inside the Torah, use the same method to list the internal parallels (the Decalogue in Ex 20 and Deut 5, the tabernacle instructions Ex 25–31 against their execution Ex 35–40, Deuteronomy's retellings of Numbers) and record, per pair, the added, omitted and changed words, the categories of Kalimi's rewriting table that the paper reproduces (addition, omission, name change).
• I3. Keep the paper's evaluation design as Mova's habit for any similarity measure: score true pairs against the mean of all non-pairs and report the gap, not only the hit rate, since a measure that rates everything similar looks good on hits alone.

For Mova

I1 is a small tool over the Torah trees (a subcommand next to macula dict, or under tools/macula); the BHSA verse list of Samuel/Kings–Chronicles parallels is the gold, the books themselves are outside our trees, so the benchmark needs the MACULA text of those books. I2 is a count and a table for Discourse: narrative, speech and the shape of a text (repetition and retelling) and an aid to the Torah dictionary's examples. I3 is a measurement rule for chapter 13.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. arXiv 2506.24117

@misc{smiley2025intertextual,
  title = {{Intertextual Parallel Detection in Biblical Hebrew: A Transformer-Based Benchmark}},
  author = {Smiley, David M.},
  year = {2025},
  eprint = {2506.24117},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2506.24117}
}

References

22 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019) “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding”, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 4171–4186. https://doi.org/10.18653/ v1/N19-1423 ↗
2. Enders, J., Millar, W., & Burns, J. (1998). Chronicles and Its Synoptic Parallels in Samuel, Kings, and Related Biblical Texts. The Liturgical Press. find
3. Feng, F., Yang, Y., Cer, D., et al., (2022). Language-agnostic BERT Sentence Embedding. https://doi.org/10.48550/arXiv.2007.01852 ↗
4. Fewell, D. (1992). Reading between Texts: Intertextuality and the Hebrew Bible. Louisville: Westminster John Knox Press. find
5. Fishbane, M. (1985). Biblical Interpretation in Ancient Israel. New York: Oxford Press. find
6. Gomez, J., and Vázquez, P. (2022). An Empirical Evaluation of Document Embeddings and Similarity Metrics for Scientific Articles. Applied Sciences, 12 (11), 5664. https://doi.org/10.3390/app12115664 ↗
7. Harvey, J. (2004). Retelling the Torah: The Deuteronomistic Historian’s Use of Tetra-teuchal Narratives. New York: Bloomsbury. find
8. Kalimi, I. (2004). The Reshaping of Ancient Israelite History in Chronicles. Eisenbrauns. find
9. Leo, J., Ge, E., and Li, S., (2023) Wasserstein Distance in Deep Learning. http://dx.doi.org/10.2139/ssrn.4368733 ↗
10. Mars, M. (2022). From Word Embeddings to Pre-Trained Language Models: A State-of-the-Art Walkthrough. Applied Sciences, 12 (17), 8805. https://doi.org/10.3390/ app12178805 ↗
11. Miller, G. (2011). Intertextuality in Old Testament Research. Currents in Biblical Research, 9, pp. 283–309. find
12. Rainio, O., Tueho, J., and Klén, R. (2024). Evaluation Metrics and Statistical Tests for Machine Learning. Scientific Reports, 14, 6086. https://doi.org/10.1038/ s41598-024-56706-x ↗
13. Schnittjer, G. (2021). Old Testament Use of the Old Testament. Grand Rapids: Zondervan. find
14. Seker, M., Bandel, E., Barekat, D., et al., (2021). AlephBERT: A Hebrew Large Pre-Trained Language Model to Start-off your Hebrew NLP Application With. In Proceedings of the 2022 Conference on Computational Natural Language Processing, pp. 321–330. https://doi.org/10.48550/arXiv.2104.04052 ↗ our card
15. Shmidman, A., Koppel, M., and Porat, E. (2018). Identification of Parallel Passages Across a Large Hebrew/Aramaic Corpus. Journal of Data Mining and Digital Humanities. https://doi.org/10.46298/jdmdh.1388 ↗ our card
16. Shmidman, A. (2022). Automatic Identification of Biblical Citations and Allusions in Hebrew Texts. In Jewish Studies in the Digital Age, pp. 335-348. https://doi.org/ 10.1515/9783110744828-015 ↗
17. Singh, N., and Habash, N. (2012). Hebrew Morphological Preprocessing for Statistical Machine Translation. In Proceedings of the 16th EAMT Conference, pp. 43-50. find our card
18. Song, K., Tan, X., Qin, T., et al., (2020). MPNet: Masked and Permuted Pre-training for Language Understanding. In Proceedings of the 34th International Conference on Neural Information Processing Systems, pp. 16857–16867. https://doi.org/10.48550/ arXiv.2004.09297 ↗
19. van Peursen, W., and Talstra, E. (2007). Computer-Assisted Analysis of Parallel Texts in the Bible. Vetus Testamentum, 57 (1), pg. 45–72. find
20. van Peursen, W., Sikkel, C., and Roorda, D. (2015). Hebrew Text Database ETCBC4b. https://doi.org/10.17026/dans-z6y-skyh ↗
21. Vaswani, A., Shazeer, N., Parmar, N., et al., (2017). Attention is All You Need. In Advances in Neural Information Processing Systems 30 (NIPS 2017), pp. 5998–6008. https://doi.org/10.48550/arXiv.1706.03762 ↗
22. Wang, L., Yang, N., Huang, X., et al., (2024). Multilingual E5 Text Embeddings: A Technical Report. https://doi.org/10.48550/arXiv.2402.05672 ↗

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/hbo-2025-smiley-intertextual-parallel-detection-biblical-hebrew/