Mova0.5 pre-release⌕
☰
home / docs / papers / Ancient Hebrew / Critical biblical studies via word frequency analysis: unveiling text authorship

Critical biblical studies via word frequency analysis: unveiling text authorship

What it is about

Fifty chapters whose source is broadly agreed (old Deuteronomy D, the Deuteronomistic History DtrH in Joshua–Kings, and Priestly texts P in Genesis–Leviticus) are turned into lemma counts using OSHB's lemma numbers, with all proper names collapsed to one code and gentilics to another. Two word-frequency tables are compared by Higher Criticism (Kipnis–Donoho): many two-sample binomial tests, one per lemma, combined so the score is sensitive to a few words that deviate strongly, with no word list chosen in advance. In leave-one-out tests only 2 of 50 chapters are wrongly rejected from their corpus, and attribution to the closest corpus is right for 84% (P 95%, D 78%, DtrH 72%); D and DtrH sit close, P apart. Accuracy depends on length: under 10 verses it is poor, 10–30 verses about 80%. The method names the discriminating lemmas behind each decision and is then applied to disputed texts (Deut 4, Lev 26, Gen 14–15, 20, 22, 24, and others).

Ideas

For Mova

I1 is a small Rust module beside the parse database (or macula) with its own negative control (shuffled chapters must score as same-source). I2 feeds Discourse: narrative, speech and the shape of a text (genres) and candidate genre cues in tools/macula/src/prag.rs. I3 is a check reported in 13-mova-tools.md; I4 is a convention for every measure there. Mova takes no side on source criticism; the corpora are only a labelled test.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. arXiv 2410.19883

@misc{faigenbaumgolovin2024critical,
  title = {{Critical biblical studies via word frequency analysis: unveiling text authorship}},
  author = {Faigenbaum-Golovin, Shira and Kipnis, Alon and Buhler, Axel and Piasetzky, Eli and Romer, Thomas and Finkelstein, Israel},
  year = {2024},
  eprint = {2410.19883},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2410.19883}
}

References

65 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Schniedewind, W. M., How the Bible Became a Book: The Textualization of Ancient Israel (Cambridge: Cambridge University Press; 2004. find
  2. Kratz, R. G., The composition of the Narrative Books of the Old Testament (A&C Black); 2005. find
  3. Schmid, K., The Old Testament: A Literary History (Fortress Press); 2012. find
  4. R ö mer, T., The so-called Deuteronomistic history: a sociological, historical and literary introduction (London); 2005. find
  5. Finkelstein, I., Essays on biblical historiography: from Jeroboam II to John Hyrcanus I. Essays on Biblical historiography. 2022; 1–601. find
  6. Stamatatos, E. A survey of modern authorship attribution methods. J. Am. Soc. for information Sci. Technol.; 2009; 60 (32):538-556. find
  7. Swain, S., Mishra, G., & Sindhu, C., Recent approaches on authorship attribution techniques—An overview. In 2017 Int. Conf. of Electronics, Communication and Aerospace Technology, 2017; 1:557-566. find
  8. Tyo, J., Dhingra, B., & Lipton, Z. C., Valla: Standardizing and Benchmarking Authorship Attribution and Verification Through Empirical Evaluation and Comparative Analysis. In Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics, 2023; 649-660. find
  9. Portnoy, S. L., Petersen, D. L., Biblical texts and statistical analysis: Zechariah and beyond. J. Biblical Lit. 1984; 103:11–21. find
  10. Holmes, D. I., Authorship attribution. Comput. Humanit. 1994; 28:87–106. find
  11. Stamatatos, E., A survey of modern authorship attribution methods. J. Am. Soc. for information Sci. Technol. 2009; 60:538–556. find
  12. Jockers, M. L., Witten, D. M., A comparative study of machine learning methods for authorship attribution. Lit. Linguist. Comput. 2010; 25:215–223. find
  13. Neal, T., Sundararajan, K., Fatima, A., Yan, Y., Xiang, Y., & Woodard, D, Surveying stylometry techniques and applications. ACM Computing Surveys (CSuR), 2017; 50(6):1-36. find
  14. Koppel, M. Akiva, N., Dershowitz, I., Dershowitz, N., Unsupervised decomposition of a document into authorial components in Proc. of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies. 2011; 1356–1364. find our card
  15. Dershowitz, I., Akiva, N., Koppel, M., Dershowitz, N., Computerized source criticism of biblical texts. J. Biblical Lit. 2015; 134:253–271. find
  16. Yoffe, Gideon, Axel Bühler, Nachum Dershowitz, Israel Finkelstein, Eli Piasetzky, Thomas Römer, and Barak Sober. “A Statistical Exploration of Text Partition into Constituents: The Case of the Priestly Source in the Books of Genesis and Exodus.” Findings of the Association for Computational Linguistics. 2023; 1918–40. find
  17. Bühler, Axel, Gideon Yoffe, Nachum Dershowitz, Eli Piasetzky, Israel Finkelstein, Thomas Römer, and Barak Sober. “Exploring the Stylistic Uniqueness of the Priestly Source in Genesis and Exodus Through a Statistical/Computational Lens.” Zeitschrift Für Die Alttestamentliche Wissenschaft. 2024; 136 (2): 165–90. find
  18. Stamatatos, E., et al., Overview of the authorship verification task at PAN 2022. In CEUR workshop proceedings 2022; 3180:2301-2313. find
  19. Xia, S. A. Genre analysis in the digital era: Developments and challenges. ESP Today, 8(1), 2020:141-159. find
  20. Shrestha, P., Sierra, S., González, F. A., Montes, M., Rosso, P., & Solorio, T. Convolutional neural networks for authorship attribution of short texts in Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, 2017: 2: 669–674. find
  21. Wang, L., News authorship identification with deep learning in Conference and Labs of the Evaluation Forum, Portugal 2016. find
  22. Kipnis, A., Higher criticism for discriminating word-frequency tables and authorship attribution. The Annals Appl. Stat. 2022; 16:1236–1252. find
  23. Kipnis, A., Donoho, D. L., Two-sample testing of discrete distributions under rare/weak perturbations in IEEE Int. Sym. on Inf. Theory (ISIT) 2021; 3314–3319. find
  24. Donoho, D. L., Kipnis, A., Higher criticism to compare two large frequency tables, with sensitivity to possible rare and weak differences. The Annals Stat. 2022; 50:1447–1472. find
  25. Donoho, D. L., Jin, J., sample testing of discrete distributions under rare/weak pertuxtures. The Annals Stat. 2004; 32:962–994. find
  26. Donoho, D. L., Jin, J., Higher criticism for large-scale inference, especially for rare and weak effects. Stat. Sci. 2015; 30(1):1-25. find
  27. Mosteller, F., Wallace, D. L., Inference in an authorship problem: A comparative study of discrimination methods applied to the authorship of the disputed federalist papers. J. Am. Stat. Assoc. 1963; 58:275–309. find
  28. Otto, E., Deuteronomium 1 - 11. Teilbd. 1: 1,1 - 4,43 (Freiburg Basel Wien Herder 2012) 2012: deut, 2. find
  29. Preuss, H.D., Deuteronomium. (Wissenschaftl. Buchgesellschaft) 1982; 164. find
  30. Dietrich, W., Mathys, H. P., R ö mer, T., Smend, R., Die Entstehung des Alten Testaments. (Kohlhammer Verlag) 2014. find
  31. Brown, I. V., The higher criticism comes to America, 1880—1900. J. Presbyt. Hist. Soc. (1943-220 1961) 1960; 193–212. find
  32. Nihan, C., From Priestly Torah to Pentateuch: A study in the composition of the Book of Leviticus. (Mohr Siebeck) 2007; 25. find
  33. Schmid, K, The Neo-Documentarian Manifesto: A Critical Reading. Journal of Biblical Literature, 2021; 140 (3):461-479. find
  34. Faigenbaum-Golovin, S., Shaus, A., Sober, B., Levin, D., Na’aman, N., Sass, B., et al. Algorithmic handwriting analysis of Judah’s military correspondence sheds light on composition of biblical texts. Proceedings of the National Academy of Sciences, 2016; 113 (17):4664-4669. find
  35. Faigenbaum-Golovin, S., Shaus, A., Sober, B., Turkel, E., Piasetzky, E., Finkelstein, I. Algorithmic handwriting analysis of the Samaria inscriptions illuminates bureaucratic apparatus in biblical Israel. Plos One 2020; 15 (1) find
  36. Li, J. S., Chen, L. C., Monaco, J. V., Singh, P., & Tappert, C. C., A comparison of classifiers and features for authorship authentication of social networking messages. Concurrency and Computation: Practice and Experience, 2017; 29(14): 3918. find
  37. Oliveira Jr, W., Justino, E., Oliveira, L. S. Comparing compression models for authorship attribution. Forensic science international, 2013; 228 (1-3):100-104. find
  38. Murauer, B., & Specht, G., Developing a benchmark for reducing data bias in authorship attribution. In Proc. of the 2nd Workshop on Eval. and Comparison of NLP Sys. 2021; 179-188. find
  39. Salton, G., Automatic information organization and retrieval” McGraw-Hill, New York. 1968. find
  40. Spärck Jones, K., A statistical interpretation of term specificity and its application in retrieval, Journal of Documentation, 1972:28:11–21. find
  41. Open scriptures Hebrew Bible project (2019). Original work of the Open Scriptures Hebrew Bible available at https://github.com/openscriptures/morphhb (Accessed May 8, 2024). ↗
  42. Faigenbaum-Golovin, S., Kipnis, A., Bühler, A., Piasetzky, E., Römer, T., & Finkelstein, I. (2024). Data from: Critical biblical studies via word frequency analysis: Unveiling text authorship. Duke Research Data Repository. https://doi.org/10.7924/r4cr61n46. ↗ our card
  43. Donoho, D. L., Jin, J., Higher criticism thresholding: optimal feature selection when useful features are rare and weak. Proc. Natl. Acad. Sci. 2008; 105:14790–14795. find
  44. Bevendorff, J., et al., Overview of PAN 2020: Authorship verification, celebrity profiling, profiling fake news spreaders on Twitter, and style change detection. In Experimental IR Meets Multilinguality, Multimodality, and Interaction: 11th International Conference of the CLEF Association, Proc. 2020; 11:372-383. find
  45. Finkelstein, I., R ö mer, T., The historical and archaeological background behind the old Israel ark narrative. Peeters, 2020. find
  46. Japhet, S., I and II Chronicles: a commentary. (Westminster John Knox Press) 1993. find
  47. Abadie, P., Introduction à l’Ancien Testament. (Labor et fides) 2009; 49. find
  48. Finkelstein, I., R ö mer, T., Comments on the historical background of the Abraham narrative. between “Realia” and “Exegetica”. Hebr. Bible Anc. Isr. 2014; 3:3–23. find
  49. Becker, U., Richterzeit und Königtum: Redaktionsgeschichtliche Studien zum Richterbuch. (Walter de Gruyter) 1990; 192. find
  50. Lanoir, C., “Juges”, in P. Abadie, Introduction à l’Ancien Testament. (Labor et fides) 2009; 49. find
  51. Knauf, E. A., “L “historiographie Deutéronomiste” existe-t-elle”, in Israël Construit Son Histoire. L’Historiographie Deutéronomiste à La Lumière Des Recherches Récentes, ed. by Albert de Pury, Thomas Römer, and Jean-Daniel Macchi (Genève: Labor et Fides) 1996: 409–418 find
  52. Finkelstein, I., R ö mer, T., Comments on the historical background of the Jacob narrative in Genesis. Zeitschrift für die alttestamentliche Wissenschaft 2014; 126:317–338. find
  53. Macchi, J. D., Le Livre d’Esther, Commentaire de l’Ancien Testament XIV (Genève: Labor et Fides) (2016). find
  54. Steymans, H. U., Verheißung und Drohung: Lev 26, in: H.-J. Fabry et H.-W. Jüngling (éd), Levitikus als Buch (BBB 119), Berlin - Bodenheim b. Mainz: Philo, 1999; 263-307. find
  55. Stackert, J., Rewriting the Torah: literary revision in Deuteronomy and the Holiness Legislation 52 (Forschungen zum Alten Testament,), Tübingen: Mohr Siebeck, 2007. find
  56. Nihan, C., Heiligkeitsgesetz und Pentateuch. Traditions- und kompositionsgeschichtliche Aspekte von Levitikus 26, in: F. Hartenstein et K. Schmid (éd), Abschied von der Priesterschrift? Zum Stand der Pentateuchdebatte (Veröffentlichungen der Wissenschaftlichen Gesellschaft für Theologie 40), Leipzig: Evangelische Verlagsanstalt 2015; 186-218. find
  57. Rost, L., Die Überlieferung von der Thronnachfolge Davids. (Kohlhammer) 2011; 42. find
  58. Campbell, A.F., I Samuel. (Wm. B. Eerdmans Publishing), 2003. find
  59. Bodner, K., Ark-Eology: Shifting Emphases in ‘Ark Narrative’Scholarship. Curr. biblical research, 2006; 4:169– 197. find
  60. Porter, M. F., An algorithm for suffix stripping. Program 1980. find
  61. Lovins, J. B., Development of a stemming algorithm. Mech. Transl. Comput. Linguist. 1968; 11:22–31. find
  62. Jivani, A. G., et al., A comparative study of stemming algorithms. Int. J. Comp. Tech. Appl 2011; 2:1930–1938. find
  63. Tsarfaty, R., Seker, A., Sadde, S., Klein, S., What’s wrong with Hebrew NLP? and how to make it right. arXiv 2019. arXiv:1908.05453. (Accessed May 8, 2024). ↗ our card
  64. Qi, P., Dozat, T., Zhang, Y., Manning, C. D., Universal dependency parsing from scratch. arXiv 2019. arXiv:1901.10457 (Accessed May 8, 2024). ↗
  65. Analytical tools for Hebrew texts. Dicta. Available at https://dicta.org.il/ (Accessed May 8, 2024). ↗

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/hbo-2024-faigenbaum-golovin-critical-biblical-studies-word-frequency/