The Hebrew Bible as Data: Laboratory – Sharing – Experiences
What it is about
A history-and-practice report on how the ETCBC (formerly WIVU) text database of the Hebrew Bible was opened up. The database, built since the 1970s on BHS, holds observations rather than a theory: words, phrases, clauses and sentences as objects with features, in several overlapping hierarchies (Doedens' text model, queried through Emdros/MQL). The author converted it to the ISO Linguistic Annotation Framework (stand-off annotation over an immutable Unicode text: 426 555 words, about 946 000 objects, 25 million features), wrote LAF-Fabric to compile that XML into fast binary structures, and built SHEBANQ, a site where saved queries show up as margin annotations on the verses they hit. Lessons: immutable primary text makes stand-off annotation work; publish feature documentation with frequency lists of every value; a researcher is closer to a scientific programmer than to a software engineer.
Ideas
- I1. Queries as annotations: a saved
maculaquery (a pattern over our trees, e.g. "fronted subject before qatal") is stored with its author's note and its hits, and the Torah reader shows the hits beside each verse. A cheap way to makelanguages/hbo/examples live. - I2. For every FEATS and MISC key the converter writes, publish the full value set with counts (one table, regenerated by the tool), as the ETCBC feature docs do.
- I3. Keep the WLC text immutable and every layer (trees, prag, dictionary links) stand-off, keyed by word position; check that a rebuild never changes the base text (byte-identical gate).
- I4. Compile once, load fast: keep a binary cache of the parsed Torah trees for
praganddictruns, rebuilt only when the hash of the input changes.
For Mova
I1 fits the parse database and the Torah course (13-mova-tools.md §10) and the examples of every chapter. I2 belongs in 13-mova-tools.md §5 (FEATS and MISC) as a generated table from tools/macula. I3 and I4 are engineering rules for tools/macula (and match the world store's hashing); I3 is a gate that should be shown red once on a deliberately altered text.
Cite
@misc{roorda2015hebrew,
title = {{The Hebrew Bible as Data: Laboratory – Sharing – Experiences}},
author = {Roorda, Dirk},
year = {2015},
eprint = {1501.01866},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/1501.01866}
}References
- Bouda, Peter (2013-2014), graf-python. Python software on Github. https://github.com/cidles/ graf-python. ↗
- Bouda, Peter, Vera Ferreira, and António Lopes (2012), Poio API - an annotation framework to bridge language documentation and natural language processing., Proceedings of The Second Workshop on Annotation of Corpora for Research in the Humanities, Lisbon, 2012, Lisbon, Portugal. ISBN: 978-989-689-273-9, http://alfclul.clul.ul.pt/crpc/acrh2/ACRH-2_papers/ Bouda-Ferreira-Lopes.pdf. ↗
- Di Pierro, Massimo (2015), Web2py. full stack web framework, 6th edition. Online book. http://web2py.com/book. ↗
- Doedens, Crist-Jan (1994), Text Databases. One Database Model and Several Retrieval Languages, number 14 in Language and Computers, Editions Rodopi, Amsterdam, Netherlands and Atlanta, USA. ISBN: 90-5183-729-1, http://books.google.nl/books?id=9ggOBRz1dO4C. ↗
- Gleick, James, editor (2011), The Information: a History, a Theory, a Flood, HarperCollins. ISBN: 0007225741, http://en.wikipedia.org/wiki/The_Information:_A_History,_a_Theory, _a_Flood. ↗
- Ide, Nancy and Laurent Romary (2012), Linguistic Annotation Framework. ISO standard 24612:2012. Edition 1, 2012-06-15. http://www.iso.org/iso/home/store/catalogue_tc/ catalogue_detail.htm?csnumber=37326. ↗
- Kalkman, Gino J. (2013), Functions of asyndetic clause relations in biblical Hebrew. IPython Notebook. http://nbviewer.ipython.org/github/ETCBC/Biblical_Hebrew_Analysis/blob/ master/Miscellaneous/AsyndeticClauseFunctions.ipynb. ↗
- Kalkman, Gino J. (to be published 2015), Verbal Forms in Biblical Hebrew Poetry: Poetical Freedom or Linguistic System?, PhD thesis, VU University, Amsterdam. http://nbviewer.ipython.org/github/ETCBC/Biblical_Hebrew_Analysis/blob/master/PhD/Introduction.ipynb. ↗
- Kittel, Rudolf, editor (1968-1997), Biblia Hebraica Stuttgartensia, Deutsche Bibelgesellschaft, Stuttgart, Germany. http://www.bibelwissenschaft.de/startseite/ wissenschaftliche-bibelausgaben/biblia-hebraica/bhs/. ↗
- Pérez, Fernando and Brian E. Granger (2007), IPython: a system for interactive scientific computing, Computing in Science and Engineering 9 (3), pp. 21–29, IEEE Computer Society. http:// ipython.org, ISSN: 1521-9615, DOI: 10.1109/MCSE.2007.53. ↗
- Petersen, Ulrik (2002-2014), Emdros. text database engine for analyzed or annotated text. Open Source software. http://emdros.org. ↗
- Petersen, Ulrik (2004), Emdros - a text database engine for analyzed or annotated text, Proceedings of COLING 2004, p. 11901193. http://emdros.org/petersen-emdros-COLING-2004.pdf. ↗
- Petersen, Ulrik (2006), Principles, Implementation Strategies, and Evaluation of a Corpus Query System, Vol. 4002, Springer, p. 215226. http://link.springer.com/chapter/10.1007% 2F11780885_21. ↗
- Peursen, Wido Th. van and Dirk Roorda (2014), Hebrew text database in Linguistic Annotation Framework. Dataset available online at Data Archiving and Networked services, Den Haag, Netherlands. PID: urn:nbn:nl:ui:13-048i-71, http://www.persistent-identifier.nl/ ?identifier=urn:nbn:nl:ui:13-048i-71. ↗
- Roorda, Dirk (2012), Queries-as-annotations. Github repository. https://github.com/Dans-labs/ annotation-paradigm. ↗
- Roorda, Dirk (2013-2014b), LAF-Fabric. workbench for analysing LAF resources. Python software on Github. https://github.com/ETCBC/laf-fabric. ↗
- Roorda, Dirk (2014a), LAF-Fabric notebooks. examples of data processing for the hebrew bible. Python software on Github. https://github.com/ETCBC/laf-fabric-nbs. ↗
- Roorda, Dirk and Charles M.J.M. van den Heuvel (2012), Annotation as a new paradigm in research archiving, Proceedings of ASIS&T 2012 Annual Meeting. Final Papers, Panels and Posters. https://www.asis.org/asist2012/proceedings/Submissions/84.pdf (author’s version: http://arxiv.org/abs/1412.6069). ↗ our card
- Roorda, Dirk, Constantijn Sikkel, and Wido Th van Peursen (2014), Feature documentation of shebanq. Github repository published on readthedocs. https://github.com/ETCBC/ shebanq-doc. ↗
- Roorda, Dirk, Jan Krans, Bert-Jan Lietaert-Peerbolte, Wido Th. van Peursen, Ulrik Sandborg-Petersen, and Eep Talstra (2012), Scientific report of the workshop biblical scholarship and humanities computing: Data types, text, language and interpretation, held at the lorentz centre leiden from 6 feb 2012 through 10 feb 2012, Technical report, Lorentz Center, Leiden. http://www.lorentzcenter.nl/lc/web/2012/480/report.php3?wsid=480&venue=Oort. ↗
- Roorda, Dirk, Martijn Naaijer, Gino J. Kalkman, and Andreas van Cranenburgh (to appear 2015), Laf-fabric: a data analysis tool for linguistic annotation framework with an application to the hebrew bible, Computational Linguistics in the Netherlands Journal. ISSN: 2211-4009, preprint http://arxiv.org/abs/1410.0286. ↗ our card
- Talstra, Eep and Constantijn J. Sikkel (2000), Genese und kategorienentwicklung der wivu-datenbank, in Hardmeier, C. et al, editor, Ad Fontes! Quellen erfassen lesen deuten. Wat ist Computerphilologie? Ansatzpunkte und Methodologie Instrument und Praxis, VU University Press, Amsterdam, Netherlands, pp. 33–68. find
- Talstra, Eep, Constantijn J. Sikkel, Oliver Glanz, Reinoud Oosting, and Janet W. Dyk (2012), Text database of the hebrew bible. Dataset available online after permission of the depositor at Data Archiving and Networked services, Den Haag, Netherlands. http://www.persistent-identifier.nl/?identifier=urn:nbn:nl:ui:13-ukhm-eb. ↗
- van Peursen, Wido Th., Dirk Roorda, Henk van den Berg, and Heleen van de Schraaf (2014), SHEBANQ, a search engine and query saver for the etcbc text database of the hebrew bible. web application. http://shebanq.ancient-data.org. ↗
- van Zundert, Joris et al. (2012), Proceedings of the leuven2012 interedition workshop. Wiki page. http://interedition.eu/wiki/index.php/Leuven2012_Proceedings. ↗
This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/hbo-2015-roorda-hebrew-bible-data-laboratory-sharing/