LAF-Fabric: a data analysis tool for Linguistic Annotation Framework with an application to the Hebrew Bible

What it is about

The paper traces the ETCBC (formerly WIVU) Hebrew Bible database from the 1970s to its conversion into ISO LAF stand-off annotation (the SHEBANQ project), and presents LAF-Fabric, a Python tool that loads the whole annotated Bible into memory for fast feature queries (a precursor of Text-Fabric). Three pilot uses follow. (1) Presence/absence of verb and common-noun lexemes per book, drawn as a force-directed graph, groups the late books together. (2) Counts of asyndetic mother–daughter clause pairs by verb form in prose, poetry and prophecy (11,111 pairs; the top ten patterns cover about 70%) suggest one verbal system across genres, with the real split being narrative vs direct speech rather than prose vs poetry. (3) The database is exported as constituency trees with discontinuous constituents; recurring tree fragments are extracted, and a Data-Oriented Parsing grammar trained on 50,000 sentences scores 90.0 labelled F1 and 75.3% exact match on the next 2,000 (short) sentences, with gold words and tags.

Ideas

• I1. Reproduce the clause-pair table on the Torah trees: for each clause linked to its head clause without a conjunction, count (head verb form, dependent verb form, or nominal), split by narration vs character speech (the prag voice column). The paper's claim that mode of communication, not genre, drives the pattern can then be checked inside one prose corpus.
• I2. Tree-fragment mining (largest common subtrees between pairs of sentences) as a way to find recurring constructions and formulas in the Torah ("and the LORD spoke unto Moses saying", legal protases); a fragment list ranked by count is a cheap source of rules and of examples for the syntax chapter.
• I3. The DOP figures (90.0 F1 with gold tags, short sentences, held-out by position) are a baseline worth quoting when Mova reports parsing scores on Hebrew, with the caveat that ETCBC constituency and UD dependencies are not comparable directly.

For Mova

I1 is a count for Discourse: narrative, speech and the shape of a text and 06-verb.md, needing only the trees plus tools/macula/src/prag.rs voice. I2 is a tool idea for the syntax chapter (08-syntax.md) and for finding speech formulas the lemor/amar rules use. I3 belongs in 12-biblical-mishnaic-modern.md (codes and treebanks) or 13-mova-tools.md (PTNK and other measures).

Cite

The paper itself, not this card. BibTeX from the library's record of the work. arXiv 1410.0286

@article{roorda2014laf,
  title = {{LAF-Fabric: a data analysis tool for Linguistic Annotation Framework with an application to the Hebrew Bible}},
  author = {Roorda, Dirk and Kalkman, Gino and Naaijer, Martijn and Cranenburgh, Andreas van},
  journal = {Computational Linguistics in the Netherlands Journal, Volume 4, December 2014, pp. 105-109},
  year = {2014},
  eprint = {1410.0286},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1410.0286}
}

References

35 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Bergsträsser, Gotthelf (1926, 1986), Hebräische Grammatik I/II (’Hebrew Grammar’), Georg Olms Verlag, Hildesheim, Germany. https://archive.org/details/hebrischegramm00gese. ↗
2. Bod, Rens (1992), A computational model of language performance: Data-Oriented Parsing, Proceedings COLING, pp. 855–859. http://aclweb.org/anthology/C92-3126. ↗
3. Bouda, Peter et al. (2013-2014), graf-python. Python software on Github. https://github.com/ cidles/graf-python. ↗
4. Bouda, Peter, Vera Ferreira, and António Lopes (2012), POIO API - An annotation framework to bridge language documentation and natural language processing., Proceedings of The Second Workshop on Annotation of Corpora for Research in the Humanities, Lisbon, 2012, Lisbon, Portugal. ISBN: 978-989-689-273-9, http://alfclul.clul.ul.pt/crpc/acrh2/ACRH-2_papers/ Bouda-Ferreira-Lopes.pdf. ↗
5. Collins, Michael (1997), Three generative, lexicalised models for statistical parsing, Proceedings of ACL, pp. 16–23. http://aclweb.org/anthology/P97-1003. ↗
6. Doedens, Crist-Jan (1994), Text Databases. One Database Model and Several Retrieval Languages, number 14 in Language and Computers, Editions Rodopi, Amsterdam, Netherlands and Atlanta, USA. ISBN: 90-5183-729-1, http://books.google.nl/books?id=9ggOBRz1dO4C. ↗
7. Gesenius, Wilhelm (1813), Geschichte der Hebräischen Sprache und Schrift: Eine philologisch-historische Einleitung in die Sprachlehren und Wörterbücher der Hebräischen Sprache (’History of the Hebrew language and script: a philologico-historical introduction to the grammar and dictionaries of the Hebrew language’), Vogel, Leipzig. https://archive.org/details/ geschichtederheb00geseuoft. ↗
8. Hurvitz, Avi (1974), The date of the prose-tale of Job linguistically reconsidered, Harvard Theological Review. http://www.ericlevy.com/Revel/Avi%20Hurvitz%20-%20The%20Date%20of% 20the%20Prose%20Tale%20of%20Job.PDF. ↗
9. Ide, Nancy and Laurent Romary (2012), Linguistic Annotation Framework. ISO standard 24612:2012. Edition 1, 2012-06-15. http://www.iso.org/iso/home/store/catalogue_tc/ catalogue_detail.htm?csnumber=37326. ↗
10. Kalkman, Gino J. (2013), Functions of asyndetic clause relations in Biblical Hebrew. IPython Notebook. http://nbviewer.ipython.org/github/ETCBC/Biblical_Hebrew_Analysis/blob/ master/Miscellaneous/AsyndeticClauseFunctions.ipynb. ↗
11. Kalkman, Gino J. (to appear 2014), In search of a verbal system in Biblical Hebrew poetry; a computer-assisted analysis of syntactic patterns, Digital Humanities Quarterly, Alliance of Digital Humanities Organizations. find
12. Kalkman, Gino J. (to be published 2015), Verbal Forms in Biblical Hebrew Poetry: Poetical Freedom or Linguistic System?, PhD thesis, VU University, Amsterdam. http://nbviewer.ipython.org/github/ETCBC/Biblical_Hebrew_Analysis/blob/master/PhD/Introduction.ipynb. ↗
13. Kittel, Rudolf, editor (1968-1997), Biblia Hebraica Stuttgartensia, Deutsche Bibelgesellschaft, Stuttgart, Germany. http://www.bibelwissenschaft.de/startseite/ wissenschaftliche-bibelausgaben/biblia-hebraica/bhs/. ↗
14. Naaijer, Martijn (2012), The common nouns in the book of Esther. A new quantitative approach to the linguistic relationships of Biblical books, Master’s thesis, Radboud University, Nijmegen, Netherlands. find
15. Niccacci, Alviero (1990), The Syntax of the Verb in Classical Hebrew Prose, Vol. 86 of Journal for the Study of the Old Testament, Supplement Series, Sheffield Academic Press, Sheffield. ISBN 1-85075-226-5, http://books.google.nl/books?id=LdbsaZ7di5YC. ↗
16. Pérez, Fernando and Brian E. Granger (2007), IPython: a system for interactive scientific computing, Computing in Science and Engineering 9 (3), pp. 21–29, IEEE Computer Society. http:// ipython.org, ISSN: 1521-9615, DOI: 10.1109/MCSE.2007.53. ↗
17. Petersen, Ulrik (2002-2014), EMDROS. Text database engine for analyzed or annotated text. Open Source software. http://emdros.org. ↗
18. Petersen, Ulrik (2004), EMDROS - a text database engine for analyzed or annotated text, Proceedings of COLING 2004, p. 11901193. http://emdros.org/petersen-emdros-COLING-2004.pdf. ↗
19. Petersen, Ulrik (2006), Principles, Implementation Strategies, and Evaluation of a Corpus Query System, Vol. 4002, Springer, p. 215226. http://link.springer.com/chapter/10.1007% 2F11780885_21. ↗
20. Rendsburg, Gary (1990a), Diglossia in Ancient Hebrew, Vol. 72 of American Oriental Society, Eisenbrauns, New Haven. ISBN-13: 978-0940490727, http://books.google.nl/books?id= hRliAAAAMAAJ. ↗
21. Rendsburg, Gary (1990b), Linguistic Evidence for the Northern Origin of Selected Psalms, Vol. 43 of Society of Biblical Literature Monograph Series, Scholars Press, Atlanta. http://books.google.nl/books?id=xbiDQgAACAAJ. ↗
22. Roorda, Dirk (2013-2014), LAF-Fabric. Workbench for analysing LAF resources. Python software on Github. https://github.com/ETCBC/laf-fabric. ↗
23. Roorda, Dirk and van den Charles M.J.M. Heuvel (2012), Annotation as a new paradigm in research archiving, Proceedings of ASIS&T 2012 Annual Meeting. Final Papers, Panels and Posters. https://www.asis.org/asist2012/proceedings/Submissions/84.pdf (author’s version: http://annotation-paradigm.readthedocs.org/en/latest/_downloads/ ASIST2012-Annot-DR-ChvdH-final-submission.pdf). ↗ our card
24. Sangati, Federico, Willem Zuidema, and Rens Bod (2010), Efficiently extract recurring tree fragments from large treebanks, Proceedings of LREC, pp. 219–226. http://dare.uva.nl/record/371504. ↗
25. Scha, Remko (1990), Language theory and language technology; competence and performance, in de Kort, Q.A.M. and G.L.J. Leerdam, editors, Computertoepassingen in de Neerlandistiek, LVVN, Almere, the Netherlands, pp. 7–22. Original title: Taaltheorie en taaltechnologie; competence en performance. Translation available at http://iaaa.nl/rs/LeerdamE.html. ↗
26. Schneider, Wolfgang (1974), Grammatik des biblischen Hebräisch (’Grammar of Biblical Hebrew’), Claudius Verlag, München. ISBN-13: 978-3532711514, http://books.google.nl/books/?id= 2bLFnQEACAAJ. ↗
27. Skut, Wojciech, Brigitte Krenn, Thorsten Brants, and Hans Uszkoreit (1997), An annotation scheme for free word order languages, Proceedings of ANLP, pp. 88–95. find
28. Talstra, Eep and Constantijn J. Sikkel (2000), Genese und Kategorienentwicklung der WIVU-Datenbank (’Origin and category development of the WIVU database’), in Hardmeier, C. et al, editor, Ad Fontes! Quellen erfassen lesen deuten. Was ist Computerphilologie? Ansatzpunkte und Methodologie Instrument und Praxis, VU University Press, Amsterdam, Netherlands, pp. 33–68. find
29. Talstra, Eep, Constantijn J. Sikkel, Oliver Glanz, Reinoud Oosting, and Janet W. Dyk (2012), Text database of the Hebrew Bible. Dataset available online after permission of the depositor at Data Archiving and Networked services, Den Haag, Netherlands. http://www.persistent-identifier.nl/?identifier=urn:nbn:nl:ui:13-ukhm-eb. ↗
30. van Cranenburgh, Andreas (2014), Linear average time extraction of phrase-structure fragments, Computational Linguistics in the Netherlands Journal. ISSN: 2211-4009. find
31. van Cranenburgh, Andreas and Rens Bod (2013), Discontinuous parsing with an efficient and accurate DOP model, Proceedings of the International Conference on Parsing Technologies, Nara, Japan, 27–29 November. http://acl.cs.qc.edu/iwpt2013/proceedings/Splits/9_pdfsam_ IWPTproceedings.pdf. ↗
32. van Gompel, Maarten (2013), FoLiA: Format for Linguistic Annotation. Documentation, Technical Report 01, Radboud University Nijmegen. http://proycon.github.io/folia. ↗
33. van Peursen, Wido Th. and Dirk Roorda (2014), Hebrew Text Database in Linguistic Annotation Framework. Dataset available online at Data Archiving and Networked services, Den Haag, Netherlands. PID: http://www.persistent-identifier.nl/?identifier=urn:nbn:nl:ui: 13-048i-71. van Peursen, Wido Th. and Percy S.F. van Keulen, editors (2006), Corpus Linguistics and Textual History, Brill, Leiden, Netherlands. ISBN13: 9789023241942. van Peursen, Wido Th., Ernst D. Thoutenhoofd, and Adriaan H. van der Weel, editors (2010), Text Comparison and Digital Creativity. The Production of Presence and Meaning in Digital Text Scholarship, Brill, Leiden, Netherlands. DOI: 10.1163/ ej.9789004188655.i-328. ↗
34. Weinrich, Harald (1964), Tempus: Besprochene und erzählte Welt (’Tense: commentated and narrated world’), Vol. 16 of Sprache unde Literatur, Kohlhammer Verlag, Stuttgart. ISBN: 340647876X, http://books.google.nl/books/?id=MdYVTlzLeiUC. ↗
35. Young, Ian, Robert Rezetko, and Martin Ehrensvärd (2008), Linguistic Dating of Biblical Texts, Equinox Publishing, London. 2 vols. ISBN-13: 978-1845530815, http://books.google.nl/ books?id=-b9AAQAAIAAJ. ↗

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/hbo-2014-roorda-laf-fabric-data-analysis-tool/