A Transformer-based parser for Syriac morphology

What it is about

The authors train a Transformer sequence-to-sequence model from scratch to produce ETCBC-style morphological analyses of Syriac text, one verse per line, with each graphical unit split into words and morphemes. The Syriac data is small (5,596 verses), so they add Biblical Hebrew (22,946 verses from the Masoretic Text). They compare five training regimes: Syriac only; unvocalized or vocalized Hebrew mixed with Syriac; and Hebrew first (10 epochs) followed by Syriac. The best Syriac-only model reaches 89.3% (graphical units fully correct on the test set); the best overall model, trained on unvocalized Hebrew and Syriac together, reaches 90.8%. The Hebrew-trained models are consistently better, and the Syriac-only model makes two to three times more consonantal-text errors. Most errors are grammatically valid analyses that are wrong in their context. The test set is made of blocks of consecutive verses, so that no overlapping window appears in both training and test.

Ideas

• I1. Consonant check: the first error category (the predicted consonantal form differs from the surface text) needs no linguistics. Make it a validator on every tools/hbo output, with the input text as the gold consonants; it costs nothing and catches a whole error class.
• I2. Block split: build validation and test sets from blocks of consecutive verses, not from random windows, so that no window of the test set is in the training set. Use the same rule for the cv folds of any Hebrew or Aramaic tools/hbo test.
• I3. Ketiv/qere filter: drop the passages where the written consonants and the analysis disagree, as the paper does. Apply this when macula converts the Masoretic text to UD, so that the lex and pre tables of tools/hbo are learned only from consistent pairs.

For Mova

I1 is a validator in tools/hbo and is recorded in 13. Biblical Hebrew in Mova: the tools. I2 is a rule for the test split in tools/hbo. I3 goes into tools/macula (the conversion step). The neural model is a teacher-type comparison only; its data and its error categories are what Mova can use.

Cite

The paper itself, not this card. BibTeX from the library's record of the work.

@inproceedings{naaijer2023transformer,
  title = {{A Transformer-based parser for Syriac morphology}},
  author = {Naaijer, Martijn and Sikkel, Constantijn and Coeckelbergs, Mathias and Attema, Jisk and Peursen, Willem Th. Van},
  booktitle = {Proceedings of the Ancient Language Processing Workshop},
  year = {2023},
  url = {https://aclanthology.org/2023.alp-1.3/}
}

References

15 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Shihadeh Alqrainy and Muhammed Alawairdhi. 2020. Towards Developing a Comprehensive Tag Set for the Arabic Language. Journal of Intelligent Systems 30:287–296. https://doi.org/10.1515/jisys-2019-0256. ↗
2. Wissam Antoun, Fady Baly, and Hazem Hajj. 2021. AraBERT: Transformer-based Model for Arabic Language Understanding. Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, 9–15. https://aclanthology.org/2020.osact-1.2. ↗
3. Roy Bar-Haim, Khalil Sima'an, and Yoad Winter. 2008. Part-of-speech tagging of Modern Hebrew text. Natural Language Engineering 14(2):223– 251. https://doi.org/10.1017/S135132490700455X. ↗
4. Ezra Daya, Dan Roth, and Shuly Wintner. 2004. Learning Hebrew Roots: Machine Learnng with Linguistic Constraints. Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, 357–364. find
5. Emiliano Giovannetti, Davide Albanesi, Andrea Bellandi, Simone Marchi, and Alessandra Pecchioli. 2018. Constructing an Annotated Resource for Part-Of-Speech Tagging of Mishnaic Hebrew. Proceedings of the Fifth Italian Conference on Computational Linguistics CLiC-it 2018: 10–12 December 2018, Torino. Torino: Accademia University. https://doi.org/10.4000/books.aaccademia.3394. ↗ our card
6. Moshe Koppel and Avi Shmidman. 2020. Torah Study and the Digital Revolution, a Glimpse of the Future. https://thelehrhaus.com/commentary/torah-study-and-the-digital-revolution-a-glimpse-of-the-future. ↗
7. Sandra Kübler and Emad Mohamed. 2012. Part of speech tagging for Arabic. Natural Language Engineering 18(4):521–548. https://doi.org/10.1017/S1351324911000325. ↗
8. Gennadi Lembersky, Danny Shacham, and Shuly Wintner. 2012. Morphological disambiguation of Hebrew: A case study in classifier combination, Natural Language Engineering 20(1):69–97. https://doi.org/10.1017/S1351324912000216. ↗ our card
9. Amit Seker, Elron Bandel, Dan Bareket, Idan Brusilovsky, Refael Shaked Greenfeld, and Reut Tsarfaty. 2021. AlephBERT: A Hebrew Large Pre-Trained Language Model to Start-off your Hebrew NLP Application. https://arxiv.org/abs/2104.04052. ↗ our card
10. Avi Shmidman, Shaltiel Shmidman, Moshe Koppel, and Yoav Goldberg. 2020. Nakdan: Professional Hebrew Diacritizer. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, Association for Computational Linguistics 197–203. https://doi.org/10.18653/v1/2020.acl-demos.23. ↗ our card
11. Martha Y. Tachbelie, Solomon T. Abate, and Laurent Besacier. 2011. Part-of-Speech Tagging for Under-Resourced and Morphologically Rich Languages– The Case of Amharic. Conference on Human Language Technology for Development, Alexandria, Egypt, 2–5 May 2011. https://www.cle.org.pk/hltd/pdf/HLTD201109.pdf. ↗
12. Bas Ter Haar Romeny and Willem Th. Van Peursen (eds.). 1966–. The Old Testament in Syriac according to the Peshiṭta Version. Leiden: Brill. find
13. Sebastian Ruder. 2022. The State of Multilingual AI. https://www.ruder.io/state-of-multilingual-ai. ↗
14. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in Neural Information Processing Systems, 6000–6010. https://arxiv.org/pdf/1706.03762.pdf. ↗
15. Amir Zeldes. 2018. A Characterwise Windowed Approach to Hebrew Morphological Segmentation. Proceedings of the 15th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology, 101–110. http://dx.doi.org/10.18653/v1/W18-5811. ↗ our card

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2023-naaijer-transformer-based-parser-syriac-morphology/