Camel Treebank: An Open Multi-genre Arabic Dependency Treebank

What it is about

Camel Treebank 1.0 is an open dependency treebank of about 188k words (about 242k tokens) of Modern Standard and Classical Arabic, in the Columbia Arabic Treebank (CATiB) style. It has 13 sub-corpora, from pre-Islamic odes and the Quran through novels and news to student essays and online comments, chosen so that the texts are out of copyright or openly licensed. Tokenisation, tagging and first parsing are automatic (CamelTools, then the MaltParser model of CamelParser), and four trained annotators correct them in the PALMYRA tool. The authors also align gold and predicted tokens by character-level edit distance, so that inserted and deleted tokens can be scored. Before manual correction, the parser's labelled attachment score averages 69.1% and ranges from 56.1% (Odes) to 79.7% (WikiNews). The paper also notes how often elided tokens occur, about 24 in 242k tokens, mostly in poetry.

Ideas

• I1. Score per genre, not on average: the spread from 56 to 80 labelled attachment points across genres is larger than most differences between parsers. For Mova's parser evaluation, report the score per sub-corpus and per genre, and treat a drop in one genre as a failure even when the mean rises.
• I2. Token-aligned scoring when the tokenisation differs: align the gold and predicted tokens by edit distance, count inserted and deleted tokens as null tokens, and score POS and arcs only on aligned tokens. This suits the Hebrew segmenter in tools/hbo, whose splits differ from gold splits in the MACULA and PTNK trees.
• I3. Verse and sentence boundaries as hard rules: segment on sentence-final marks, merge or split only by written guidelines, never split inside a dependent clause, and keep verse boundaries in scripture (the paper keeps Quran, Bible and poetry lines whole). For raw Biblical and Mishnaic text in tools/langseg and tools/hbo, a verse is a boundary the segmenter must not cross.
• I4. Explicit empty tokens for ellipsis: allow an empty node, marked with a star, when a verb or noun is elided, instead of re-attaching its children to the parent. The paper's need for this comes from poetic texts, which is also the case for Biblical poetry in treebanks/hbo/ptnk.

For Mova

I1 is an evaluation rule for treebanks/ar and for every treebank Mova scores. I2 belongs in tools/hbo (scoring the segmenter against gold splits). I3 goes into tools/langseg and tools/hbo for raw scripture. I4 is a convention for treebanks/hbo/ptnk in poetic books. Camel Treebank itself is not carried into any release (licence: summary only).

Topics: Treebanks

Cite

The paper itself, not this card. BibTeX from the library's record of the work.

@inproceedings{habash2022camel,
  title = {{Camel Treebank: An Open Multi-genre Arabic Dependency Treebank}},
  author = {Habash, Nizar and AbuOdeh, Muhammed and Taji, Dima and Faraj, Reem and Gizuli, Jamila El and Kallas, Omar},
  booktitle = {Proceedings of the Thirteenth Language Resources and Evaluation Conference},
  year = {2022},
  url = {https://aclanthology.org/2022.lrec-1.286/}
}

References

47 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Abdelali, A., Darwish, K., Durrani, N., and Mubarak, H. (2016). Farasa: A Fast and Furious Segmenter for Arabic. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), pages 11–16, San Diego, California. find our card
2. Al-Ghamdi, S., Al-Khalifa, H., and Al-Salman, A. (2021). A dependency treebank for classical Arabic poetry. In Proceedings of the Sixth International Conference on Dependency Linguistics (Depling, SyntaxFest 2021), pages 1–9, Sofia, Bulgaria, December. find
3. Alfaifi, A. Y. G. (2015). Building the Arabic Learner Corpus and a System for Arabic Error Annotation. Ph.D. thesis, University of Leeds. find
4. Alkuhlani, S., Habash, N., and Roth, R. (2013). Automatic morphological enrichment of a morphologically underspecified treebank. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), pages 460–470, Atlanta, Georgia. find
5. Belkebir, R. and Habash, N. (2021). Automatic error type annotation for Arabic. In Proceedings of the 25th Conference on Computational Natural Language Learning, pages 596–606, Online, November. find
6. Buchholz, S. and Marsi, E. (2006). CoNLL-X shared task on multilingual dependency parsing. In Proceedings of the Tenth Conference on Computational Natural Language Learning (CoNLL-X), pages 149–164, New York City, June. find our card
7. Diab, M., Habash, N., Rambow, O., and Roth, R. (2013). LDC Arabic treebanks and associated corpora: Data divisions manual. arXiv preprint arXiv:1309.5652. ↗
8. Dukes, K. and Buckwalter, T. (2010). A Dependency Treebank of the Quran using Traditional Arabic Grammar. In Proceedings of the Conference on Informatics and Systems (INFOS), Cairo, Egypt. find
9. Dukes, K. and Habash, N. (2010). Morphological Annotation of Quranic Arabic. In Proceedings of the Language Resources and Evaluation Conference (LREC), Valetta, Malta. find our card
10. Gerdes, K., Guillaume, B., Kahane, S., and Perrier, G. (2021). Starting a new treebank? Go SUD! Theoretical and practical benefits of the Surface-Syntactic distributional approach. In Sixth International Conference on Dependency Linguistics (Depling, SyntaxFest 2021), Sofia, Bulgaria. find
11. Habash, N. and Palfreyman, D. (2022). ZAEBUC: An annotated Arabic-English bilingual writer corpus. In Proceedings of the Language Resources and Evaluation Conference (LREC), Marseille, France. find
12. Habash, N. and Roth, R. (2009). CATiB: The Columbia Arabic Treebank. In Proceedings of the Joint Conference of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing (ACL-IJCNLP), pages 221–224, Suntec, Singapore. find our card
13. Habash, N., Soudi, A., and Buckwalter, T. (2007). On Arabic Transliteration. In A. van den Bosch et al., editors, Arabic Computational Morphology: Knowledge-based and Empirical Methods, pages 15–22. Springer, Netherlands. find
14. Habash, N., Faraj, R., and Roth, R. (2009). Syntactic Annotation in the Columbia Arabic Treebank. In Proceedings of the International Conference on Arabic Language Resources and Tools, Cairo, Egypt. find
15. Habash, N. Y. (2010). Introduction to Arabic natural language processing, volume 3. Morgan & Claypool Publishers. find
16. Halabi, D., Fayyoumi, E., and Awajan, A. (2021). I3rab: A new Arabic dependency treebank based on Arabic grammatical theory. Transactions on Asian and Low-Resource Language Information Processing, 21(2):1– 32. find our card
17. Inoue, G., Alhafni, B., Baimukan, N., Bouamor, H., and Habash, N. (2021). The interplay of variant, size, and task type in Arabic pre-trained language models. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, Kyiv, Ukraine (Virtual), April. find
18. Khalifa, S., Obeid, O., and Habash, N. (2021). Character Edit Distance Based Word Alignment. https://github.com/CAMeL-Lab/ced_word_alignment. ↗
19. Maamouri, M., Bies, A., Buckwalter, T., and Mekki, W. (2004). The Penn Arabic Treebank: Building a Large-Scale Annotated Arabic Corpus. In Proceedings of the International Conference on Arabic Language Resources and Tools, pages 102–109, Cairo, Egypt. find
20. Maamouri, M., Bies, A., Buckwalter, T., Diab, M., Habash, N., Rambow, O., and Tabessi, D. (2006). Developing and Using a Pilot Dialectal Arabic Treebank. In Proceedings of the Language Resources and Evaluation Conference (LREC), Genoa, Italy. find
21. Maamouri, M., Graff, D., Bouziri, B., Krouna, S., Bies, A., and Kulick, S. (2010). Ldc standard Arabic morphological analyzer (sama) version 3.1. find
22. Maamouri, M., Zaghouani, W., Cavalli-Sforza, V., Graff, D., and Ciul, M. (2012). Developing aret: an nlp-based educational tool set for Arabic reading enhancement. In Proceedings of the Workshop on Building Educational Applications using NLP (BEA), pages 127–135. find
23. More, A., Seker, A., Basmova, V., and Tsarfaty, R. (2019). Joint transition-based models for morphosyntactic parsing: Parsing strategies for MRLs and a case study from Modern Hebrew. Transactions of the Association for Computational Linguistics, 7:33–48. find
24. Müller-Eberstein, M., van der Goot, R., and Plank, B. (2021). Genre as weak supervision for cross-lingual dependency parsing. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 4786–4802, Online and Punta Cana, Dominican Republic, November. find
25. Nivre, J., Hall, J., and Nilsson, J. (2006). MaltParser: A data-driven parser-generator for dependency parsing. In Proceedings of the Language Resources and Evaluation Conference (LREC), pages 2216–2219, Genoa, Italy. find
26. Obeid, O., Zalmout, N., Khalifa, S., Taji, D., Oudah, M., Alhafni, B., Inoue, G., Eryani, F., Erdmann, A., and Habash, N. (2020). CAMeL tools: An open source python toolkit for Arabic natural language processing. In Proceedings of The 12th Language Resources and Evaluation Conference, pages 7022–7032, Marseille, France, May. find
27. Pasha, A., Al-Badrashiny, M., Diab, M., Kholy, A. E., Eskander, R., Habash, N., Pooleery, M., Rambow, O., and Roth, R. (2014). Madamira: A fast, comprehensive tool for morphological analysis and disambiguation of Arabic. In Proceedings of the Language Resources and Evaluation Conference (LREC), pages 1094–1101, Reykjavik, Iceland. find our card
28. Shahrour, A., Khalifa, S., Taji, D., and Habash, N. (2016). CamelParser: A system for Arabic syntactic analysis and morphological disambiguation. In Proceedings of the International Conference on Computational Linguistics (COLING), pages 228–232. find our card
29. Shao, Y., Hardmeier, C., and Nivre, J. (2018). Universal word segmentation: Implementation and interpretation. Transactions of the Association for Computational Linguistics, 6:421–435. find
30. Smrž, O., Šnaidauf, J., and Zemánek, P. (2002). Prague Dependency Treebank for Arabic: Multi-Level Annotation of Arabic Corpus. In Proceedings of the International Symposium on Processing of Arabic, pages 147–155, Manouba, Tunisia. find
31. Taji, D. and Habash, N. (2020). PALMYRA 2.0: A configurable multilingual platform independent tool for morphology and syntax annotation. In Proceedings of the Fourth Workshop on Universal Dependencies (UDW 2020), pages 168–177, Barcelona, Spain (Online), December. find
32. Taji, D., Habash, N., and Zeman, D. (2017). Universal dependencies for Arabic. In Proceedings of the Workshop for Arabic Natural Language Processing (WANLP), Valencia, Spain. find our card
33. Taji, D., El Gizuli, J., and Habash, N. (2018). An Arabic dependency treebank in the travel domain. In Proceedings of the Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT), Miyazaki, Japan. find our card
34. Zaghouani, W., Mohit, B., Habash, N., Obeid, O., Tomeh, N., Rozovskaya, A., Farra, N., Alkuhlani, S., and Oflazer, K. (2014). Large Scale Arabic Error Annotation: Guidelines and Framework. In Proceedings of the Language Resources and Evaluation Conference (LREC), Reykjavik, Iceland. 10. find
35. Language Resource References Abdelali, A., Darwish, K., Durrani, N., and Mubarak, H. (2016). Farasa: A Fast and Furious Segmenter for Arabic. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), pages 11–16, San Diego, California. find our card
36. Al-Akkad, A. M. (1938). Sarah. Hindawi. al Bukhari, I. M. (846). Sahih al-Bukhari. Dar Ibn Khathir. find
37. Alfaifi, A. Y. G. (2015). Building the Arabic Learner Corpus and a System for Arabic Error Annotation. Ph.D. thesis, University of Leeds. find
38. Altammami, S., Atwell, E., and Alsalka, A. (2019). The arabic–english parallel corpus of authentic hadith. International Journal on Islamic Applications in Computer Science And Technology-IJASAT. find
39. Dukes, K., Atwell, E., and Habash, N. (2013). Supervised collaboration for syntactic annotation of quranic arabic. Language resources and evaluation, 47(1):33– 62. find
40. Eck, M. and Hori, C. (2005). Overview of the IWSLT 2005 Evaluation Campaign. In Proceedings of the International Workshop on Spoken Language Translation (IWSLT). find
41. Habash, N. and Palfreyman, D. (2022). ZAEBUC: An annotated Arabic-English bilingual writer corpus. In Proceedings of the Language Resources and Evaluation Conference (LREC), Marseille, France. find
42. Mohit, B., Rozovskaya, A., Habash, N., Zaghouani, W., and Obeid, O. (2014). The first QALB shared task on automatic text correction for Arabic. In Proceedings of the Workshop for Arabic Natural Language Processing (WANLP), pages 39–47, Doha, Qatar. find
43. Smith, E. and Van Dyck, C. (1860). New Testament (Arabic Translation). find
44. Smith, E. and Van Dyck, C. (1865). Old Testament (Arabic Translation). find
45. Taji, D., El Gizuli, J., and Habash, N. (2018). An Arabic dependency treebank in the travel domain. In Proceedings of the Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT), Miyazaki, Japan. find our card
46. Takezawa, T., Kikui, G., Mizushima, M., and Sumita, E. (2007). Multilingual Spoken Language Corpus Development for Communication Research. Computational Linguistics and Chinese Language Processing, 12(3):303–324. Tufail, I. (1150). Hayy ibn Yaqdhan. Hindawi. Unknown. (12th century). One Thousand and One Nights. find
47. Zaghouani, W., Mohit, B., Habash, N., Obeid, O., Tomeh, N., Rozovskaya, A., Farra, N., Alkuhlani, S., and Oflazer, K. (2014). Large Scale Arabic Error Annotation: Guidelines and Framework. In Proceedings of the Language Resources and Evaluation Conference (LREC), Reykjavik, Iceland. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/lingbulk-2022-habash-camel-treebank-open-multi-genre/