Diacritic Annotation in the Arabic Treebank and its Impact on Parser Evaluation

What it is about

The Arabic Treebank is released in a vocalized and an unvocalized form of each file, and most parsing work used the unvocalized trees as "real-world" input. The authors show that this is not so. The unvocalized trees are the vocalized forms with short vowels stripped, not the source text with clitics split off, and the vocalized forms can carry normalizations that the source does not have. In part 3, section 3, 13,298 of 355,870 tokens (3.7%) differ from the source. One example: a source token with the preposition li and the article Al gets an alif in the treebank that the source lacks. The release therefore adds pointers from split tokens to source tokens, and a tree format whose leaves carry the source token, the unvocalized and vocalized forms, the lemma and the gloss. The paper argues that any parsing experiment must state which of these inputs it uses.

Ideas

• I1. A check in tools/hbo: for each gold token, the segments with their unpointed forms must concatenate to the source token. Count the mismatches on the gold set. The 3.7% figure says that a released "unpointed" form can be a hybrid.
• I2. Standoff pointers in the tools/hbo output: each CoNLL-U token keeps the index of its source token (in the MISC column), so segmentation, pointed and unpointed levels can be compared without realignment.
• I3. Report parser scores on source-form input and on pointed-form input as separate rows, never a mixture, both for tools/hbo and for trees from macula.

For Mova

I1 and I2 belong to tools/hbo, with the output format described in 13. Biblical Hebrew in Mova: the tools. I3 is a rule for every gparse report.

Topics: Treebanks

Cite

The paper itself, not this card. BibTeX from the library's record of the work.

@inproceedings{maamouri2008diacritic,
  title = {{Diacritic Annotation in the Arabic Treebank and its Impact on Parser Evaluation}},
  author = {Maamouri, Mohamed and Kulick, Seth and Bies, Ann},
  booktitle = {Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC'08)},
  year = {2008},
  url = {https://aclanthology.org/L08-1361/}
}

References

6 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Daniel M. Bikel. 2004. On the Parameter Space of Lexicalized Statistical Parsing Models. Ph.D. thesis, Department of Computer and Information Sciences, University of Pennsylvania. find
2. Tim Buckwalter. 2004. Arabic morphological analyzer version 2.0. LDC2004L02. Linguistic Data Consortium. find
3. Nizar Habash and Owen Rambow. 2005. Arabic tokenization, part-of-speech tagging and morphological disambiguation in one fell swoop. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL’05), pages 573–580, Ann Arbor, Michigan, June. Association for Computational Linguistics. find our card
4. Seth Kulick, Ryan Gabbard, and Mitchell Marcus. 2006. Parsing the Arabic Treebank: Analysis and improvements. In Proceedings of TLT 2006. Treebanks and Linguistic Theories. find
5. Mohamed Maamouri and Ann Bies. 2004. Developing an arabic treebank: Methods, guidelines, procedures, and tools. In Ali Farghaly and Karine Megerdoomian, editors, COLING 2004 Computational Approaches to Arabic Script-based Languages, pages 2–9, Geneva, Switzerland, August 28th. COLING. find our card
6. Mohamed Maamouri, Ann Bies, and Seth Kulick. 2006. Diacritization: A challenge to Arabic treebank annotation and parsing. In Proceedings of the British Computer Society Arabic NLP/MT Conference, London, UK, October. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/bank-2008-maamouri-diacritic-annotation-arabic-treebank-its/