SVM Model Tampering and Anchored Learning: A Case Study in Hebrew NP Chunking
What it is about
The paper studies an SVM chunker for Hebrew Simple NPs (about 4,500 training sentences, 500 test, 27K chunks) and opens the trained model in two ways. Model tampering zeroes features in the support vectors after training; keeping only the top 500 of about 30,000 lexical features gives F 93.60 against 93.58 for all of them. Keeping only the top 10 drops F to 78.32 with none and to 90.21 with ten. Lexical features matter most for prepositions, punctuation and the conjunction; features for open classes add little. Anchored learning gives every training sample its own feature, so the SVM can memorise hard cases; the heaviest anchors point at corpus errors. Of about 120 flagged locations, more than half were real errors, and 77 of 99 further similar locations were errors too. Punctuation normalisation adds about 1 F point, and a 10-fold run on the noisy tagger output reaches F 92.2.
Ideas
- I1. Anchor-weight mining as a corpus audit for Hebrew: train with one unique anchor per sentence, rank the locations by anchor weight, and review the top ones by hand. Record the yield of real errors per reviewed location.
- I2. Keep lexical features only for the closed classes (prepositions, the conjunction "and", punctuation) in any rule-based Hebrew NP or chunk rule, and test by removing the open-class lexicon on noisy tagging. The paper reports that such a small lexical set is enough on gold tags and collapses on noisy tags, so the test must use noisy tags.
- I3. Normalise punctuation inside gold NP spans before scoring, and report the score with and without the normalisation.
For Mova
I1 is an audit step for treebanks/he/ and treebanks/hbo/ptnk, run after the tree conversion. I2 belongs to the preposition and conjunction rules in Prepositions, prefixes, conjunctions and particles and The verb (the wa- conjunction). I3 goes into the scorer of tools/macula compare. The "et" marker and the "shel" attachment cases in the paper are open questions for 7. Syntax.
Cite
@inproceedings{goldberg2007svm,
title = {{SVM Model Tampering and Anchored Learning: A Case Study in Hebrew NP Chunking}},
author = {Goldberg, Yoav and Elhadad, Michael},
booktitle = {Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics},
year = {2007},
url = {https://aclanthology.org/P07-1029/}
}References
- S. Abney, R. Schapire, and Y. Singer. 1999. Boosting applied to tagging and PP attachment. EMNLP-1999. find
- M. Adler and M. Elhadad. 2006. An unsupervised morpheme-based hmm for hebrew morphological disambiguation. In COLING/ACL2006. find our card
- C. Cardie and D. Pierce. 1998. Error-driven pruning of treebank grammars for base noun phrase identification. In ACL-1998. find
- Y. Goldberg, M. Adler, and M. Elhadad. 2006. Noun phrase chunking in hebrew: Influence of lexical and morphological features. In COLING/ACL2006. find our card
- T. Kudo and Y. Matsumoto. 2000. Use of support vector learning for chunk identification. In CoNLL-2000. find
- T. Kudo and Y. Matsumoto. 2003. Fast methods for kernel-based text analysis. In ACL-2003. find
- M. Marcus and L. Ramshaw. 1995. Text Chunking Using Transformation-Based Learning. In Proc. of the 3rd ACL Workshop on Very Large Corpora. find
- T. Nakagawa and Y. Matsumoto. 2002. Detecting errors in corpora using support vector machines. In COLING-2002. find
- Erik F. Tjong Kim Sang and S. Buchholz. 2000. Introduction to the conll-2000 shared task: chunking. In CoNLL-2000. find
- K. Sima’an, A. Itai, Y. Winter, A. Altman, and N. Nativ. 2001. Building a tree-bank of modern hebrew text. Traitement Automatique des Langues, 42(2). find
- V. Vapnik. 1995. The nature of statistical learning theory. Springer-Verlag New York, Inc. find
This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2007-goldberg-svm-model-tampering-anchored-learning/