English Word Sense Disambiguation in 2026: When the Labels Become the Bottleneck

What it is about

The paper argues that on English all-words WSD the gold labels now limit the measured ranking, because frontier LLMs are close to the labels. The authors build lexEN-v1, a correction layer over the Maru et al. 2022 ALL_NEW benchmark (211 labels changed, 56 removed), and an evaluation harness with 57 models and 192 runs. On lexEN-v1 the best model reaches 95.6%, while the classic supervised systems sit near 80 to 85%. Fleiss kappa among three reviewers is 0.537 on the hard items, and coarser sense inventories raise agreement and accuracy together. Relabeling SemCor with two LLMs changes 21.30% (GPT-5.5) and 21.85% (Gemma) of the gold keys, and retraining the classic systems on it adds several F1 points on untouched test sets.

Ideas

• I1. onto/src/wsd.rs: keep a lineage record for every sense label that annot or a rule changes (who or what changed it, the evidence, the date). Measure how many labels in a silver set change after a review pass, before any model is tuned on them.
• I2. Coarse check in onto: score the deterministic WSD at WordNet lexfile (supersense) level next to the fine sense level on the same test items. The paper reports coarsening lifts agreement and accuracy together; check whether the same holds for the deterministic system.
• I3. Relabeling pilot in annot: relabel a small sample of en silver senses with a large model offline and measure the disagreement rate by part of speech. The paper finds verbs disagree most (about 30%), so verbs should be reviewed first.

For Mova

The touch points are onto/src/wsd.rs and the annot teacher side, which may relabel offline but never answers at test time. lexEN and SenseBench are research or test data with unclear licences, so use them for tests only. Measure first the coarse-versus-fine gap of the deterministic WSD on a held-out sample, with a second identical run to rule out noise.

Topics: Word sense disambiguation

Cite

The paper itself, not this card. BibTeX from the library's record of the work. arXiv 2609.17554

@misc{philippov2026english,
  title = {{English Word Sense Disambiguation in 2026: When the Labels Become the Bottleneck}},
  author = {Philippov, Vassili and Salman, Amro and Andreev, Dmitrii and Hands, Penny},
  year = {2026},
  eprint = {2609.17554},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2609.17554}
}

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/lex-2026-philippov-english-wsd-labels/