Adversarial Examples for Evaluating Reading Comprehension Systems

What it is about

To test whether SQuAD readers understand or only match words, the authors add one distracting sentence to the paragraph that shares many words with the question but does not answer it. AddSent builds it by changing the question's content words (nouns and adjectives to nearby words, named entities and numbers to others), turning it into a statement with a fake answer of the right type, and having crowd workers fix grammar; the worst of up to five such sentences is used. AddOneSent adds a random approved one (model-independent); AddAny appends ten arbitrary words chosen by search. Average F1 of sixteen published models falls from 75% to 36% under AddSent, and to 6.7% under AddAny for four models, while humans lose only 3.4 points under AddOneSent. In 96.6% of failures the model answers from the added sentence; models survive more when the question shares a long n-gram with the passage. Training on generated distractors protects against the same generator but not against a variant that uses other fake answers and puts the sentence first.

Ideas

• I1. An AddSent-style distractor gate for the SLM student, fully deterministic: for a course question, take the lesson sentence that answers it, replace the entity in the questioned role by another entity of the same type from the lesson (or a fixed list), change one other content word of the question (verb or a modifier), and insert the sentence into the lesson. The right answer must not change. Because the student matches roles in frames, it should be far more stable than word matchers; measure the share of answers that stay right, per course.
• I2. Run the distractor both appended and prepended, as AddSentMod does: a difference shows a position or recency bias in how the student picks among supporting frames (salience, last-mentioned), which should be explainable by the trace or removed.
• I3. Use the failures as a list of where role matching is too loose: a distractor accepted means a frame that differs from the question in one role still counted as support. This feeds item 1 of docs/mmm-diagnosis.md «Next» (penalise unsupported or contradicting content) with concrete cases.

For Mova

30. Pragmatics and discourse: overview §9 (measurements). Code: world/src/knowledge.rs (support scoring), world/src/report.rs. Measures: share of answers unchanged under distractors (append/prepend), per course; course tests unchanged without distractors (the gate must not cost accuracy).

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/d17-1215 · arXiv 1707.07328

@inproceedings{jia2017adversarial,
  title = {{Adversarial Examples for Evaluating Reading Comprehension Systems}},
  author = {Jia, Robin and Liang, Percy},
  booktitle = {Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing},
  year = {2017},
  doi = {10.18653/v1/d17-1215},
  eprint = {1707.07328},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/d17-1215}
}

References

35 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. E. M. Bender, H. Daume III, A. Ettinger, H. Kannan, S. Rao, and E. Rothschild. 2017. Build it, break it: The language edition. https://bibinlp. umiacs.umd.edu/. find
2. S. R. Bowman, L. Vilnis, O. Vinyals, A. M. Dai, R. Jozefowicz, and S. Bengio. 2016. Generating sentences from a continuous space. In Computational Natural Language Learning (CoNLL). find
3. N. Dalvi, P. Domingos, Mausam, S. Sanghai, and D. Verma. 2004. Adversarial classification. In International Conference on Knowledge Discovery and Data Mining (KDD). find
4. C. Fellbaum. 1998. WordNet: An Electronic Lexical Database. MIT Press. find
5. W. N. Francis and H. Kucera. 1979. Brown Corpus Manual. find
6. A. Globerson and S. Roweis. 2006. Nightmare at test time: robust learning by feature deletion. In International Conference on Machine Learning (ICML), pages 353–360. find
7. Y. Gong and S. R. Bowman. 2017. Ruminating reader: Reasoning with gated multi-hop attention. arXiv. find
8. I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. 2014. Generative adversarial nets. In Advances in Neural Information Processing Systems (NIPS). find
9. I. J. Goodfellow, J. Shlens, and C. Szegedy. 2015. Explaining and harnessing adversarial examples. In International Conference on Learning Representations (ICLR). find
10. M. Hu, Y. Peng, and X. Qiu. 2017. Mnemonic reader for machine comprehension. arXiv. find
11. R. Jia and P. Liang. 2016. Data recombination for neural semantic parsing. In Association for Computational Linguistics (ACL). find
12. K. Lee, S. Salant, T. Kwiatkowski, A. Parikh, D. Das, and J. Berant. 2017. Learning recurrent span representations for extractive question answering. arXiv. find
13. H. J. Levesque. 2013. On our best behaviour. In International Joint Conference on Artificial Intelligence (IJCAI). find
14. C. D. Manning, M. Surdeanu, J. Bauer, J. Finkel, S. J. Bethard, and D. McClosky. 2014. The stanford coreNLP natural language processing toolkit. In ACL system demonstrations. find
15. M. Marcus, B. Santorini, M. A. Marcinkiewicz, and A. Taylor. 1999. Treebank-3. find
16. S. Moosavi-Dezfooli, A. Fawzi, O. Fawzi, and P. Frossard. 2017. Universal adversarial perturbations. In Computer Vision and Pattern Recognition (CVPR). find
17. N. Narodytska and S. P. Kasiviswanathan. 2016. Simple black-box adversarial perturbations for deep networks. arXiv preprint arXiv:1612.06299. — [arXiv:1612.06299](https://arxiv.org/abs/1612.06299) find
18. D. Paperno, G. Kruszewski, A. Lazaridou, Q. N. Pham, R. Bernardi, S. Pezzelle, M. Baroni, G. Boleda, and R. Fernandez. 2016. The LAMBADA dataset: Word prediction requiring a broad discourse context. In Association for Computational Linguistics (ACL). find
19. N. Papernot, P. McDaniel, I. Goodfellow, S. Jha, Z. Celik, and A. Swami. 2017. Practical black-box attacks against deep learning systems using adversarial examples. In Proceedings of the ACM Asia Conference on Computer and Communications Security. find
20. J. Pennington, R. Socher, and C. D. Manning. 2014. Glove: Global vectors for word representation. In Empirical Methods in Natural Language Processing (EMNLP). find
21. P. Rajpurkar, J. Zhang, K. Lopyrev, and P. Liang. 2016. Squad: 100,000+ questions for machine comprehension of text. In Empirical Methods in Natural Language Processing (EMNLP). find
22. L. Rimell, S. Clark, and M. Steedman. 2009. Unbounded dependency recovery for parser evaluation. In Empirical Methods in Natural Language Processing (EMNLP). find our card
23. M. Seo, A. Kembhavi, A. Farhadi, and H. Hajishirzi. 2016. Bidirectional attention flow for machine comprehension. arXiv. find
24. J. Li, W. Monroe, T. Shi, A. Ritter, and D. Jurafsky. 2017. Adversarial learning for neural dialogue generation. arXiv preprint arXiv:1701.06547. find
25. Y. Shen, P. Huang, J. Gao, and W. Chen. 2017. Reasonet: Learning to stop reading in machine comprehension. In International Conference on Knowledge Discovery and Data Mining (KDD). find
26. R. Liu, J. Hu, W. Wei, Z. Yang, and E. Nyberg. 2017. Structural embedding of syntactic trees for machine comprehension. arXiv. find
27. N. A. Smith. 2012. Adversarial evaluation for models of natural language. arXiv preprint arXiv:1207.0245. — [arXiv:1207.0245](https://arxiv.org/abs/1207.0245) find
28. D. Lowd and C. Meek. 2005. Adversarial learning. In International Conference on Knowledge Discovery and Data Mining (KDD). find
29. C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus. 2014. Intriguing properties of neural networks. In International Conference on Learning Representations (ICLR). find
30. N. Madnani and B. J. Dorr. 2010. Generating phrasal and sentential paraphrases: A survey of data-driven methods. Computational Linguistics, 36(3):341– 387. find our card
31. S. Wang and J. Jiang. 2016. Machine comprehension using match-LSTM and answer pointer. arXiv preprint arXiv:1608.07905. find
32. Z. Wang, H. Mi, W. Hamza, and R. Florian. 2016. Multi-perspective context matching for machine comprehension. arXiv. find
33. D. Weissenborn, G. Wiese, and L. Seiffe. 2017. Making neural qa as simple as possible but not simpler. arXiv. find
34. Y. Yu, W. Zhang, K. Hasan, M. Yu, B. Xiang, and B. Zhou. 2016. End-to-end answer chunk extraction and ranking for reading comprehension. arXiv. find
35. J. Zhang, X. Zhu, Q. Chen, L. Dai, S. Wei, and H. Jiang. 2017. Exploring question understanding and adaptation in neural-network-based question answering. arXiv. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/jia-2017-adversarial-examples-evaluating-reading-comprehension/