Matt Gardner

4 paper cards in Mova's library · 2019–2020

Matt Gardner wrote 4 of the papers Mova reads, in 2019–2020; often with Pradeep Dasigi, Dheeru Dua, Nelson F. Liu.

The profile is the library's match of the name; two people with one name are not always told apart.

Ideas Mova took (1)

From the ideas ledger: an idea tried, kept or queued, or measured with no gain, and what it gave.

kept — Keep events in non-real modes out of «what happened» answers: negated frames (neg), conditional ones (cond) and the unrealised complements proposed in white-2016-universal-decompositional-semantics-universal-dependencies I1 do not answer «What did the lion… (TORQUE: A Reading Comprehension Dataset of Temporal… · an «if» clause is irrealis for the student (PRAG_OFF=cond-irrealis): 7702/2200→7704/2206 (35db4e3f7))

Evaluating Models' Local Decision Boundaries via Contrast Sets (2020) measured
Standard test sets come from the same distribution as the training data, so a dataset with systematic gaps or annotation artifacts lets models score well with shallow heuristics.
TORQUE: A Reading Comprehension Dataset of Temporal Ordering Questions (2020) 1 idea kept
TORQUE asks temporal relations as reading-comprehension questions instead of labelling event pairs with Allen-style relations.
DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs (2019)
DROP has 96,567 questions on about 7,000 Wikipedia passages rich in numbers (American-football game summaries, history), written adversarially against a reading model so that answering needs discrete operations over several places in the text.
Quoref: A Reading Comprehension Dataset with Questions Requiring Coreferential Reasoning (2019)
Quoref has over 24K span-answer questions on 4.7K Wikipedia paragraphs (about 40% film plot summaries, which mention many characters), written by crowd workers so that answering needs linking two mentions of one entity.