Papers of the day   All papers

Evaluating NLP Models via Contrast Sets


Zachary Lipton: Before the media blitz & retweet party get out of control, this idea exists, has been published, has a name, and a clearer justification. It is called ***Counterfactually-Augmented Data*** and here's the published paper (spotlight at #ICLR2020).

9 replies, 364 likes

Matt Gardner: Evaluating NLP Models via Contrast Sets New work that is a collaboration between 26 people at 10 institutions (!) Trying to tag everyone at the top of the thread, here it goes:

11 replies, 358 likes

Noah Smith: new work by @nlpmattg of @ai2_allennlp, with a cast of dozens: contrast sets

0 replies, 34 likes

John Platt: Adding local perturbations to NLP test sets highlights fragility of some newer models.

0 replies, 6 likes

lazary: @jxmorris12 Looks really interesting! It reminds me of the recent "minimal pair" literature, that aims to perform minimal changes to examples that *do* change the meaning, followed by an evaluation. by @dkaushik96 et al. by @nlpmattg et al.

1 replies, 1 likes


Found on Apr 07 2020 at

PDF content of a computer science paper: Evaluating NLP Models via Contrast Sets