Annotation Artifacts in Natural Language Inference Data

Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel R. Bowman, Noah A. Smith

Type: Preprint

Publication Date: 2018-03-06

Citations: 106

Locations

arXiv (Cornell University) - View

Similar Works

Action	Title	Year	Authors
+	Annotation Artifacts in Natural Language Inference Data	2018	Suchin Gururangan Swabha Swayamdipta Omer Levy Roy Schwartz Samuel R. Bowman Noah A. Smith
+	Asking Crowdworkers to Write Entailment Examples: The Best of Bad Options	2020	Clara Vania Ruijie Chen Samuel R. Bowman
+ PDF Chat	Asking Crowdworkers to Write Entailment Examples: The Best of Bad Options	2020	Clara Vania Ruijie Chen Samuel R. Bowman
+ PDF Chat	Hypothesis-only Biases in Large Language Model-Elicited Natural Language Inference	2024	Grace Proebsting Adam Poliak
+	Hypothesis Only Baselines in Natural Language Inference	2018	Adam Poliak Jason Naradowsky Aparajita Haldar Rachel Rudinger Benjamin Van Durme
+	Hypothesis Only Baselines in Natural Language Inference	2018	Adam Poliak Jason Naradowsky Aparajita Haldar Rachel Rudinger Benjamin Van Durme
+	Hypothesis Only Baselines in Natural Language Inference	2018	Adam Poliak Jason Naradowsky Aparajita Haldar Rachel Rudinger Benjamin Van Durme
+	Investigating Reasons for Disagreement in Natural Language Inference	2022	Nanjiang Jiang Marie-Catherine de Marneffe
+	Not another Negation Benchmark: The NaN-NLI Test Suite for Sub-clausal Negation	2022	Hung Thinh Truong Yulia Otmakhova Timothy Baldwin Trevor Cohn Karin Verspoor Jey Han Lau
+	Uncertain Natural Language Inference	2020	Tongfei Chen Zhengping Jiang Adam Poliak Keisuke Sakaguchi Benjamin Van Durme
+ PDF Chat	Investigating Reasons for Disagreement in Natural Language Inference	2022	Nanjiang Jiang Marie-Catherine de Marneffe
+	Annotation Artifacts in Natural Language Inference Data	2018	Suchin Gururangan Swabha Swayamdipta Omer Levy Roy Schwartz Samuel R. Bowman Noah A. Smith
+	Without a 'doubt'? Unsupervised discovery of downward-entailing operators	2009	Cristian Danescu-Niculescu-Mizil Lillian Lee Richard Ducott
+	Uncertain Natural Language Inference	2019	Tongfei Chen Zhengping Jiang Adam Poliak Keisuke Sakaguchi Benjamin Van Durme
+	Collecting Diverse Natural Language Inference Problems for Sentence Representation Evaluation	2018	Adam Poliak Aparajita Haldar Rachel Rudinger Jingyue Hu Ellie Pavlick Aaron Steven White Benjamin Van Durme
+	A Linguistic Investigation of Machine Learning based Contradiction Detection Models: An Empirical Analysis and Future Perspectives	2022	Maren Pielka Felix Rode Lisa Pucknat Tobias Deußer Rafet Sifa
+ PDF Chat	A Linguistic Investigation of Machine Learning based Contradiction Detection Models: An Empirical Analysis and Future Perspectives	2022	Maren Pielka Felix Rode Lisa Pucknat Tobias Deußer Rafet Sifa
+	PECO: Examining Single Sentence Label Leakage in Natural Language Inference Datasets through Progressive Evaluation of Cluster Outliers	2021	Michael Saxon Xinyi Wang Wenda Xu William Yang Wang
+	PECO: Examining Single Sentence Label Leakage in Natural Language Inference Datasets through Progressive Evaluation of Cluster Outliers	2023	Michael Saxon Xinyi Wang Wenda Xu William Yang Wang
+	No Strong Feelings One Way or Another: Re-operationalizing Neutrality in Natural Language Inference	2023	Animesh Nighojkar Antonio Laverghetta John Licato

Works That Cite This (93)

Action	Title	Year	Authors
+ PDF Chat	Data and its (dis)contents: A survey of dataset development and use in machine learning research	2021	Amandalynne Paullada Inioluwa Deborah Raji Emily M. Bender Emily Denton Alex Hanna
+ PDF Chat	Jointly learning sentence embeddings and syntax with unsupervised Tree-LSTMs	2019	Jean Maillard Stephen Clark Dani Yogatama
+	Teaching Pre-Trained Models to Systematically Reason Over Implicit Knowledge.	2020	Alon Talmor Oyvind Tafjord Peter E. Clark Yoav Goldberg Jonathan Berant
+	Do Neural Dialog Systems Use the Conversation History Effectively? An Empirical Study	2019	Chinnadhurai Sankar Sandeep Subramanian Christopher Pal Sarath Chandar Yoshua Bengio
+	Explaining Question Answering Models through Text Generation	2020	Veronica Latcinnik Jonathan Berant
+ PDF Chat	ReClor: A Reading Comprehension Dataset Requiring Logical Reasoning	2020	Weihao Yu Zihang Jiang Yanfei Dong Jiashi Feng
+	Variational Information Bottleneck for Effective Low-Resource Fine-Tuning	2021	Rabeeh Karimi Mahabadi Yonatan Belinkov James Henderson
+	Explaining and Improving Model Behavior with k Nearest Neighbor Representations	2020	Nazneen Fatema Rajani Ben Krause Wenpeng Yin Tong Niu Richard Socher Caiming Xiong
+	The Importance of Being Recurrent for Modeling Hierarchical Structure	2018	Ke Tran Arianna Bisazza Christof Monz
+	Visual Entailment Task for Visually-Grounded Language Learning	2019	Ning Xie Farley Lai Derek Doran Asim Kadav

Works Cited by This (6)

Action	Title	Year	Authors
+	A large annotated corpus for learning natural language inference	2015	Samuel R. Bowman Gabor Angeli Christopher Potts Christopher D. Manning
+	Natural Language Inference over Interaction Space	2017	Yichen Gong Heng Luo Jian Zhang
+ PDF Chat	Neural Natural Language Inference Models Enhanced with External Knowledge	2018	Chen Qian Xiaodan Zhu Zhen-Hua Ling Diana Inkpen Si Wei
+	Analyzing the Behavior of Visual Question Answering Models	2016	Aishwarya Agrawal Dhruv Batra Devi Parikh
+	A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference	2018	Adina Williams Nikita Nangia Samuel Bowman
+	The Effect of Different Writing Tasks on Linguistic Style: A Case Study of the ROC Story Cloze Task	2017	Roy Schwartz Maarten Sap Ioannis Konstas Leila Zilles Yejin Choi Noah A. Smith