Reducing Sentiment Bias in Language Models via Counterfactual Evaluation

Po-Sen Huang, Huan Zhang, Ray Jiang, Robert Stanforth, Johannes Welbl, Jack W. Rae, Vishal Maini, Dani Yogatama, Pushmeet Kohli

Type: Article

Publication Date: 2020-01-01

Citations: 104

DOI: https://doi.org/10.18653/v1/2020.findings-emnlp.7

Locations

arXiv (Cornell University) - View - PDF

Similar Works

Action	Title	Year	Authors
+	Exploring the Efficacy of Automatically Generated Counterfactuals for Sentiment Analysis	2021	Linyi Yang Jiazheng Li Pádraig Cunningham Yue Zhang Barry Smyth Ruihai Dong
+	Reducing Sentiment Bias in Language Models via Counterfactual Evaluation	2019	Po-Sen Huang Huan Zhang Ray Jiang Robert Stanforth Johannes Welbl Jack W. Rae Vishal Maini Dani Yogatama Pushmeet Kohli
+ PDF Chat	Speaking Multiple Languages Affects the Moral Bias of Language Models	2023	Katharina Haemmerl Bjoern Deiseroth Patrick Schramowski Jindřich Libovický Constantin A. Rothkopf Alexander Fraser Kristian Kersting
+ PDF Chat	Mitigating Gender Bias in Distilled Language Models via Counterfactual Role Reversal	2022	Umang Gupta Jwala Dhamala Varun Kumar Apurv Verma Yada Pruksachatkun Satyapriya Krishna Ashutosh Gupta Kai-Wei Chang Greg Ver Steeg Aram Galstyan
+	Societal Biases in Language Generation: Progress and Challenges	2021	Emily Sheng Kai-Wei Chang Prem Natarajan Nanyun Peng
+ PDF Chat	On Transferability of Bias Mitigation Effects in Language Model Fine-Tuning	2021	Xisen Jin Francesco Barbieri Brendan Kennedy Aida Mostafazadeh Davani Leonardo Neves Xiang Ren
+	Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Models	2021	Tongshuang Wu Marco Túlio Ribeiro Jeffrey Heer Daniel S. Weld
+	Cognitive Reframing of Negative Thoughts through Human-Language Model Interaction	2023	Ashish Sharma Kevin Rushton Inna Wanyin Lin David Wadden Khendra G. Lucas Adam S. Miner Theresa Nguyen Tim Althoff
+	On Positivity Bias in Negative Reviews	2021	Madhusudhan Aithal Chenhao Tan
+	Examining the Inductive Bias of Neural Language Models with Artificial Languages	2021	Jennifer C. White Ryan Cotterell
+	Bias Beyond English: Counterfactual Tests for Bias in Sentiment Analysis in Four Languages	2023	Seraphina Goldfarb-Tarrant Adam Lopez Roi Blanco Diego Marcheggiani
+ PDF Chat	Bias Beyond English: Counterfactual Tests for Bias in Sentiment Analysis in Four Languages	2023	Seraphina Goldfarb-Tarrant Adam Lopez Roi Blanco Diego Marcheggiani
+ PDF Chat	LLMs for Generating and Evaluating Counterfactuals: A Comprehensive Study	2024	Van Bach Nguyen Paul Youssef Jörg Schlötterer Christin Seifert
+	A Natural Bias for Language Generation Models	2023	Clara Meister Wojciech Stokowiec Tiago Pimentel Lei Yu Laura Rimell Adhiguna Kuncoro
+	Double Trouble: How to not explain a text classifier's decisions using counterfactuals synthesized by masked language models?	2021	Thang M. Pham Trung Bui Long Mai Anh‐Tu Nguyen
+ PDF Chat	Rating Sentiment Analysis Systems for Bias Through a Causal Lens	2024	Kausik Lakkaraju Biplav Srivastava Marco Valtorta
+ PDF Chat	Bias Neutralization Framework: Measuring Fairness in Large Language Models with Bias Intelligence Quotient (BiQ)	2024	Malur Narayan John Pasmore Elton Sampaio Raghavan Vijay Gabriella Waters
+ PDF Chat	Mitigating Social Bias in Large Language Models: A Multi-Objective Approach within a Multi-Agent Framework	2024	Zian Xu Wenqing Chen Yi Tang Xuanying Li Cheng Hu Zhixuan Chu Kui Ren Zibin Zheng Zhichao Lu
+	Simple synthetic data reduces sycophancy in large language models	2023	Jerry Wei Da Huang Yifeng Lu Denny Zhou Quoc V. Le
+	Language model acceptability judgements are not always robust to context	2023	Koustuv Sinha Jon Gauthier Aaron Mueller Kanishka Misra Keren Fuentes Roger Lévy Adina Williams

Works That Cite This (61)

Action	Title	Year	Authors
+ PDF Chat	Beneath the Tip of the Iceberg: Current Challenges and New Directions in Sentiment Analysis Research	2020	Soujanya Poria Devamanyu Hazarika Navonil Majumder Rada Mihalcea
+ PDF Chat	Bias of AI-Generated Content: An Examination of News Produced by Large Language Models	2023	Xiao Fang Shangkun Che Minjia Mao Hongzhe Zhang Ming Zhao Xiaohang Zhao
+	Societal Biases in Language Generation: Progress and Challenges	2021	Emily Sheng Kai-Wei Chang Prem Natarajan Nanyun Peng
+	Intersectional Bias in Causal Language Models	2021	Liam Magee Lida Ghahremanlou Karen Soldatić Shanthi Robertson
+	What do Bias Measures Measure	2021	Sunipa Dev Emily Sheng Jieyu Zhao Jiao Sun Yu Hou Mattie Sanseverino Ji‐In Kim Nanyun Peng Kai-Wei Chang
+ PDF Chat	A review of causality-based fairness machine learning	2022	Cong Su Guoxian Yu Jun Wang Zhongmin Yan Lizhen Cui
+	Investigating Societal Biases in a Poetry Composition System	2020	Emily Sheng David Uthus
+ PDF Chat	The Sentiment Problem: A Critical Survey towards Deconstructing Sentiment Analysis	2023	Pranav Narayanan Venkit Mukund Srinath Sanjana Gautam Saranya Venkatraman Vipul Gupta Rebecca J. Passonneau Shomir Wilson
+ PDF Chat	Bias and Fairness in Large Language Models: A Survey	2024	Isabel O. Gallegos Ryan A. Rossi Joe Barrow Md Mehrab Tanjim Sungchul Kim Franck Dernoncourt Tong Yu Ruiyi Zhang Nesreen K. Ahmed
+ PDF Chat	Generate Your Counterfactuals: Towards Controlled Counterfactual Generation for Text	2021	Nishtha Madaan Inkit Padhi Naveen Panwar Diptikalyan Saha

Works Cited by This (40)

Action	Title	Year	Authors
+	The Variational Fair Autoencoder	2015	Christos Louizos Kevin Swersky Yujia Li Max Welling Richard S. Zemel
+ PDF Chat	Fairness through awareness	2012	Cynthia Dwork Moritz Hardt Toniann Pitassi Omer Reingold Richard S. Zemel
+	Pointer Sentinel Mixture Models	2016	Stephen Merity Caiming Xiong James Bradbury Richard Socher
+	Equality of Opportunity in Supervised Learning	2016	Moritz Hardt Eric Price Nathan Srebro
+	Avoiding Discrimination through Causal Reasoning	2017	Niki Kilbertus Mateo Rojas-Carulla Giambattista Parascandolo Moritz Hardt Dominik Janzing Bernhard Schölkopf
+	Data Decisions and Theoretical Implications when Adversarially Learning Fair Representations	2017	Alex Beutel Ed H. Jilin Chen Zhe Zhao
+	Counterfactual Fairness	2017	Matt J. Kusner Joshua R. Loftus Chris Russell Ricardo Silva
+	Universal Sentence Encoder	2018	Daniel Cer Yinfei Yang Sheng-yi Kong Nan Hua Nicole Limtiaco Rhomni St. John Noah Constant Mario Guajardo-Céspedes Steve Yuan Chris Tar
+	Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems	2018	Svetlana Kiritchenko Saif M. Mohammad
+	Gender Bias in Neural Natural Language Processing	2018	Kaiji Lu Piotr Mardziel Fang‐Jing Wu Preetam Amancharla Anupam Datta