Fairness Evaluation in Text Classification: Machine Learning Practitioner Perspectives of Individual and Group Fairness

Zahra Ashktorab, Benjamin Hoover, Mayank Agarwal, Casey Dugan, Werner Geyer, Hao Yang, Mikhail Yurochkin

Type: Preprint

Publication Date: 2023-04-19

Citations: 9

DOI: https://doi.org/10.1145/3544548.3581227

Locations

arXiv (Cornell University) - View - PDF
DataCite API - View

Similar Works

Action	Title	Year	Authors
+	Measuring Fairness of Text Classifiers via Prediction Sensitivity	2022	Satyapriya Krishna Rahul Gupta Apurv Verma Jwala Dhamala Yada Pruksachatkun Kai-Wei Chang
+ PDF Chat	Analyzing Fairness of Classification Machine Learning Model with Structured Dataset	2024	Ahmed Nabih Zaki Rashed Abdelkrim Kallich Mohamed A. El-Tayeb
+	Counterfactual Fairness in Text Classification through Robustness	2018	Sahaj Garg Vinçent Pérot Nicole Limtiaco Ankur Taly Ed H. Alex Beutel
+	Counterfactual Fairness in Text Classification through Robustness	2019	Sahaj Garg Vincent Perot Nicole Limtiaco Ankur Taly Ed H. Alex Beutel
+	Fairness in Machine Learning: A Survey	2020	Simon Caton Christian Haas
+	Counterfactual Fairness in Text Classification through Robustness	2018	Sahaj Garg Vincent Perot Nicole Limtiaco Ankur Taly Ed H. Alex Beutel
+ PDF Chat	Perceived Fairness of the Machine Learning Development Process: Concept Scale Development	2025	Anoop Kumar Mishra Deepak Khazanchi
+ PDF Chat	Fairness in Machine Learning: A Survey	2023	Simon Caton Christian Haas
+ PDF Chat	A Review of Fairness and A Practical Guide to Selecting Context-Appropriate Fairness Metrics in Machine Learning	2024	C. Barr Olivia Johanna Erdélyi Paul D. Docherty Randolph C. Grace
+	Assessing Perceived Fairness from Machine Learning Developer's Perspective	2023	Anoop Kumar Mishra Deepak Khazanchi
+ PDF Chat	Improving Fairness in Machine Learning Systems	2019	Kenneth Holstein Jennifer Wortman Vaughan Hal Daumé Miro Dudík Hanna Wallach
+ PDF Chat	A Catalog of Fairness-Aware Practices in Machine Learning Engineering	2024	Gianmario Voria Giulia Sellitto Carmine Ferrara Francesco Abate Andrea De Lucia Filomena Ferrucci Gemma Catolino Fabio Palomba
+	Metrics and methods for a systematic comparison of fairness-aware machine learning algorithms.	2020	Gareth Jones James M. Hickey Pietro G. Di Stefano Charanpal Dhanjal Laura C. Stoddart Vlasios Vasileiou
+	Metrics and methods for a systematic comparison of fairness-aware machine learning algorithms	2020	Gareth Jones James M. Hickey Pietro G. Di Stefano Charanpal Dhanjal Laura C. Stoddart V. Vasileiou
+ PDF Chat	Improving Fairness in Machine Learning Systems: What Do Industry Practitioners Need?	2019	Kenneth Holstein Jennifer Wortman Vaughan Hal Daumé Miro Dudík Hanna Wallach
+	Do the machine learning models on a crowd sourced platform exhibit bias? an empirical study on model fairness	2020	Sumon Biswas Hridesh Rajan
+	Measuring Non-Expert Comprehension of Machine Learning Fairness Metrics.	2019	Debjani Saha Candice Schumann Duncan C. McElfresh John P. Dickerson Michelle L. Mazurek Michael Carl Tschantz
+ PDF Chat	Analyzing Fairness of Computer Vision and Natural Language Processing Models	2024	Ahmed Nabih Zaki Rashed Abdelkrim Kallich Mohamed M. Eltayeb
+	Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements	2019	Alex Beutel Jilin Chen Tulsee Doshi Hai Qian Allison Woodruff Christine Luu Pierre Kreitmann Jonathan Bischof Ed H.
+ PDF Chat	Through the Data Management Lens: Experimental Analysis and Evaluation of Fair Classification	2022	Maliha Tashfia Islam Anna Fariha Alexandra Meliou Babak Salimi

Works That Cite This (2)

Action	Title	Year	Authors
+	Towards a Non-Ideal Methodological Framework for Responsible ML	2024	Ramaravind Kommiya Mothilal Shion Guha Syed Ishtiaque Ahmed
+	Data Ethics Emergency Drill: A Toolbox for Discussing Responsible AI for Industry Teams	2024	Vanessa Hanschke Dylan Rees Merve Alanyali David Hopkinson Paul Marshall

Works Cited by This (45)

Action	Title	Year	Authors
+ PDF Chat	Fairness through awareness	2012	Cynthia Dwork Moritz Hardt Toniann Pitassi Omer Reingold Richard S. Zemel
+ PDF Chat	Inherent Tradeoffs in the Fair Determination of Risk Scores	2023	Manish Raghavan
+	Equality of Opportunity in Supervised Learning	2016	Moritz Hardt Eric Price Nathan Srebro
+ PDF Chat	Fairness in Criminal Justice Risk Assessments: The State of the Art	2018	Richard A. Berk Hoda Heidari Shahin Jabbari Michael Kearns Aaron Roth
+	Beyond Parity: Fairness Objectives for Collaborative Filtering	2017	Sirui Yao Bert Huang
+	Fairness and Accountability Design Needs for Algorithmic Support in High-Stakes Public Sector Decision-Making	2018	Michael Veale Max Van Kleek Reuben Binns
+	BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding	2018	Jacob Devlin Ming‐Wei Chang Kenton Lee Kristina Toutanova
+	How Do Fairness Definitions Fare? Examining Public Attitudes Towards Algorithmic Definitions of Fairness	2018	Nripsuta Ani Saxena Karen Huang Evan DeFilippis Goran Radanović David C. Parkes Yang Liu
+ PDF Chat	Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification	2019	Daniel Borkan Lucas Dixon Jeffrey Sorensen Nithum Thain Lucy Vasserman
+	Metric Learning for Individual Fairness	2019	Christina Ilvento