SU\G(𝔸)/K·U
Sign Up for free Log In

Author Description

Login to generate an author description

Ask a Question About This Mathematician

Harms of Gender Exclusivity and Challenges in Non-Binary Representation in Language Technologies

2021-01-01
Sunipa Dev , Masoud Monajatipoor , Anaelia Ovalle +3 more
Gender is widely discussed in the context of language tasks and when examining the stereotypes propagated by language models. However, current discussions primarily treat gender as binary, which can perpetuate … Gender is widely discussed in the context of language tasks and when examining the stereotypes propagated by language models. However, current discussions primarily treat gender as binary, which can perpetuate harms such as the cyclical erasure of non-binary gender identities. These harms are driven by model and dataset biases, which are consequences of the non-recognition and lack of understanding of non-binary genders in society. In this paper, we explain the complexity of gender and language around it, and survey non-binary persons to understand harms associated with the treatment of gender as binary in English language technologies. We also detail how current language representations (e.g., GloVe, BERT) capture and perpetuate these harms and related challenges that need to be acknowledged and addressed for representations to equitably encode gender information.
View PDF Chat

Queer In AI: A Case Study in Community-Led Participatory AI

2023-06-12
Organizers of QueerInAI , Anaelia Ovalle , Arjun Subramonian +7 more
We present Queer in AI as a case study for community-led participatory design in AI. We examine how participatory design and intersectional tenets started and shaped this community's programs over … We present Queer in AI as a case study for community-led participatory design in AI. We examine how participatory design and intersectional tenets started and shaped this community's programs over the years. We discuss different challenges that emerged in the process, look at ways this organization has fallen short of operationalizing participatory and intersectional principles, and then assess the organization's impact. Queer in AI provides important lessons and insights for practitioners and theorists of participatory methods broadly through its rejection of hierarchy in favor of decentralization, success at building aid and programs by and for the queer community, and effort to change actors and institutions outside of the queer community. Finally, we theorize how communities like Queer in AI contribute to the participatory design in AI more broadly by fostering cultures of participation in AI, welcoming and empowering marginalized participants, critiquing poor or exploitative participatory practices, and bringing participation to institutions outside of individual research projects. Queer in AI's work serves as a case study of grassroots activism and participatory methods within AI, demonstrating the potential of community-led participatory methods and intersectional praxis, while also providing challenges, case studies, and nuanced insights to researchers developing and using participatory methods.
View PDF Chat

“I’m fully who I am”: Towards Centering Transgender and Non-Binary Voices to Measure Biases in Open Language Generation

2023-06-12
Anaelia Ovalle , Palash Goyal , Jwala Dhamala +5 more
Warning: This paper contains examples of gender non-affirmative language which could be offensive, upsetting, and/or triggering. Transgender and non-binary (TGNB) individuals disproportionately experience discrimination and exclusion from daily life. Given … Warning: This paper contains examples of gender non-affirmative language which could be offensive, upsetting, and/or triggering. Transgender and non-binary (TGNB) individuals disproportionately experience discrimination and exclusion from daily life. Given the recent popularity and adoption of language generation technologies, the potential to further marginalize this population only grows. Although a multitude of NLP fairness literature focuses on illuminating and addressing gender biases, assessing gender harms for TGNB identities requires understanding how such identities uniquely interact with societal gender norms and how they differ from gender binary-centric perspectives. Such measurement frameworks inherently require centering TGNB voices to help guide the alignment between gender-inclusive NLP and whom they are intended to serve. Towards this goal, we ground our work in the TGNB community and existing interdisciplinary literature to assess how the social reality surrounding experienced marginalization of TGNB persons contributes to and persists within Open Language Generation (OLG). This social knowledge serves as a guide for evaluating popular large language models (LLMs) on two key aspects: (1) misgendering and (2) harmful responses to gender disclosure. To do this, we introduce TANGO, a dataset of template-based real-world text curated from a TGNB-oriented community. We discover a dominance of binary gender norms reflected by the models; LLMs least misgendered subjects in generated text when triggered by prompts whose subjects used binary pronouns. Meanwhile, misgendering was most prevalent when triggering generation with singular they and neopronouns. When prompted with gender disclosures, TGNB disclosure generated the most stigmatizing language and scored most toxic, on average. Our findings warrant further research on how TGNB harms manifest in LLMs and serve as a broader case study toward concretely grounding the design of gender-inclusive AI in community voices and interdisciplinary literature.
View PDF Chat

Factoring the Matrix of Domination: A Critical Review and Reimagination of Intersectionality in AI Fairness

2023-08-08
Anaelia Ovalle , Arjun Subramonian , Vagrant Gautam +2 more
Intersectionality is a critical framework that, through inquiry and praxis, allows us to examine how social inequalities persist through domains of structure and discipline. Given AI fairness' raison d'être of … Intersectionality is a critical framework that, through inquiry and praxis, allows us to examine how social inequalities persist through domains of structure and discipline. Given AI fairness' raison d'être of "fairness," we argue that adopting intersectionality as an analytical framework is pivotal to effectively operationalizing fairness. Through a critical review of how intersectionality is discussed in 30 papers from the AI fairness literature, we deductively and inductively: 1) map how intersectionality tenets operate within the AI fairness paradigm and 2) uncover gaps between the conceptualization and operationalization of intersectionality. We find that researchers overwhelmingly reduce intersectionality to optimizing for fairness metrics over demographic subgroups. They also fail to discuss their social context and when mentioning power, they mostly situate it only within the AI pipeline. We: 3) outline and assess the implications of these gaps for critical inquiry and praxis, and 4) provide actionable recommendations for AI fairness researchers to engage with intersectionality in their work by grounding it in AI epistemology.
View PDF Chat

Bound by the Bounty: Collaboratively Shaping Evaluation Processes for Queer AI Harms

2023-08-08
Nathaniel Dennler , Anaelia Ovalle , Ashwin Singh +7 more
Bias evaluation benchmarks and dataset and model documentation have emerged as central processes for assessing the biases and harms of artificial intelligence (AI) systems. However, these auditing processes have been … Bias evaluation benchmarks and dataset and model documentation have emerged as central processes for assessing the biases and harms of artificial intelligence (AI) systems. However, these auditing processes have been criticized for their failure to integrate the knowledge of marginalized communities and consider the power dynamics between auditors and the communities. Consequently, modes of bias evaluation have been proposed that engage impacted communities in identifying and assessing the harms of AI systems (e.g., bias bounties). Even so, asking what marginalized communities want from such auditing processes has been neglected. In this paper, we ask queer communities for their positions on, and desires from, auditing processes. To this end, we organized a participatory workshop to critique and redesign bias bounties from queer perspectives. We found that when given space, the scope of feedback from workshop participants goes far beyond what bias bounties afford, with participants questioning the ownership, incentives, and efficacy of bounties. We conclude by advocating for community ownership of bounties and complementing bounties with participatory processes (e.g., co-creation).
View PDF Chat

Applying Deep Learning to Public Health: Using Unbalanced Demographic Data to Predict Thyroid Disorder

2018-11-01
Yasser Attiga , Shih‐Yin Chen , John LaGue +6 more
This study investigates the use of Deep Neural Learning to predict propensity for disease from demographic information alone, with thyroid disease as the test application. The imbalanced dataset of 747,301 … This study investigates the use of Deep Neural Learning to predict propensity for disease from demographic information alone, with thyroid disease as the test application. The imbalanced dataset of 747,301 samples contained 13 demographic predictor variables that were not known to be associated with the disease, and had much missing information. A TensorFlow feed-forward neural network was trained to predict thyroid disease. Different activation functions and a variety of up-sampling and down-sampling methods were employed. The lift statistic was used to evaluate success in identifying patients with a propensity for thyroid disease. The DNN model outperformed the Random Forest model with a 36.63% improvement in the lift statistic. These results suggest that deep learning may be successfully employed to select candidates for early intervention for improved health outcomes, utilizing a large dataset with only minimal demographic variables, similar to datasets that are held by the marketing arms of healthcare providers.

Harms of Gender Exclusivity and Challenges in Non-Binary Representation in Language Technologies

2021-01-01
Sunipa Dev , Masoud Monajatipoor , Anaelia Ovalle +3 more
Gender is widely discussed in the context of language tasks and when examining the stereotypes propagated by language models. However, current discussions primarily treat gender as binary, which can perpetuate … Gender is widely discussed in the context of language tasks and when examining the stereotypes propagated by language models. However, current discussions primarily treat gender as binary, which can perpetuate harms such as the cyclical erasure of non-binary gender identities. These harms are driven by model and dataset biases, which are consequences of the non-recognition and lack of understanding of non-binary genders in society. In this paper, we explain the complexity of gender and language around it, and survey non-binary persons to understand harms associated with the treatment of gender as binary in English language technologies. We also detail how current language representations (e.g., GloVe, BERT) capture and perpetuate these harms and related challenges that need to be acknowledged and addressed for representations to equitably encode gender information.
View PDF Chat

Tokenization Matters: Navigating Data-Scarce Tokenization for Gender Inclusive Language Technologies

2024-01-01
Anaelia Ovalle , Ninareh Mehrabi , Palash Goyal +6 more
View PDF Chat

Should they? Mobile Biometrics and Technopolicy Meet Queer Community Considerations

2023-10-29
Anaelia Ovalle , Davi Liang , Alicia Boyd
Smartphones are integral to our daily lives and activities, providing us with basic functions like texting and phone calls to more complex motion-based functionalities like navigation, mobile gaming, and fitness-tracking. … Smartphones are integral to our daily lives and activities, providing us with basic functions like texting and phone calls to more complex motion-based functionalities like navigation, mobile gaming, and fitness-tracking. To facilitate these functionalities, smartphones rely on integrated sensors like accelerometers and gyroscopes. These sensors provide personalized measurements that, in turn, contribute to tasks such as analyzing biometric data for mobile health purposes. In addition to benefiting smartphone users, biometric data holds significant value for researchers engaged in biometric identification research. Nonetheless, utilizing this user data for biometric identification tasks, such as gait and gender recognition, raises serious privacy, normative, and ethical concerns, particularly within the queer community. Concerns of algorithmic bias and algorithmically-driven dysphoria surface from a historical backdrop of marginalization, surveillance, harassment, discrimination, and violence against the queer community. In this position paper, we contribute to the timely discourse on safeguarding human rights within AI-driven systems by providing a sense of challenges, tensions, and opportunities for new data protections and biometric collection practices in a way that grapples with the sociotechnical realities of the queer community.
View PDF Chat

Survey of Bias In Text-to-Image Generation: Definition, Evaluation, and Mitigation

2024-04-01
Yixin Wan , Arjun Subramonian , Anaelia Ovalle +6 more
The recent advancement of large and powerful models with Text-to-Image (T2I) generation abilities -- such as OpenAI's DALLE-3 and Google's Gemini -- enables users to generate high-quality images from textual … The recent advancement of large and powerful models with Text-to-Image (T2I) generation abilities -- such as OpenAI's DALLE-3 and Google's Gemini -- enables users to generate high-quality images from textual prompts. However, it has become increasingly evident that even simple prompts could cause T2I models to exhibit conspicuous social bias in generated images. Such bias might lead to both allocational and representational harms in society, further marginalizing minority groups. Noting this problem, a large body of recent works has been dedicated to investigating different dimensions of bias in T2I systems. However, an extensive review of these studies is lacking, hindering a systematic understanding of current progress and research gaps. We present the first extensive survey on bias in T2I generative models. In this survey, we review prior studies on dimensions of bias: Gender, Skintone, and Geo-Culture. Specifically, we discuss how these works define, evaluate, and mitigate different aspects of bias. We found that: (1) while gender and skintone biases are widely studied, geo-cultural bias remains under-explored; (2) most works on gender and skintone bias investigated occupational association, while other aspects are less frequently studied; (3) almost all gender bias works overlook non-binary identities in their studies; (4) evaluation datasets and metrics are scattered, with no unified framework for measuring biases; and (5) current mitigation methods fail to resolve biases comprehensively. Based on current limitations, we point out future research directions that contribute to human-centric definitions, evaluations, and mitigation of biases. We hope to highlight the importance of studying biases in T2I systems, as well as encourage future efforts to holistically understand and tackle biases, building fair and trustworthy T2I technologies for everyone.
View PDF Chat

COVID-19 and Big Data: Multi-faceted Analysis for Spatio-temporal Understanding of the Pandemic with Social Media Conversations

2021-01-01
Shayan Fazeli , Davina Zamanzadeh , Anaelia Ovalle +3 more
COVID-19 has been devastating the world since the end of 2019 and has continued to play a significant role in major national and worldwide events, and consequently, the news. In … COVID-19 has been devastating the world since the end of 2019 and has continued to play a significant role in major national and worldwide events, and consequently, the news. In its wake, it has left no life unaffected. Having earned the world's attention, social media platforms have served as a vehicle for the global conversation about COVID-19. In particular, many people have used these sites in order to express their feelings, experiences, and observations about the pandemic. We provide a multi-faceted analysis of critical properties exhibited by these conversations on social media regarding the novel coronavirus pandemic. We present a framework for analysis, mining, and tracking the critical content and characteristics of social media conversations around the pandemic. Focusing on Twitter and Reddit, we have gathered a large-scale dataset on COVID-19 social media conversations. Our analyses cover tracking potential reports on virus acquisition, symptoms, conversation topics, and language complexity measures through time and by region across the United States. We also present a BERT-based model for recognizing instances of hateful tweets in COVID-19 conversations, which achieves a lower error-rate than the state-of-the-art performance. Our results provide empirical validation for the effectiveness of our proposed framework and further demonstrate that social media data can be efficiently leveraged to provide public health experts with inexpensive but thorough insight over the course of an outbreak.

ChatGPT for Us: Preserving Data Privacy in ChatGPT via Dialogue Text Ambiguation to Expand Mental Health Care Delivery

2023-01-01
Anaelia Ovalle , Mehrab Beikzadeh , Parshan Teimouri +2 more
Large language models have been useful in expanding mental health care delivery. ChatGPT, in particular, has gained popularity for its ability to generate human-like dialogue. However, data-sensitive domains -- including … Large language models have been useful in expanding mental health care delivery. ChatGPT, in particular, has gained popularity for its ability to generate human-like dialogue. However, data-sensitive domains -- including but not limited to healthcare -- face challenges in using ChatGPT due to privacy and data-ownership concerns. To enable its utilization, we propose a text ambiguation framework that preserves user privacy. We ground this in the task of addressing stress prompted by user-provided texts to demonstrate the viability and helpfulness of privacy-preserved generations. Our results suggest that chatGPT recommendations are still able to be moderately helpful and relevant, even when the original user text is not provided.
View PDF Chat

Improving Adversarial Robustness to Sensitivity and Invariance Attacks with Deep Metric Learning

2022-01-01
Anaelia Ovalle , Evan Czyzycki , Cho‐Jui Hsieh
Intentionally crafted adversarial samples have effectively exploited weaknesses in deep neural networks. A standard method in adversarial robustness assumes a framework to defend against samples crafted by minimally perturbing a … Intentionally crafted adversarial samples have effectively exploited weaknesses in deep neural networks. A standard method in adversarial robustness assumes a framework to defend against samples crafted by minimally perturbing a sample such that its corresponding model output changes. These sensitivity attacks exploit the model's sensitivity toward task-irrelevant features. Another form of adversarial sample can be crafted via invariance attacks, which exploit the model underestimating the importance of relevant features. Previous literature has indicated a tradeoff in defending against both attack types within a strictly L_p bounded defense. To promote robustness toward both types of attacks beyond Euclidean distance metrics, we use metric learning to frame adversarial regularization as an optimal transport problem. Our preliminary results indicate that regularizing over invariant perturbations in our framework improves both invariant and sensitivity defense.
View PDF Chat

Auditing Algorithmic Fairness in Machine Learning for Health with Severity-Based LOGAN

2022-01-01
Anaelia Ovalle , Sunipa Dev , Jieyu Zhao +2 more
Auditing machine learning-based (ML) healthcare tools for bias is critical to preventing patient harm, especially in communities that disproportionately face health inequities. General frameworks are becoming increasingly available to measure … Auditing machine learning-based (ML) healthcare tools for bias is critical to preventing patient harm, especially in communities that disproportionately face health inequities. General frameworks are becoming increasingly available to measure ML fairness gaps between groups. However, ML for health (ML4H) auditing principles call for a contextual, patient-centered approach to model assessment. Therefore, ML auditing tools must be (1) better aligned with ML4H auditing principles and (2) able to illuminate and characterize communities vulnerable to the most harm. To address this gap, we propose supplementing ML4H auditing frameworks with SLOGAN (patient Severity-based LOcal Group biAs detectioN), an automatic tool for capturing local biases in a clinical prediction task. SLOGAN adapts an existing tool, LOGAN (LOcal Group biAs detectioN), by contextualizing group bias detection in patient illness severity and past medical history. We investigate and compare SLOGAN's bias detection capabilities to LOGAN and other clustering techniques across patient subgroups in the MIMIC-III dataset. On average, SLOGAN identifies larger fairness disparities in over 75% of patient groups than LOGAN while maintaining clustering quality. Furthermore, in a diabetes case study, health disparity literature corroborates the characterizations of the most biased clusters identified by SLOGAN. Our results contribute to the broader discussion of how machine learning biases may perpetuate existing healthcare disparities.

Queer In AI: A Case Study in Community-Led Participatory AI

2023-01-01
Organizers Of Queer in AI , Anaelia Ovalle , Arjun Subramonian +7 more
We present Queer in AI as a case study for community-led participatory design in AI. We examine how participatory design and intersectional tenets started and shaped this community's programs over … We present Queer in AI as a case study for community-led participatory design in AI. We examine how participatory design and intersectional tenets started and shaped this community's programs over the years. We discuss different challenges that emerged in the process, look at ways this organization has fallen short of operationalizing participatory and intersectional principles, and then assess the organization's impact. Queer in AI provides important lessons and insights for practitioners and theorists of participatory methods broadly through its rejection of hierarchy in favor of decentralization, success at building aid and programs by and for the queer community, and effort to change actors and institutions outside of the queer community. Finally, we theorize how communities like Queer in AI contribute to the participatory design in AI more broadly by fostering cultures of participation in AI, welcoming and empowering marginalized participants, critiquing poor or exploitative participatory practices, and bringing participation to institutions outside of individual research projects. Queer in AI's work serves as a case study of grassroots activism and participatory methods within AI, demonstrating the potential of community-led participatory methods and intersectional praxis, while also providing challenges, case studies, and nuanced insights to researchers developing and using participatory methods.

Factoring the Matrix of Domination: A Critical Review and Reimagination of Intersectionality in AI Fairness

2023-01-01
Anaelia Ovalle , Arjun Subramonian , Vagrant Gautam +2 more
Intersectionality is a critical framework that, through inquiry and praxis, allows us to examine how social inequalities persist through domains of structure and discipline. Given AI fairness' raison d'etre of … Intersectionality is a critical framework that, through inquiry and praxis, allows us to examine how social inequalities persist through domains of structure and discipline. Given AI fairness' raison d'etre of "fairness", we argue that adopting intersectionality as an analytical framework is pivotal to effectively operationalizing fairness. Through a critical review of how intersectionality is discussed in 30 papers from the AI fairness literature, we deductively and inductively: 1) map how intersectionality tenets operate within the AI fairness paradigm and 2) uncover gaps between the conceptualization and operationalization of intersectionality. We find that researchers overwhelmingly reduce intersectionality to optimizing for fairness metrics over demographic subgroups. They also fail to discuss their social context and when mentioning power, they mostly situate it only within the AI pipeline. We: 3) outline and assess the implications of these gaps for critical inquiry and praxis, and 4) provide actionable recommendations for AI fairness researchers to engage with intersectionality in their work by grounding it in AI epistemology.
View PDF Chat

Bound by the Bounty: Collaboratively Shaping Evaluation Processes for Queer AI Harms

2023-01-01
Organizers of QueerInAI , Nathan Dennler , Anaelia Ovalle +7 more
Bias evaluation benchmarks and dataset and model documentation have emerged as central processes for assessing the biases and harms of artificial intelligence (AI) systems. However, these auditing processes have been … Bias evaluation benchmarks and dataset and model documentation have emerged as central processes for assessing the biases and harms of artificial intelligence (AI) systems. However, these auditing processes have been criticized for their failure to integrate the knowledge of marginalized communities and consider the power dynamics between auditors and the communities. Consequently, modes of bias evaluation have been proposed that engage impacted communities in identifying and assessing the harms of AI systems (e.g., bias bounties). Even so, asking what marginalized communities want from such auditing processes has been neglected. In this paper, we ask queer communities for their positions on, and desires from, auditing processes. To this end, we organized a participatory workshop to critique and redesign bias bounties from queer perspectives. We found that when given space, the scope of feedback from workshop participants goes far beyond what bias bounties afford, with participants questioning the ownership, incentives, and efficacy of bounties. We conclude by advocating for community ownership of bounties and complementing bounties with participatory processes (e.g., co-creation).
View PDF Chat

Decoding The Digital Fuku: Deciphering Colonial Legacies to Critically Assess ChatGPT in Dominican Education

2023-01-01
Anaelia Ovalle
Educational disparities within the Dominican Republic (DR) have long-standing origins rooted in economic, political, and social inequity. Addressing these challenges has necessarily called for capacity building with respect to educational … Educational disparities within the Dominican Republic (DR) have long-standing origins rooted in economic, political, and social inequity. Addressing these challenges has necessarily called for capacity building with respect to educational materials, high-quality instruction, and structural resourcing. Generative AI tools like ChatGPT have begun to pique the interest of Dominican educators due to their perceived potential to bridge these educational gaps. However, a substantial body of AI fairness literature has documented ways AI disproportionately reinforces power dynamics reflective of jurisdictions driving AI development and deployment policies, collectively termed the AI Global North. As such, indiscriminate adoption of this technology for DR education, even in part, risks perpetuating forms of digital coloniality. Therefore, this paper centers embracing AI-facilitated educational reform by critically examining how AI-driven tools like ChatGPT in DR education may replicate facets of digital colonialism. We provide a concise overview of 20th-century Dominican education reforms following the 1916 US occupation. Then, we employ identified neocolonial aspects historically shaping Dominican education to interrogate the perceived advantages of ChatGPT for contemporary Dominican education, as outlined by a Dominican scholar. This work invites AI Global North & South developers, stakeholders, and Dominican leaders alike to exercise a relational contextualization of data-centric epistemologies like ChatGPT to reap its transformative benefits while remaining vigilant of safeguarding Dominican digital sovereignty.
View PDF Chat

Should they? Mobile Biometrics and Technopolicy meet Queer Community Considerations

2023-01-01
Anaelia Ovalle , Davi Liang , Alicia Boyd
Smartphones are integral to our daily lives and activities, providing us with basic functions like texting and phone calls to more complex motion-based functionalities like navigation, mobile gaming, and fitness-tracking. … Smartphones are integral to our daily lives and activities, providing us with basic functions like texting and phone calls to more complex motion-based functionalities like navigation, mobile gaming, and fitness-tracking. To facilitate these functionalities, smartphones rely on integrated sensors like accelerometers and gyroscopes. These sensors provide personalized measurements that, in turn, contribute to tasks such as analyzing biometric data for mobile health purposes. In addition to benefiting smartphone users, biometric data holds significant value for researchers engaged in biometric identification research. Nonetheless, utilizing this user data for biometric identification tasks, such as gait and gender recognition, raises serious privacy, normative, and ethical concerns, particularly within the queer community. Concerns of algorithmic bias and algorithmically-driven dysphoria surface from a historical backdrop of marginalization, surveillance, harassment, discrimination, and violence against the queer community. In this position paper, we contribute to the timely discourse on safeguarding human rights within AI-driven systems by providing a sense of challenges, tensions, and opportunities for new data protections and biometric collection practices in a way that grapples with the sociotechnical realities of the queer community.
View PDF Chat

Are you talking to ['xem'] or ['x', 'em']? On Tokenization and Addressing Misgendering in LLMs with Pronoun Tokenization Parity

2023-01-01
Anaelia Ovalle , Ninareh Mehrabi , Palash Goyal +6 more
A large body of NLP research has documented the ways gender biases manifest and amplify within large language models (LLMs), though this research has predominantly operated within a gender binary-centric … A large body of NLP research has documented the ways gender biases manifest and amplify within large language models (LLMs), though this research has predominantly operated within a gender binary-centric context. A growing body of work has identified the harmful limitations of this gender-exclusive framing; many LLMs cannot correctly and consistently refer to persons outside the gender binary, especially if they use neopronouns. While data scarcity has been identified as a possible culprit, the precise mechanisms through which it influences LLM misgendering remain underexplored. Our work addresses this gap by studying data scarcity's role in subword tokenization and, consequently, the formation of LLM word representations. We uncover how the Byte-Pair Encoding (BPE) tokenizer, a backbone for many popular LLMs, contributes to neopronoun misgendering through out-of-vocabulary behavior. We introduce pronoun tokenization parity (PTP), a novel approach to reduce LLM neopronoun misgendering by preserving a token's functional structure. We evaluate PTP's efficacy using pronoun consistency-based metrics and a novel syntax-based metric. Through several controlled experiments, finetuning LLMs with PTP improves neopronoun consistency from 14.5% to 58.4%, highlighting the significant role tokenization plays in LLM pronoun consistency.

The Cake that is Intelligence and Who Gets to Bake it: An AI Analogy and its Implications for Participation

2025-02-05
Martin Mundt , Anaelia Ovalle , Felix Friedrich +5 more
In a widely popular analogy by Turing Award Laureate Yann LeCun, machine intelligence has been compared to cake - where unsupervised learning forms the base, supervised learning adds the icing, … In a widely popular analogy by Turing Award Laureate Yann LeCun, machine intelligence has been compared to cake - where unsupervised learning forms the base, supervised learning adds the icing, and reinforcement learning is the cherry on top. We expand this 'cake that is intelligence' analogy from a simple structural metaphor to the full life-cycle of AI systems, extending it to sourcing of ingredients (data), conception of recipes (instructions), the baking process (training), and the tasting and selling of the cake (evaluation and distribution). Leveraging our re-conceptualization, we describe each step's entailed social ramifications and how they are bounded by statistical assumptions within machine learning. Whereas these technical foundations and social impacts are deeply intertwined, they are often studied in isolation, creating barriers that restrict meaningful participation. Our re-conceptualization paves the way to bridge this gap by mapping where technical foundations interact with social outcomes, highlighting opportunities for cross-disciplinary dialogue. Finally, we conclude with actionable recommendations at each stage of the metaphorical AI cake's life-cycle, empowering prospective AI practitioners, users, and researchers, with increased awareness and ability to engage in broader AI discourse.
View PDF Chat

The Cake that is Intelligence and Who Gets to Bake it: An AI Analogy and its Implications for Participation

2025-02-05
Martin Mundt , Anaelia Ovalle , Felix Friedrich +5 more
In a widely popular analogy by Turing Award Laureate Yann LeCun, machine intelligence has been compared to cake - where unsupervised learning forms the base, supervised learning adds the icing, … In a widely popular analogy by Turing Award Laureate Yann LeCun, machine intelligence has been compared to cake - where unsupervised learning forms the base, supervised learning adds the icing, and reinforcement learning is the cherry on top. We expand this 'cake that is intelligence' analogy from a simple structural metaphor to the full life-cycle of AI systems, extending it to sourcing of ingredients (data), conception of recipes (instructions), the baking process (training), and the tasting and selling of the cake (evaluation and distribution). Leveraging our re-conceptualization, we describe each step's entailed social ramifications and how they are bounded by statistical assumptions within machine learning. Whereas these technical foundations and social impacts are deeply intertwined, they are often studied in isolation, creating barriers that restrict meaningful participation. Our re-conceptualization paves the way to bridge this gap by mapping where technical foundations interact with social outcomes, highlighting opportunities for cross-disciplinary dialogue. Finally, we conclude with actionable recommendations at each stage of the metaphorical AI cake's life-cycle, empowering prospective AI practitioners, users, and researchers, with increased awareness and ability to engage in broader AI discourse.
View PDF Chat

Survey of Bias In Text-to-Image Generation: Definition, Evaluation, and Mitigation

2024-04-01
Yixin Wan , Arjun Subramonian , Anaelia Ovalle +6 more
The recent advancement of large and powerful models with Text-to-Image (T2I) generation abilities -- such as OpenAI's DALLE-3 and Google's Gemini -- enables users to generate high-quality images from textual … The recent advancement of large and powerful models with Text-to-Image (T2I) generation abilities -- such as OpenAI's DALLE-3 and Google's Gemini -- enables users to generate high-quality images from textual prompts. However, it has become increasingly evident that even simple prompts could cause T2I models to exhibit conspicuous social bias in generated images. Such bias might lead to both allocational and representational harms in society, further marginalizing minority groups. Noting this problem, a large body of recent works has been dedicated to investigating different dimensions of bias in T2I systems. However, an extensive review of these studies is lacking, hindering a systematic understanding of current progress and research gaps. We present the first extensive survey on bias in T2I generative models. In this survey, we review prior studies on dimensions of bias: Gender, Skintone, and Geo-Culture. Specifically, we discuss how these works define, evaluate, and mitigate different aspects of bias. We found that: (1) while gender and skintone biases are widely studied, geo-cultural bias remains under-explored; (2) most works on gender and skintone bias investigated occupational association, while other aspects are less frequently studied; (3) almost all gender bias works overlook non-binary identities in their studies; (4) evaluation datasets and metrics are scattered, with no unified framework for measuring biases; and (5) current mitigation methods fail to resolve biases comprehensively. Based on current limitations, we point out future research directions that contribute to human-centric definitions, evaluations, and mitigation of biases. We hope to highlight the importance of studying biases in T2I systems, as well as encourage future efforts to holistically understand and tackle biases, building fair and trustworthy T2I technologies for everyone.
View PDF Chat

Tokenization Matters: Navigating Data-Scarce Tokenization for Gender Inclusive Language Technologies

2024-01-01
Anaelia Ovalle , Ninareh Mehrabi , Palash Goyal +6 more
View PDF Chat

Should they? Mobile Biometrics and Technopolicy Meet Queer Community Considerations

2023-10-29
Anaelia Ovalle , Davi Liang , Alicia Boyd
Smartphones are integral to our daily lives and activities, providing us with basic functions like texting and phone calls to more complex motion-based functionalities like navigation, mobile gaming, and fitness-tracking. … Smartphones are integral to our daily lives and activities, providing us with basic functions like texting and phone calls to more complex motion-based functionalities like navigation, mobile gaming, and fitness-tracking. To facilitate these functionalities, smartphones rely on integrated sensors like accelerometers and gyroscopes. These sensors provide personalized measurements that, in turn, contribute to tasks such as analyzing biometric data for mobile health purposes. In addition to benefiting smartphone users, biometric data holds significant value for researchers engaged in biometric identification research. Nonetheless, utilizing this user data for biometric identification tasks, such as gait and gender recognition, raises serious privacy, normative, and ethical concerns, particularly within the queer community. Concerns of algorithmic bias and algorithmically-driven dysphoria surface from a historical backdrop of marginalization, surveillance, harassment, discrimination, and violence against the queer community. In this position paper, we contribute to the timely discourse on safeguarding human rights within AI-driven systems by providing a sense of challenges, tensions, and opportunities for new data protections and biometric collection practices in a way that grapples with the sociotechnical realities of the queer community.
View PDF Chat

Bound by the Bounty: Collaboratively Shaping Evaluation Processes for Queer AI Harms

2023-08-08
Nathaniel Dennler , Anaelia Ovalle , Ashwin Singh +7 more
Bias evaluation benchmarks and dataset and model documentation have emerged as central processes for assessing the biases and harms of artificial intelligence (AI) systems. However, these auditing processes have been … Bias evaluation benchmarks and dataset and model documentation have emerged as central processes for assessing the biases and harms of artificial intelligence (AI) systems. However, these auditing processes have been criticized for their failure to integrate the knowledge of marginalized communities and consider the power dynamics between auditors and the communities. Consequently, modes of bias evaluation have been proposed that engage impacted communities in identifying and assessing the harms of AI systems (e.g., bias bounties). Even so, asking what marginalized communities want from such auditing processes has been neglected. In this paper, we ask queer communities for their positions on, and desires from, auditing processes. To this end, we organized a participatory workshop to critique and redesign bias bounties from queer perspectives. We found that when given space, the scope of feedback from workshop participants goes far beyond what bias bounties afford, with participants questioning the ownership, incentives, and efficacy of bounties. We conclude by advocating for community ownership of bounties and complementing bounties with participatory processes (e.g., co-creation).
View PDF Chat

Factoring the Matrix of Domination: A Critical Review and Reimagination of Intersectionality in AI Fairness

2023-08-08
Anaelia Ovalle , Arjun Subramonian , Vagrant Gautam +2 more
Intersectionality is a critical framework that, through inquiry and praxis, allows us to examine how social inequalities persist through domains of structure and discipline. Given AI fairness' raison d'être of … Intersectionality is a critical framework that, through inquiry and praxis, allows us to examine how social inequalities persist through domains of structure and discipline. Given AI fairness' raison d'être of "fairness," we argue that adopting intersectionality as an analytical framework is pivotal to effectively operationalizing fairness. Through a critical review of how intersectionality is discussed in 30 papers from the AI fairness literature, we deductively and inductively: 1) map how intersectionality tenets operate within the AI fairness paradigm and 2) uncover gaps between the conceptualization and operationalization of intersectionality. We find that researchers overwhelmingly reduce intersectionality to optimizing for fairness metrics over demographic subgroups. They also fail to discuss their social context and when mentioning power, they mostly situate it only within the AI pipeline. We: 3) outline and assess the implications of these gaps for critical inquiry and praxis, and 4) provide actionable recommendations for AI fairness researchers to engage with intersectionality in their work by grounding it in AI epistemology.
View PDF Chat

“I’m fully who I am”: Towards Centering Transgender and Non-Binary Voices to Measure Biases in Open Language Generation

2023-06-12
Anaelia Ovalle , Palash Goyal , Jwala Dhamala +5 more
Warning: This paper contains examples of gender non-affirmative language which could be offensive, upsetting, and/or triggering. Transgender and non-binary (TGNB) individuals disproportionately experience discrimination and exclusion from daily life. Given … Warning: This paper contains examples of gender non-affirmative language which could be offensive, upsetting, and/or triggering. Transgender and non-binary (TGNB) individuals disproportionately experience discrimination and exclusion from daily life. Given the recent popularity and adoption of language generation technologies, the potential to further marginalize this population only grows. Although a multitude of NLP fairness literature focuses on illuminating and addressing gender biases, assessing gender harms for TGNB identities requires understanding how such identities uniquely interact with societal gender norms and how they differ from gender binary-centric perspectives. Such measurement frameworks inherently require centering TGNB voices to help guide the alignment between gender-inclusive NLP and whom they are intended to serve. Towards this goal, we ground our work in the TGNB community and existing interdisciplinary literature to assess how the social reality surrounding experienced marginalization of TGNB persons contributes to and persists within Open Language Generation (OLG). This social knowledge serves as a guide for evaluating popular large language models (LLMs) on two key aspects: (1) misgendering and (2) harmful responses to gender disclosure. To do this, we introduce TANGO, a dataset of template-based real-world text curated from a TGNB-oriented community. We discover a dominance of binary gender norms reflected by the models; LLMs least misgendered subjects in generated text when triggered by prompts whose subjects used binary pronouns. Meanwhile, misgendering was most prevalent when triggering generation with singular they and neopronouns. When prompted with gender disclosures, TGNB disclosure generated the most stigmatizing language and scored most toxic, on average. Our findings warrant further research on how TGNB harms manifest in LLMs and serve as a broader case study toward concretely grounding the design of gender-inclusive AI in community voices and interdisciplinary literature.
View PDF Chat

Queer In AI: A Case Study in Community-Led Participatory AI

2023-06-12
Organizers of QueerInAI , Anaelia Ovalle , Arjun Subramonian +7 more
We present Queer in AI as a case study for community-led participatory design in AI. We examine how participatory design and intersectional tenets started and shaped this community's programs over … We present Queer in AI as a case study for community-led participatory design in AI. We examine how participatory design and intersectional tenets started and shaped this community's programs over the years. We discuss different challenges that emerged in the process, look at ways this organization has fallen short of operationalizing participatory and intersectional principles, and then assess the organization's impact. Queer in AI provides important lessons and insights for practitioners and theorists of participatory methods broadly through its rejection of hierarchy in favor of decentralization, success at building aid and programs by and for the queer community, and effort to change actors and institutions outside of the queer community. Finally, we theorize how communities like Queer in AI contribute to the participatory design in AI more broadly by fostering cultures of participation in AI, welcoming and empowering marginalized participants, critiquing poor or exploitative participatory practices, and bringing participation to institutions outside of individual research projects. Queer in AI's work serves as a case study of grassroots activism and participatory methods within AI, demonstrating the potential of community-led participatory methods and intersectional praxis, while also providing challenges, case studies, and nuanced insights to researchers developing and using participatory methods.
View PDF Chat

Queer In AI: A Case Study in Community-Led Participatory AI

2023-01-01
Organizers Of Queer in AI , Anaelia Ovalle , Arjun Subramonian +7 more
We present Queer in AI as a case study for community-led participatory design in AI. We examine how participatory design and intersectional tenets started and shaped this community's programs over … We present Queer in AI as a case study for community-led participatory design in AI. We examine how participatory design and intersectional tenets started and shaped this community's programs over the years. We discuss different challenges that emerged in the process, look at ways this organization has fallen short of operationalizing participatory and intersectional principles, and then assess the organization's impact. Queer in AI provides important lessons and insights for practitioners and theorists of participatory methods broadly through its rejection of hierarchy in favor of decentralization, success at building aid and programs by and for the queer community, and effort to change actors and institutions outside of the queer community. Finally, we theorize how communities like Queer in AI contribute to the participatory design in AI more broadly by fostering cultures of participation in AI, welcoming and empowering marginalized participants, critiquing poor or exploitative participatory practices, and bringing participation to institutions outside of individual research projects. Queer in AI's work serves as a case study of grassroots activism and participatory methods within AI, demonstrating the potential of community-led participatory methods and intersectional praxis, while also providing challenges, case studies, and nuanced insights to researchers developing and using participatory methods.

Factoring the Matrix of Domination: A Critical Review and Reimagination of Intersectionality in AI Fairness

2023-01-01
Anaelia Ovalle , Arjun Subramonian , Vagrant Gautam +2 more
Intersectionality is a critical framework that, through inquiry and praxis, allows us to examine how social inequalities persist through domains of structure and discipline. Given AI fairness' raison d'etre of … Intersectionality is a critical framework that, through inquiry and praxis, allows us to examine how social inequalities persist through domains of structure and discipline. Given AI fairness' raison d'etre of "fairness", we argue that adopting intersectionality as an analytical framework is pivotal to effectively operationalizing fairness. Through a critical review of how intersectionality is discussed in 30 papers from the AI fairness literature, we deductively and inductively: 1) map how intersectionality tenets operate within the AI fairness paradigm and 2) uncover gaps between the conceptualization and operationalization of intersectionality. We find that researchers overwhelmingly reduce intersectionality to optimizing for fairness metrics over demographic subgroups. They also fail to discuss their social context and when mentioning power, they mostly situate it only within the AI pipeline. We: 3) outline and assess the implications of these gaps for critical inquiry and praxis, and 4) provide actionable recommendations for AI fairness researchers to engage with intersectionality in their work by grounding it in AI epistemology.
View PDF Chat

ChatGPT for Us: Preserving Data Privacy in ChatGPT via Dialogue Text Ambiguation to Expand Mental Health Care Delivery

2023-01-01
Anaelia Ovalle , Mehrab Beikzadeh , Parshan Teimouri +2 more
Large language models have been useful in expanding mental health care delivery. ChatGPT, in particular, has gained popularity for its ability to generate human-like dialogue. However, data-sensitive domains -- including … Large language models have been useful in expanding mental health care delivery. ChatGPT, in particular, has gained popularity for its ability to generate human-like dialogue. However, data-sensitive domains -- including but not limited to healthcare -- face challenges in using ChatGPT due to privacy and data-ownership concerns. To enable its utilization, we propose a text ambiguation framework that preserves user privacy. We ground this in the task of addressing stress prompted by user-provided texts to demonstrate the viability and helpfulness of privacy-preserved generations. Our results suggest that chatGPT recommendations are still able to be moderately helpful and relevant, even when the original user text is not provided.
View PDF Chat

Bound by the Bounty: Collaboratively Shaping Evaluation Processes for Queer AI Harms

2023-01-01
Organizers of QueerInAI , Nathan Dennler , Anaelia Ovalle +7 more
Bias evaluation benchmarks and dataset and model documentation have emerged as central processes for assessing the biases and harms of artificial intelligence (AI) systems. However, these auditing processes have been … Bias evaluation benchmarks and dataset and model documentation have emerged as central processes for assessing the biases and harms of artificial intelligence (AI) systems. However, these auditing processes have been criticized for their failure to integrate the knowledge of marginalized communities and consider the power dynamics between auditors and the communities. Consequently, modes of bias evaluation have been proposed that engage impacted communities in identifying and assessing the harms of AI systems (e.g., bias bounties). Even so, asking what marginalized communities want from such auditing processes has been neglected. In this paper, we ask queer communities for their positions on, and desires from, auditing processes. To this end, we organized a participatory workshop to critique and redesign bias bounties from queer perspectives. We found that when given space, the scope of feedback from workshop participants goes far beyond what bias bounties afford, with participants questioning the ownership, incentives, and efficacy of bounties. We conclude by advocating for community ownership of bounties and complementing bounties with participatory processes (e.g., co-creation).
View PDF Chat

Decoding The Digital Fuku: Deciphering Colonial Legacies to Critically Assess ChatGPT in Dominican Education

2023-01-01
Anaelia Ovalle
Educational disparities within the Dominican Republic (DR) have long-standing origins rooted in economic, political, and social inequity. Addressing these challenges has necessarily called for capacity building with respect to educational … Educational disparities within the Dominican Republic (DR) have long-standing origins rooted in economic, political, and social inequity. Addressing these challenges has necessarily called for capacity building with respect to educational materials, high-quality instruction, and structural resourcing. Generative AI tools like ChatGPT have begun to pique the interest of Dominican educators due to their perceived potential to bridge these educational gaps. However, a substantial body of AI fairness literature has documented ways AI disproportionately reinforces power dynamics reflective of jurisdictions driving AI development and deployment policies, collectively termed the AI Global North. As such, indiscriminate adoption of this technology for DR education, even in part, risks perpetuating forms of digital coloniality. Therefore, this paper centers embracing AI-facilitated educational reform by critically examining how AI-driven tools like ChatGPT in DR education may replicate facets of digital colonialism. We provide a concise overview of 20th-century Dominican education reforms following the 1916 US occupation. Then, we employ identified neocolonial aspects historically shaping Dominican education to interrogate the perceived advantages of ChatGPT for contemporary Dominican education, as outlined by a Dominican scholar. This work invites AI Global North & South developers, stakeholders, and Dominican leaders alike to exercise a relational contextualization of data-centric epistemologies like ChatGPT to reap its transformative benefits while remaining vigilant of safeguarding Dominican digital sovereignty.
View PDF Chat

Should they? Mobile Biometrics and Technopolicy meet Queer Community Considerations

2023-01-01
Anaelia Ovalle , Davi Liang , Alicia Boyd
Smartphones are integral to our daily lives and activities, providing us with basic functions like texting and phone calls to more complex motion-based functionalities like navigation, mobile gaming, and fitness-tracking. … Smartphones are integral to our daily lives and activities, providing us with basic functions like texting and phone calls to more complex motion-based functionalities like navigation, mobile gaming, and fitness-tracking. To facilitate these functionalities, smartphones rely on integrated sensors like accelerometers and gyroscopes. These sensors provide personalized measurements that, in turn, contribute to tasks such as analyzing biometric data for mobile health purposes. In addition to benefiting smartphone users, biometric data holds significant value for researchers engaged in biometric identification research. Nonetheless, utilizing this user data for biometric identification tasks, such as gait and gender recognition, raises serious privacy, normative, and ethical concerns, particularly within the queer community. Concerns of algorithmic bias and algorithmically-driven dysphoria surface from a historical backdrop of marginalization, surveillance, harassment, discrimination, and violence against the queer community. In this position paper, we contribute to the timely discourse on safeguarding human rights within AI-driven systems by providing a sense of challenges, tensions, and opportunities for new data protections and biometric collection practices in a way that grapples with the sociotechnical realities of the queer community.
View PDF Chat

Are you talking to ['xem'] or ['x', 'em']? On Tokenization and Addressing Misgendering in LLMs with Pronoun Tokenization Parity

2023-01-01
Anaelia Ovalle , Ninareh Mehrabi , Palash Goyal +6 more
A large body of NLP research has documented the ways gender biases manifest and amplify within large language models (LLMs), though this research has predominantly operated within a gender binary-centric … A large body of NLP research has documented the ways gender biases manifest and amplify within large language models (LLMs), though this research has predominantly operated within a gender binary-centric context. A growing body of work has identified the harmful limitations of this gender-exclusive framing; many LLMs cannot correctly and consistently refer to persons outside the gender binary, especially if they use neopronouns. While data scarcity has been identified as a possible culprit, the precise mechanisms through which it influences LLM misgendering remain underexplored. Our work addresses this gap by studying data scarcity's role in subword tokenization and, consequently, the formation of LLM word representations. We uncover how the Byte-Pair Encoding (BPE) tokenizer, a backbone for many popular LLMs, contributes to neopronoun misgendering through out-of-vocabulary behavior. We introduce pronoun tokenization parity (PTP), a novel approach to reduce LLM neopronoun misgendering by preserving a token's functional structure. We evaluate PTP's efficacy using pronoun consistency-based metrics and a novel syntax-based metric. Through several controlled experiments, finetuning LLMs with PTP improves neopronoun consistency from 14.5% to 58.4%, highlighting the significant role tokenization plays in LLM pronoun consistency.

Improving Adversarial Robustness to Sensitivity and Invariance Attacks with Deep Metric Learning

2022-01-01
Anaelia Ovalle , Evan Czyzycki , Cho‐Jui Hsieh
Intentionally crafted adversarial samples have effectively exploited weaknesses in deep neural networks. A standard method in adversarial robustness assumes a framework to defend against samples crafted by minimally perturbing a … Intentionally crafted adversarial samples have effectively exploited weaknesses in deep neural networks. A standard method in adversarial robustness assumes a framework to defend against samples crafted by minimally perturbing a sample such that its corresponding model output changes. These sensitivity attacks exploit the model's sensitivity toward task-irrelevant features. Another form of adversarial sample can be crafted via invariance attacks, which exploit the model underestimating the importance of relevant features. Previous literature has indicated a tradeoff in defending against both attack types within a strictly L_p bounded defense. To promote robustness toward both types of attacks beyond Euclidean distance metrics, we use metric learning to frame adversarial regularization as an optimal transport problem. Our preliminary results indicate that regularizing over invariant perturbations in our framework improves both invariant and sensitivity defense.
View PDF Chat

Auditing Algorithmic Fairness in Machine Learning for Health with Severity-Based LOGAN

2022-01-01
Anaelia Ovalle , Sunipa Dev , Jieyu Zhao +2 more
Auditing machine learning-based (ML) healthcare tools for bias is critical to preventing patient harm, especially in communities that disproportionately face health inequities. General frameworks are becoming increasingly available to measure … Auditing machine learning-based (ML) healthcare tools for bias is critical to preventing patient harm, especially in communities that disproportionately face health inequities. General frameworks are becoming increasingly available to measure ML fairness gaps between groups. However, ML for health (ML4H) auditing principles call for a contextual, patient-centered approach to model assessment. Therefore, ML auditing tools must be (1) better aligned with ML4H auditing principles and (2) able to illuminate and characterize communities vulnerable to the most harm. To address this gap, we propose supplementing ML4H auditing frameworks with SLOGAN (patient Severity-based LOcal Group biAs detectioN), an automatic tool for capturing local biases in a clinical prediction task. SLOGAN adapts an existing tool, LOGAN (LOcal Group biAs detectioN), by contextualizing group bias detection in patient illness severity and past medical history. We investigate and compare SLOGAN's bias detection capabilities to LOGAN and other clustering techniques across patient subgroups in the MIMIC-III dataset. On average, SLOGAN identifies larger fairness disparities in over 75% of patient groups than LOGAN while maintaining clustering quality. Furthermore, in a diabetes case study, health disparity literature corroborates the characterizations of the most biased clusters identified by SLOGAN. Our results contribute to the broader discussion of how machine learning biases may perpetuate existing healthcare disparities.

COVID-19 and Big Data: Multi-faceted Analysis for Spatio-temporal Understanding of the Pandemic with Social Media Conversations

2021-01-01
Shayan Fazeli , Davina Zamanzadeh , Anaelia Ovalle +3 more
COVID-19 has been devastating the world since the end of 2019 and has continued to play a significant role in major national and worldwide events, and consequently, the news. In … COVID-19 has been devastating the world since the end of 2019 and has continued to play a significant role in major national and worldwide events, and consequently, the news. In its wake, it has left no life unaffected. Having earned the world's attention, social media platforms have served as a vehicle for the global conversation about COVID-19. In particular, many people have used these sites in order to express their feelings, experiences, and observations about the pandemic. We provide a multi-faceted analysis of critical properties exhibited by these conversations on social media regarding the novel coronavirus pandemic. We present a framework for analysis, mining, and tracking the critical content and characteristics of social media conversations around the pandemic. Focusing on Twitter and Reddit, we have gathered a large-scale dataset on COVID-19 social media conversations. Our analyses cover tracking potential reports on virus acquisition, symptoms, conversation topics, and language complexity measures through time and by region across the United States. We also present a BERT-based model for recognizing instances of hateful tweets in COVID-19 conversations, which achieves a lower error-rate than the state-of-the-art performance. Our results provide empirical validation for the effectiveness of our proposed framework and further demonstrate that social media data can be efficiently leveraged to provide public health experts with inexpensive but thorough insight over the course of an outbreak.

Harms of Gender Exclusivity and Challenges in Non-Binary Representation in Language Technologies

2021-01-01
Sunipa Dev , Masoud Monajatipoor , Anaelia Ovalle +3 more
Gender is widely discussed in the context of language tasks and when examining the stereotypes propagated by language models. However, current discussions primarily treat gender as binary, which can perpetuate … Gender is widely discussed in the context of language tasks and when examining the stereotypes propagated by language models. However, current discussions primarily treat gender as binary, which can perpetuate harms such as the cyclical erasure of non-binary gender identities. These harms are driven by model and dataset biases, which are consequences of the non-recognition and lack of understanding of non-binary genders in society. In this paper, we explain the complexity of gender and language around it, and survey non-binary persons to understand harms associated with the treatment of gender as binary in English language technologies. We also detail how current language representations (e.g., GloVe, BERT) capture and perpetuate these harms and related challenges that need to be acknowledged and addressed for representations to equitably encode gender information.
View PDF Chat

Harms of Gender Exclusivity and Challenges in Non-Binary Representation in Language Technologies

2021-01-01
Sunipa Dev , Masoud Monajatipoor , Anaelia Ovalle +3 more
Gender is widely discussed in the context of language tasks and when examining the stereotypes propagated by language models. However, current discussions primarily treat gender as binary, which can perpetuate … Gender is widely discussed in the context of language tasks and when examining the stereotypes propagated by language models. However, current discussions primarily treat gender as binary, which can perpetuate harms such as the cyclical erasure of non-binary gender identities. These harms are driven by model and dataset biases, which are consequences of the non-recognition and lack of understanding of non-binary genders in society. In this paper, we explain the complexity of gender and language around it, and survey non-binary persons to understand harms associated with the treatment of gender as binary in English language technologies. We also detail how current language representations (e.g., GloVe, BERT) capture and perpetuate these harms and related challenges that need to be acknowledged and addressed for representations to equitably encode gender information.
View PDF Chat

Applying Deep Learning to Public Health: Using Unbalanced Demographic Data to Predict Thyroid Disorder

2018-11-01
Yasser Attiga , Shih‐Yin Chen , John LaGue +6 more
This study investigates the use of Deep Neural Learning to predict propensity for disease from demographic information alone, with thyroid disease as the test application. The imbalanced dataset of 747,301 … This study investigates the use of Deep Neural Learning to predict propensity for disease from demographic information alone, with thyroid disease as the test application. The imbalanced dataset of 747,301 samples contained 13 demographic predictor variables that were not known to be associated with the disease, and had much missing information. A TensorFlow feed-forward neural network was trained to predict thyroid disease. Different activation functions and a variety of up-sampling and down-sampling methods were employed. The lift statistic was used to evaluate success in identifying patients with a propensity for thyroid disease. The DNN model outperformed the Random Forest model with a 36.63% improvement in the lift statistic. These results suggest that deep learning may be successfully employed to select candidates for early intervention for improved health outcomes, utilizing a large dataset with only minimal demographic variables, similar to datasets that are held by the marketing arms of healthcare providers.
Coauthor Papers Together
Kai-Wei Chang 10
Arjun Subramonian 7
William S. Agnew 5
Luca Soldaini 4
Avijit Ghosh 4
Zeerak Talat 4
Nathaniel Dennler 3
Sunipa Dev 3
Palash Goyal 3
Majid Sarrafzadeh 3
Gilbert C. Gee 3
Jwala Dhamala 3
Martin Mundt 3
Richard S. Zemel 3
Aram Galstyan 3
Irene Font Peradejordi 2
Ashwin Singh 2
Marc Peter Deisenroth 2
María Leonor Pacheco 2
Milind Agarwal 2
Manu Saraswat 2
Natalia Y. Bilenko 2
Filip Klubička 2
Nikhil Vytla 2
Tanvi Anand 2
Alex Markham 2
Yuval Pinter 2
Yanan Long 2
Alicia Boyd 2
Sharvani Jha 2
Jeff M. Phillips 2
Masoud Monajatipoor 2
Claas Voelcker 2
Jaidev Shriram 2
Nyx McLean 2
Huy Tu 2
Maria Ryskina 2
Andrew McNamara 2
Ruchira Ray 2
Raj Korpan 2
Jackie Kay 2
Kyra Yee 2
Maarten Sap 2
Raphael Gontijo-Lopes 2
Luke Stark 2
Ninareh Mehrabi 2
Vagrant Gautam 2
Evyn Dǒng 2
Mayra Russo 2
Aditya Joshi 2