SU\G(𝔸)/K·U
Sign Up for free Log In

Author Description

Login to generate an author description

Ask a Question About This Mathematician

The Tools and Tactics Used in Intimate Partner Surveillance: An Analysis of Online Infidelity Forums

2020-08-01
Emily Tseng , Rosanna Bellini , Nora McDonald +5 more
Abusers increasingly use spyware apps, account compromise, and social engineering to surveil their intimate partners, causing substantial harms that can culminate in violence. This form of privacy violation, termed intimate … Abusers increasingly use spyware apps, account compromise, and social engineering to surveil their intimate partners, causing substantial harms that can culminate in violence. This form of privacy violation, termed intimate partner surveillance (IPS), is a profoundly challenging problem to address due to the physical access and trust present in the relationship between the target and attacker. While previous research has examined IPS from the perspectives of survivors, we present the first measurement study of online forums in which (potential) attackers discuss IPS strategies and techniques. In domains such as cybercrime, child abuse, and human trafficking, studying the online behaviors of perpetrators has led to better threat intelligence and techniques to combat attacks. We aim to provide similar insights in the context of IPS. We identified five online forums containing discussion of monitoring cellphones and other means of surveilling an intimate partner, including three within the context of investigating relationship infidelity. We perform a mixed-methods analysis of these forums, surfacing the tools and tactics that attackers use to perform surveillance. Via qualitative analysis of forum content, we present a taxonomy of IPS strategies used and recommended by attackers, and synthesize lessons for technologists seeking to curb the spread of IPS.

Participation in the age of foundation models

2024-06-03
Harini Suresh , Emily Tseng , Meg Young +3 more
Growing interest and investment in the capabilities of foundation models has positioned such systems to impact a wide array of public services. Alongside these opportunities is the risk that these … Growing interest and investment in the capabilities of foundation models has positioned such systems to impact a wide array of public services. Alongside these opportunities is the risk that these systems reify existing power imbalances and cause disproportionate harm to marginalized communities. Participatory approaches hold promise to instead lend agency and decision-making power to marginalized stakeholders. But existing approaches in participatory AI/ML are typically deeply grounded in context - how do we apply these approaches to foundation models, which are, by design, disconnected from context? Our paper interrogates this question. First, we examine existing attempts at incorporating participation into foundation models. We highlight the tension between participation and scale, demonstrating that it is intractable for impacted communities to meaningfully shape a foundation model that is intended to be universally applicable. In response, we develop a blueprint for participatory foundation models that identifies more local, application-oriented opportunities for meaningful participation. In addition to the "foundation" layer, our framework proposes the "subfloor'' layer, in which stakeholders develop shared technical infrastructure, norms and governance for a grounded domain, and the "surface'' layer, in which affected communities shape the use of a foundation model for a specific downstream task. The intermediate "subfloor'' layer scopes the range of potential harms to consider, and affords communities more concrete avenues for deliberation and intervention. At the same time, it avoids duplicative effort by scaling input across relevant use cases. Through three case studies in clinical care, financial services, and journalism, we illustrate how this multi-layer model can create more meaningful opportunities for participation than solely intervening at the foundation layer.
View PDF Chat

The Human Factor in AI Red Teaming: Perspectives from Social and Collaborative Computing

2024-11-11
Alice Qian Zhang , Ryland Shaw , Jacy Reese Anthis +7 more
Rapid progress in general-purpose AI has sparked significant interest in "red teaming,'' a practice of adversarial testing originating in military and cybersecurity applications. AI red teaming raises many questions about … Rapid progress in general-purpose AI has sparked significant interest in "red teaming,'' a practice of adversarial testing originating in military and cybersecurity applications. AI red teaming raises many questions about the human factor, such as how red teamers are selected, biases and blindspots in how tests are conducted, and harmful content's psychological effects on red teamers. A growing body of HCI and CSCW literature examines related practices-including data labeling, content moderation, and algorithmic auditing. However, few, if any, have investigated red teaming itself. This workshop seeks to consider the conceptual and empirical challenges associated with this practice, often rendered opaque by non-disclosure agreements. Future studies may explore topics ranging from fairness to mental health and other areas of potential harm. We aim to facilitate a community of researchers and practitioners who can begin to meet these challenges with creativity, innovation, and thoughtful reflection.
View PDF Chat

The Tools and Tactics Used in Intimate Partner Surveillance: An Analysis of Online Infidelity Forums

2020-01-01
Emily Tseng , Rosanna Bellini , Nora McDonald +5 more
Abusers increasingly use spyware apps, account compromise, and social engineering to surveil their intimate partners, causing substantial harms that can culminate in violence. This form of privacy violation, termed intimate … Abusers increasingly use spyware apps, account compromise, and social engineering to surveil their intimate partners, causing substantial harms that can culminate in violence. This form of privacy violation, termed intimate partner surveillance (IPS), is a profoundly challenging problem to address due to the physical access and trust present in the relationship between the target and attacker. While previous research has examined IPS from the perspectives of survivors, we present the first measurement study of online forums in which (potential) attackers discuss IPS strategies and techniques. In domains such as cybercrime, child abuse, and human trafficking, studying the online behaviors of perpetrators has led to better threat intelligence and techniques to combat attacks. We aim to provide similar insights in the context of IPS. We identified five online forums containing discussion of monitoring cellphones and other means of surveilling an intimate partner, including three within the context of investigating relationship infidelity. We perform a mixed-methods analysis of these forums, surfacing the tools and tactics that attackers use to perform surveillance. Via qualitative analysis of forum content, we present a taxonomy of IPS strategies used and recommended by attackers, and synthesize lessons for technologists seeking to curb the spread of IPS.

SoK: Safer Digital-Safety Research Involving At-Risk Users

2023-01-01
Rosanna Bellini , Emily Tseng , Noel Warford +7 more
Research involving at-risk users -- that is, users who are more likely to experience a digital attack or to be disproportionately affected when harm from such an attack occurs -- … Research involving at-risk users -- that is, users who are more likely to experience a digital attack or to be disproportionately affected when harm from such an attack occurs -- can pose significant safety challenges to both users and researchers. Nevertheless, pursuing research in computer security and privacy is crucial to understanding how to meet the digital-safety needs of at-risk users and to design safer technology for all. To standardize and bolster safer research involving such users, we offer an analysis of 196 academic works to elicit 14 research risks and 36 safety practices used by a growing community of researchers. We pair this inconsistent set of reported safety practices with oral histories from 12 domain experts to contribute scaffolded and consolidated pragmatic guidance that researchers can use to plan, execute, and share safer digital-safety research involving at-risk users. We conclude by suggesting areas for future research regarding the reporting, study, and funding of at-risk user research

The Human Factor in AI Red Teaming: Perspectives from Social and Collaborative Computing

2024-07-10
Alice Qian Zhang , Ryland Shaw , Jacy Reese Anthis +7 more
Rapid progress in general-purpose AI has sparked significant interest in "red teaming," a practice of adversarial testing originating in military and cybersecurity applications. AI red teaming raises many questions about … Rapid progress in general-purpose AI has sparked significant interest in "red teaming," a practice of adversarial testing originating in military and cybersecurity applications. AI red teaming raises many questions about the human factor, such as how red teamers are selected, biases and blindspots in how tests are conducted, and harmful content's psychological effects on red teamers. A growing body of HCI and CSCW literature examines related practices-including data labeling, content moderation, and algorithmic auditing. However, few, if any have investigated red teaming itself. Future studies may explore topics ranging from fairness to mental health and other areas of potential harm. We aim to facilitate a community of researchers and practitioners who can begin to meet these challenges with creativity, innovation, and thoughtful reflection.
View PDF Chat

Mitigating Trauma in Qualitative Research Infrastructure: Roles for Machine Assistance and Trauma-Informed Design

2024-12-22
Emily Tseng , Thomas Ristenpart , Nicola Dell
Researchers increasingly look to understand experiences of pain, harm, and marginalization via qualitative analysis. Such work is needed to understand and address social ills, but poses risks to researchers' well-being: … Researchers increasingly look to understand experiences of pain, harm, and marginalization via qualitative analysis. Such work is needed to understand and address social ills, but poses risks to researchers' well-being: sifting through volumes of data on painful human experiences risks incurring traumatic exposure in the researcher. In this paper, we explore how the principles of trauma-informed computing (TIC) can be applied to reimagine healthier tools and workflows for qualitative analysis. We apply TIC to create a design provocation called TIQA, a system for qualitative coding that leverages language modeling, semantic search, and recommendation systems to measure and mitigate an analyst's exposure to concepts they find traumatic. Through a formative study of TIQA with 15 participants, we illuminate the complexities of enacting TIC in qualitative knowledge infrastructure, and potential roles for machine assistance in mitigating researchers' trauma. To assist scholars in translating the high-level principles of TIC into sociotechnical system design, we argue for: (a) a conceptual shift from safety as exposure reduction towards safety as enablement; and (b) renewed attention to evaluating the trauma-informedness of design processes, in tandem with the outcomes of designed objects on users' well-being.
View PDF Chat

"Ownership, Not Just Happy Talk": Co-Designing a Participatory Large Language Model for Journalism

2025-01-28
Emily Tseng , Meg Young , Marianne Aubin Le Quéré +2 more
Journalism has emerged as an essential domain for understanding the uses, limitations, and impacts of large language models (LLMs) in the workplace. News organizations face divergent financial incentives: LLMs already … Journalism has emerged as an essential domain for understanding the uses, limitations, and impacts of large language models (LLMs) in the workplace. News organizations face divergent financial incentives: LLMs already permeate newswork processes within financially constrained organizations, even as ongoing legal challenges assert that AI companies violate their copyright. At stake are key questions about what LLMs are created to do, and by whom: How might a journalist-led LLM work, and what can participatory design illuminate about the present-day challenges about adapting ``one-size-fits-all'' foundation models to a given context of use? In this paper, we undertake a co-design exploration to understand how a participatory approach to LLMs might address opportunities and challenges around AI in journalism. Our 20 interviews with reporters, data journalists, editors, labor organizers, product leads, and executives highlight macro, meso, and micro tensions that designing for this opportunity space must address. From these desiderata, we describe the result of our co-design work: organizational structures and functionality for a journalist-controlled LLM. In closing, we discuss the limitations of commercial foundation models for workplace use, and the methodological implications of applying participatory methods to LLM co-design.
View PDF Chat

Mitigating Trauma in Qualitative Research Infrastructure: Roles for Machine Assistance and Trauma-Informed Design

2025-05-02
Emily Tseng , Thomas Ristenpart , Nicola Dell
Researchers increasingly look to understand experiences of pain, harm, and marginalization via qualitative analysis. Such work is needed to understand and address social ills, but poses risks to researchers' well-being: … Researchers increasingly look to understand experiences of pain, harm, and marginalization via qualitative analysis. Such work is needed to understand and address social ills, but poses risks to researchers' well-being: sifting through volumes of data on painful human experiences risks incurring traumatic exposure in the researcher. In this paper, we explore how the principles of trauma-informed computing (TIC) can be applied to reimagine healthier tools and workflows for qualitative analysis. We apply TIC to create a design provocation called TIQA, a system for qualitative coding that leverages language modeling, semantic search, and recommendation systems to measure and mitigate an analyst's exposure to concepts they find traumatic. Through a formative study of TIQA with 15 participants, we illuminate the complexities of enacting TIC in qualitative knowledge infrastructure, and potential roles for machine assistance in mitigating researchers' trauma. To assist scholars in translating the high-level principles of TIC into sociotechnical system design, we argue for: (a) a conceptual shift from safety as exposure reduction towards safety as enablement; and (b) renewed attention to evaluating the trauma-informedness of design processes, in tandem with the outcomes of designed objects on users' well-being.

Mitigating Trauma in Qualitative Research Infrastructure: Roles for Machine Assistance and Trauma-Informed Design

2025-05-02
Emily Tseng , Thomas Ristenpart , Nicola Dell
Researchers increasingly look to understand experiences of pain, harm, and marginalization via qualitative analysis. Such work is needed to understand and address social ills, but poses risks to researchers' well-being: … Researchers increasingly look to understand experiences of pain, harm, and marginalization via qualitative analysis. Such work is needed to understand and address social ills, but poses risks to researchers' well-being: sifting through volumes of data on painful human experiences risks incurring traumatic exposure in the researcher. In this paper, we explore how the principles of trauma-informed computing (TIC) can be applied to reimagine healthier tools and workflows for qualitative analysis. We apply TIC to create a design provocation called TIQA, a system for qualitative coding that leverages language modeling, semantic search, and recommendation systems to measure and mitigate an analyst's exposure to concepts they find traumatic. Through a formative study of TIQA with 15 participants, we illuminate the complexities of enacting TIC in qualitative knowledge infrastructure, and potential roles for machine assistance in mitigating researchers' trauma. To assist scholars in translating the high-level principles of TIC into sociotechnical system design, we argue for: (a) a conceptual shift from safety as exposure reduction towards safety as enablement; and (b) renewed attention to evaluating the trauma-informedness of design processes, in tandem with the outcomes of designed objects on users' well-being.

"Ownership, Not Just Happy Talk": Co-Designing a Participatory Large Language Model for Journalism

2025-01-28
Emily Tseng , Meg Young , Marianne Aubin Le Quéré +2 more
Journalism has emerged as an essential domain for understanding the uses, limitations, and impacts of large language models (LLMs) in the workplace. News organizations face divergent financial incentives: LLMs already … Journalism has emerged as an essential domain for understanding the uses, limitations, and impacts of large language models (LLMs) in the workplace. News organizations face divergent financial incentives: LLMs already permeate newswork processes within financially constrained organizations, even as ongoing legal challenges assert that AI companies violate their copyright. At stake are key questions about what LLMs are created to do, and by whom: How might a journalist-led LLM work, and what can participatory design illuminate about the present-day challenges about adapting ``one-size-fits-all'' foundation models to a given context of use? In this paper, we undertake a co-design exploration to understand how a participatory approach to LLMs might address opportunities and challenges around AI in journalism. Our 20 interviews with reporters, data journalists, editors, labor organizers, product leads, and executives highlight macro, meso, and micro tensions that designing for this opportunity space must address. From these desiderata, we describe the result of our co-design work: organizational structures and functionality for a journalist-controlled LLM. In closing, we discuss the limitations of commercial foundation models for workplace use, and the methodological implications of applying participatory methods to LLM co-design.
View PDF Chat

Mitigating Trauma in Qualitative Research Infrastructure: Roles for Machine Assistance and Trauma-Informed Design

2024-12-22
Emily Tseng , Thomas Ristenpart , Nicola Dell
Researchers increasingly look to understand experiences of pain, harm, and marginalization via qualitative analysis. Such work is needed to understand and address social ills, but poses risks to researchers' well-being: … Researchers increasingly look to understand experiences of pain, harm, and marginalization via qualitative analysis. Such work is needed to understand and address social ills, but poses risks to researchers' well-being: sifting through volumes of data on painful human experiences risks incurring traumatic exposure in the researcher. In this paper, we explore how the principles of trauma-informed computing (TIC) can be applied to reimagine healthier tools and workflows for qualitative analysis. We apply TIC to create a design provocation called TIQA, a system for qualitative coding that leverages language modeling, semantic search, and recommendation systems to measure and mitigate an analyst's exposure to concepts they find traumatic. Through a formative study of TIQA with 15 participants, we illuminate the complexities of enacting TIC in qualitative knowledge infrastructure, and potential roles for machine assistance in mitigating researchers' trauma. To assist scholars in translating the high-level principles of TIC into sociotechnical system design, we argue for: (a) a conceptual shift from safety as exposure reduction towards safety as enablement; and (b) renewed attention to evaluating the trauma-informedness of design processes, in tandem with the outcomes of designed objects on users' well-being.
View PDF Chat

The Human Factor in AI Red Teaming: Perspectives from Social and Collaborative Computing

2024-11-11
Alice Qian Zhang , Ryland Shaw , Jacy Reese Anthis +7 more
Rapid progress in general-purpose AI has sparked significant interest in "red teaming,'' a practice of adversarial testing originating in military and cybersecurity applications. AI red teaming raises many questions about … Rapid progress in general-purpose AI has sparked significant interest in "red teaming,'' a practice of adversarial testing originating in military and cybersecurity applications. AI red teaming raises many questions about the human factor, such as how red teamers are selected, biases and blindspots in how tests are conducted, and harmful content's psychological effects on red teamers. A growing body of HCI and CSCW literature examines related practices-including data labeling, content moderation, and algorithmic auditing. However, few, if any, have investigated red teaming itself. This workshop seeks to consider the conceptual and empirical challenges associated with this practice, often rendered opaque by non-disclosure agreements. Future studies may explore topics ranging from fairness to mental health and other areas of potential harm. We aim to facilitate a community of researchers and practitioners who can begin to meet these challenges with creativity, innovation, and thoughtful reflection.
View PDF Chat

The Human Factor in AI Red Teaming: Perspectives from Social and Collaborative Computing

2024-07-10
Alice Qian Zhang , Ryland Shaw , Jacy Reese Anthis +7 more
Rapid progress in general-purpose AI has sparked significant interest in "red teaming," a practice of adversarial testing originating in military and cybersecurity applications. AI red teaming raises many questions about … Rapid progress in general-purpose AI has sparked significant interest in "red teaming," a practice of adversarial testing originating in military and cybersecurity applications. AI red teaming raises many questions about the human factor, such as how red teamers are selected, biases and blindspots in how tests are conducted, and harmful content's psychological effects on red teamers. A growing body of HCI and CSCW literature examines related practices-including data labeling, content moderation, and algorithmic auditing. However, few, if any have investigated red teaming itself. Future studies may explore topics ranging from fairness to mental health and other areas of potential harm. We aim to facilitate a community of researchers and practitioners who can begin to meet these challenges with creativity, innovation, and thoughtful reflection.
View PDF Chat

Participation in the age of foundation models

2024-06-03
Harini Suresh , Emily Tseng , Meg Young +3 more
Growing interest and investment in the capabilities of foundation models has positioned such systems to impact a wide array of public services. Alongside these opportunities is the risk that these … Growing interest and investment in the capabilities of foundation models has positioned such systems to impact a wide array of public services. Alongside these opportunities is the risk that these systems reify existing power imbalances and cause disproportionate harm to marginalized communities. Participatory approaches hold promise to instead lend agency and decision-making power to marginalized stakeholders. But existing approaches in participatory AI/ML are typically deeply grounded in context - how do we apply these approaches to foundation models, which are, by design, disconnected from context? Our paper interrogates this question. First, we examine existing attempts at incorporating participation into foundation models. We highlight the tension between participation and scale, demonstrating that it is intractable for impacted communities to meaningfully shape a foundation model that is intended to be universally applicable. In response, we develop a blueprint for participatory foundation models that identifies more local, application-oriented opportunities for meaningful participation. In addition to the "foundation" layer, our framework proposes the "subfloor'' layer, in which stakeholders develop shared technical infrastructure, norms and governance for a grounded domain, and the "surface'' layer, in which affected communities shape the use of a foundation model for a specific downstream task. The intermediate "subfloor'' layer scopes the range of potential harms to consider, and affords communities more concrete avenues for deliberation and intervention. At the same time, it avoids duplicative effort by scaling input across relevant use cases. Through three case studies in clinical care, financial services, and journalism, we illustrate how this multi-layer model can create more meaningful opportunities for participation than solely intervening at the foundation layer.
View PDF Chat

SoK: Safer Digital-Safety Research Involving At-Risk Users

2023-01-01
Rosanna Bellini , Emily Tseng , Noel Warford +7 more
Research involving at-risk users -- that is, users who are more likely to experience a digital attack or to be disproportionately affected when harm from such an attack occurs -- … Research involving at-risk users -- that is, users who are more likely to experience a digital attack or to be disproportionately affected when harm from such an attack occurs -- can pose significant safety challenges to both users and researchers. Nevertheless, pursuing research in computer security and privacy is crucial to understanding how to meet the digital-safety needs of at-risk users and to design safer technology for all. To standardize and bolster safer research involving such users, we offer an analysis of 196 academic works to elicit 14 research risks and 36 safety practices used by a growing community of researchers. We pair this inconsistent set of reported safety practices with oral histories from 12 domain experts to contribute scaffolded and consolidated pragmatic guidance that researchers can use to plan, execute, and share safer digital-safety research involving at-risk users. We conclude by suggesting areas for future research regarding the reporting, study, and funding of at-risk user research

The Tools and Tactics Used in Intimate Partner Surveillance: An Analysis of Online Infidelity Forums

2020-08-01
Emily Tseng , Rosanna Bellini , Nora McDonald +5 more
Abusers increasingly use spyware apps, account compromise, and social engineering to surveil their intimate partners, causing substantial harms that can culminate in violence. This form of privacy violation, termed intimate … Abusers increasingly use spyware apps, account compromise, and social engineering to surveil their intimate partners, causing substantial harms that can culminate in violence. This form of privacy violation, termed intimate partner surveillance (IPS), is a profoundly challenging problem to address due to the physical access and trust present in the relationship between the target and attacker. While previous research has examined IPS from the perspectives of survivors, we present the first measurement study of online forums in which (potential) attackers discuss IPS strategies and techniques. In domains such as cybercrime, child abuse, and human trafficking, studying the online behaviors of perpetrators has led to better threat intelligence and techniques to combat attacks. We aim to provide similar insights in the context of IPS. We identified five online forums containing discussion of monitoring cellphones and other means of surveilling an intimate partner, including three within the context of investigating relationship infidelity. We perform a mixed-methods analysis of these forums, surfacing the tools and tactics that attackers use to perform surveillance. Via qualitative analysis of forum content, we present a taxonomy of IPS strategies used and recommended by attackers, and synthesize lessons for technologists seeking to curb the spread of IPS.

The Tools and Tactics Used in Intimate Partner Surveillance: An Analysis of Online Infidelity Forums

2020-01-01
Emily Tseng , Rosanna Bellini , Nora McDonald +5 more
Abusers increasingly use spyware apps, account compromise, and social engineering to surveil their intimate partners, causing substantial harms that can culminate in violence. This form of privacy violation, termed intimate … Abusers increasingly use spyware apps, account compromise, and social engineering to surveil their intimate partners, causing substantial harms that can culminate in violence. This form of privacy violation, termed intimate partner surveillance (IPS), is a profoundly challenging problem to address due to the physical access and trust present in the relationship between the target and attacker. While previous research has examined IPS from the perspectives of survivors, we present the first measurement study of online forums in which (potential) attackers discuss IPS strategies and techniques. In domains such as cybercrime, child abuse, and human trafficking, studying the online behaviors of perpetrators has led to better threat intelligence and techniques to combat attacks. We aim to provide similar insights in the context of IPS. We identified five online forums containing discussion of monitoring cellphones and other means of surveilling an intimate partner, including three within the context of investigating relationship infidelity. We perform a mixed-methods analysis of these forums, surfacing the tools and tactics that attackers use to perform surveillance. Via qualitative analysis of forum content, we present a taxonomy of IPS strategies used and recommended by attackers, and synthesize lessons for technologists seeking to curb the spread of IPS.
Coauthor Papers Together
Nicola Dell 5
Thomas Ristenpart 5
Mary L. Gray 3
Rosanna Bellini 3
Meg Young 2
Harini Suresh 2
Rachel Greenstadt 2
Julian Posada 2
Nora McDonald 2
Damon McCoy 2
Matan Danos 2
Ram Shankar Siva Kumar 2
Lama Ahmad 2
Jina Suh 2
Ashlee Milton 2
Alice Qian Zhang 2
Ryland Shaw 2
Jill Palzkill Woelfer 1
Jacy Reese Anthis 1
Noel Warford 1
Benjamin Shestakofsky 1
Sunny Consolvo 1
Alaa Daffalla 1
Emma Pierson 1
Michelle L. Mazurek 1
Patrick Gage Kelley 1
Tara Matthews 1
Dana Cuomo 1
Sarah T. Roberts 1
Aimee Rinehart 1
Sarah Roberts 1
Karen Levy 1
Benjamin Shestakofsky 1
Marianne Aubin Le Quéré 1
Jacy Reese Anthis 1