Responsible AI Research Needs Impact Statements Too

All types of research, development, and policy work can have unintended, adverse consequences - work in responsible artificial intelligence (RAI), ethical AI, or ethics in AI is no exception

Affective Conversational Agents: Understanding Expectations and Personal Influences

The rise of AI conversational agents has broadened opportunities to enhance human capabilities across various domains. As these agents become more prevalent, it is crucial to investigate the impact of different affective abilities on their performance and user experience. In this study, we surveyed 745 respondents to understand the expectations and preferences regarding affective skills in various applications. Specifically, we assessed preferences concerning AI agents that can perceive, respond to, and simulate emotions across 32 distinct scenarios. Our results indicate a preference for scenarios that involve human interaction, emotional support, and creative tasks, with influences from factors such as emotional reappraisal and personality traits. Overall, the desired affective skills in AI agents depend largely on the application's context and nature, emphasizing the need for adaptability and context-awareness in the design of affective AI conversational agents

ICE: Enabling Non-Experts to Build Models Interactively for Large-Scale Lopsided Problems

Quick interaction between a human teacher and a learning machine presents numerous benefits and challenges when working with web-scale data. The human teacher guides the machine towards accomplishing the task of interest. The learning machine leverages big data to find examples that maximize the training value of its interaction with the teacher. When the teacher is restricted to labeling examples selected by the machine, this problem is an instance of active learning. When the teacher can provide additional information to the machine (e.g., suggestions on what examples or predictive features should be used) as the learning task progresses, then the problem becomes one of interactive learning. To accommodate the two-way communication channel needed for efficient interactive learning, the teacher and the machine need an environment that supports an interaction language. The machine can access, process, and summarize more examples than the teacher can see in a lifetime. Based on the machine's output, the teacher can revise the definition of the task or make it more precise. Both the teacher and the machine continuously learn and benefit from the interaction. We have built a platform to (1) produce valuable and deployable models and (2) support research on both the machine learning and user interface challenges of the interactive learning problem. The platform relies on a dedicated, low-latency, distributed, in-memory architecture that allows us to construct web-scale learning machines with quick interaction speed. The purpose of this paper is to describe this architecture and demonstrate how it supports our research efforts. Preliminary results are presented as illustrations of the architecture but are not the primary focus of the paper

Large Language Models Produce Responses Perceived to be Empathic

Large Language Models (LLMs) have demonstrated surprising performance on many tasks, including writing supportive messages that display empathy. Here, we had these models generate empathic messages in response to posts describing common life experiences, such as workplace situations, parenting, relationships, and other anxiety- and anger-eliciting situations. Across two studies (N=192, 202), we showed human raters a variety of responses written by several models (GPT4 Turbo, Llama2, and Mistral), and had people rate these responses on how empathic they seemed to be. We found that LLM-generated responses were consistently rated as more empathic than human-written responses. Linguistic analyses also show that these models write in distinct, predictable ``styles", in terms of their use of punctuation, emojis, and certain words. These results highlight the potential of using LLMs to enhance human peer support in contexts where empathy is important

IMBUE: Improving Interpersonal Effectiveness through Simulation and Just-in-time Feedback with Human-Language Model Interaction

Navigating certain communication situations can be challenging due to individuals' lack of skills and the interference of strong emotions. However, effective learning opportunities are rarely accessible. In this work, we conduct a human-centered study that uses language models to simulate bespoke communication training and provide just-in-time feedback to support the practice and learning of interpersonal effectiveness skills. We apply the interpersonal effectiveness framework from Dialectical Behavioral Therapy (DBT), DEAR MAN, which focuses on both conversational and emotional skills. We present IMBUE, an interactive training system that provides feedback 25% more similar to experts' feedback, compared to that generated by GPT-4. IMBUE is the first to focus on communication skills and emotion management simultaneously, incorporate experts' domain knowledge in providing feedback, and be grounded in psychology theory. Through a randomized trial of 86 participants, we find that IMBUE's simulation-only variant significantly improves participants' self-efficacy (up to 17%) and reduces negative emotions (up to 25%). With IMBUE's additional just-in-time feedback, participants demonstrate 17% improvement in skill mastery, along with greater enhancements in self-efficacy (27% more) and reduction of negative emotions (16% more) compared to simulation-only. The improvement in skill mastery is the only measure that is transferred to new and more difficult situations; situation specific training is necessary for improving self-efficacy and emotion reduction

Youth WellTech: A Global Remote Co-Design Sprint for Youth Mental Health Technology

Mental health is a pressing concern in today's digital age, particularly among youth who are deeply intertwined with technology. Despite the influx of technology solutions addressing mental health issues, youth often remain sidelined during the design process. While co-design methods have been employed to improve participation by youth, many such initiatives are limited to design activities and lack training for youth to research and develop solutions for themselves. In this case study, we detail our 8-week remote, collaborative research initiative called Youth WellTech, designed to facilitate remote co-design sprints aimed at equipping youth with the tools and knowledge to envision and design tech futures for their own communities. We pilot this initiative with 12 student technology evangelists across 8 countries globally to foster the sharing of mental health challenges and diverse perspectives. We highlight insights from our experiences running this global program remotely, its structure, and recommendations for co-research.Comment: Case Study, 13 page

From User Surveys to Telemetry-Driven Agents: Exploring the Potential of Personalized Productivity Solutions

We present a comprehensive, user-centric approach to understand preferences in AI-based productivity agents and develop personalized solutions tailored to users' needs. Utilizing a two-phase method, we first conducted a survey with 363 participants, exploring various aspects of productivity, communication style, agent approach, personality traits, personalization, and privacy. Drawing on the survey insights, we developed a GPT-4 powered personalized productivity agent that utilizes telemetry data gathered via Viva Insights from information workers to provide tailored assistance. We compared its performance with alternative productivity-assistive tools, such as dashboard and narrative, in a study involving 40 participants. Our findings highlight the importance of user-centric design, adaptability, and the balance between personalization and privacy in AI-assisted productivity tools. By building on the insights distilled from our study, we believe that our work can enable and guide future research to further enhance productivity solutions, ultimately leading to optimized efficiency and user experiences for information workers

Emerging Perspectives in Human-Centered Machine Learning

Current Machine Learning (ML) models can make predictions that are as good as or better than those made by people. The rapid adoption of this technology puts it at the forefront of systems that impact the lives of many, yet the consequences of this adoption are not fully understood. Therefore, work at the intersection of people's needs and ML systems is more relevant than ever. This area of work, dubbed Human-Centered Machine Learning (HCML), re-thinks ML research and systems in terms of human goals. HCML gathers an interdisciplinary group of HCI and ML practitioners, each bringing their unique, yet related perspectives. This one-day workshop is a successor of Gillies et al. (2016) and focuses on recent advancements and emerging areas in HCML. We aim to discuss different perspectives on these areas and articulate a coordinated research agenda for the XXI century

Identifying a low-risk group for parametrial involvement in microscopic Stage IB1 cervical cancer using criteria from ongoing studies and a new MRI criterion

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited.Abstract Background There are currently three ongoing studies on less radical surgery in cervical cancer: ConCerv, GOG-278, and SHAPE. The aim of this study was to evaluate the performance of the criteria used in ongoing studies retrospectively and suggest a new, simplified criterion in microscopic Stage IB1 cervical cancer. Methods A retrospective analysis was performed in 125 Stage IB1 cervical cancer patients who had no clinically visible lesions and were allotted based on microscopic findings after conization. All patients had magnetic resonance imaging (MRI) after conization and underwent type C2 radical hysterectomy. We suggested an MRI criterion for less radical surgery candidates as patients who had no demonstrable lesions on MRI. The rates of parametrial involvement (PMI) were estimated for patients that satisfied the inclusion criteria for ongoing studies and the MRI criterion. Results The rate of pathologic PMI was 5.6% (7/125) in the study population. ConCerv and GOG-278 identified 11 (8.8%) and 14 (11.2%) patients, respectively, as less radical surgery candidates, and there were no false negative cases. SHAPE and MRI criteria identified 78 (62.4%) and 74 (59.2%) patients, respectively, as less radical surgery candidates; 67 patients were identified as less radical surgery candidates by both sets of criteria. Of these 67 patients, only one had pathologic PMI with tumor emboli. Conclusions This study suggests that the criteria used in three ongoing studies and a new, simplified criterion using MRI can identify candidates for less radical surgery with acceptable false negativity in microscopic Stage IB1 disease

Human-Centered and Computational Understanding for the Design and Adaptation of Mental Health and Well-being Interventions

Thesis (Ph.D.)--University of Washington, 2022As many as 20% of Americans suffer from diagnosable mental health disorders, but those overwhelmed with physiological and economic burdens cannot prioritize seeking support for their mental health and well-being. There are many evidence-based psychosocial interventions (EBPIs) that have been proven to be effective in treating mental health conditions. Recent initiatives to improve engagement in mental health care through technology have generated an abundance of promising digital mental health solutions. However, symptoms of stress, anxiety, and depression remain overlooked and in constant tension with life demands and disruptions, making it challenging to integrate such solutions into everyday life. My dissertation research examines the tensions between everyday life demands and mental health and well-being, where I design systems that integrate adaptations of EBPIs into everyday contexts to promote engagement. My work intersects three well-being contexts: (1) the COVID-19 pandemic, (2) co-morbid cancer and depression, and (3) workplace stress. First, I examine the situated contexts using human-centered and computational methods grounded on holistic frameworks to reveal challenges rooted in tensions among multiple needs that get in the way of engaging in mental health and well-being activities. I conduct this research in the COVID-19 pandemic and co-morbid cancer and depression contexts to demonstrate that these challenges are present at the individual, organizational, and population scales. Second, I identify modification targets to existing evidence-based psychosocial interventions that can be enhanced through the use of technology to ease the tensions among needs and to directly integrate adapted interventions into the relevant contexts. I describe the development of the collaborative behavioral activation system aimed at improving the collaboration and engagement of patients and providers in depression care. I also describe the development of a just-in-time micro-intervention system aimed at reducing stress in the workplace. Lastly, I deploy these technology-enhanced mental health and well-being systems in real-world contexts to evaluate their effectiveness in improving engagement. Through such deployment, I highlight implementation challenges to integrating patient-provider collaborative technology into a clinical care practice as well as individual, contextual, and intervention-related factors that may influence real-time engagement in digitized interventions. Across three well-being contexts, my dissertation demonstrates that contextual and continuous adaptations of EBPIs can improve engagement in mental health and well-being care. My dissertation makes theoretical contributions through the development of holistic frameworks, methodological contributions through the development of computational frameworks, and artifact contributions through the development of technology-enhanced mental health and well-being intervention systems and through the design recommendations that arise from real-world deployments