116 research outputs found
VISAR: A Human-AI Argumentative Writing Assistant with Visual Programming and Rapid Draft Prototyping
In argumentative writing, writers must brainstorm hierarchical writing goals,
ensure the persuasiveness of their arguments, and revise and organize their
plans through drafting. Recent advances in large language models (LLMs) have
made interactive text generation through a chat interface (e.g., ChatGPT)
possible. However, this approach often neglects implicit writing context and
user intent, lacks support for user control and autonomy, and provides limited
assistance for sensemaking and revising writing plans. To address these
challenges, we introduce VISAR, an AI-enabled writing assistant system designed
to help writers brainstorm and revise hierarchical goals within their writing
context, organize argument structures through synchronized text editing and
visual programming, and enhance persuasiveness with argumentation spark
recommendations. VISAR allows users to explore, experiment with, and validate
their writing plans using automatic draft prototyping. A controlled lab study
confirmed the usability and effectiveness of VISAR in facilitating the
argumentative writing planning process.Comment: 30 pages, published in UIST'2
PEANUT: A Human-AI Collaborative Tool for Annotating Audio-Visual Data
Audio-visual learning seeks to enhance the computer's multi-modal perception
leveraging the correlation between the auditory and visual modalities. Despite
their many useful downstream tasks, such as video retrieval, AR/VR, and
accessibility, the performance and adoption of existing audio-visual models
have been impeded by the availability of high-quality datasets. Annotating
audio-visual datasets is laborious, expensive, and time-consuming. To address
this challenge, we designed and developed an efficient audio-visual annotation
tool called Peanut. Peanut's human-AI collaborative pipeline separates the
multi-modal task into two single-modal tasks, and utilizes state-of-the-art
object detection and sound-tagging models to reduce the annotators' effort to
process each frame and the number of manually-annotated frames needed. A
within-subject user study with 20 participants found that Peanut can
significantly accelerate the audio-visual data annotation process while
maintaining high annotation accuracy.Comment: 18 pages, published in UIST'2
Structured Generation and Exploration of Design Space with Large Language Models for Human-AI Co-Creation
Thanks to their generative capabilities, large language models (LLMs) have
become an invaluable tool for creative processes. These models have the
capacity to produce hundreds and thousands of visual and textual outputs,
offering abundant inspiration for creative endeavors. But are we harnessing
their full potential? We argue that current interaction paradigms fall short,
guiding users towards rapid convergence on a limited set of ideas, rather than
empowering them to explore the vast latent design space in generative models.
To address this limitation, we propose a framework that facilitates the
structured generation of design space in which users can seamlessly explore,
evaluate, and synthesize a multitude of responses. We demonstrate the
feasibility and usefulness of this framework through the design and development
of an interactive system, Luminate, and a user study with 8 professional
writers. Our work advances how we interact with LLMs for creative tasks,
introducing a way to harness the creative potential of LLMs
UI Layout Generation with LLMs Guided by UI Grammar
The recent advances in Large Language Models (LLMs) have stimulated interest
among researchers and industry professionals, particularly in their application
to tasks concerning mobile user interfaces (UIs). This position paper
investigates the use of LLMs for UI layout generation. Central to our
exploration is the introduction of UI grammar -- a novel approach we proposed
to represent the hierarchical structure inherent in UI screens. The aim of this
approach is to guide the generative capacities of LLMs more effectively and
improve the explainability and controllability of the process. Initial
experiments conducted with GPT-4 showed the promising capability of LLMs to
produce high-quality user interfaces via in-context learning. Furthermore, our
preliminary comparative study suggested the potential of the grammar-based
approach in improving the quality of generative results in specific aspects.Comment: ICML 2023 Workshop on AI and HC
From Awareness to Action: Exploring End-User Empowerment Interventions for Dark Patterns in UX
The study of UX dark patterns, i.e., UI designs that seek to manipulate user
behaviors, often for the benefit of online services, has drawn significant
attention in the CHI and CSCW communities in recent years. To complement
previous studies in addressing dark patterns from (1) the designer's
perspective on education and advocacy for ethical designs; and (2) the
policymaker's perspective on new regulations, we propose an
end-user-empowerment intervention approach that helps users (1) raise the
awareness of dark patterns and understand their underlying design intents; (2)
take actions to counter the effects of dark patterns using a web augmentation
approach. Through a two-phase co-design study, including 5 co-design workshops
(N=12) and a 2-week technology probe study (N=15), we reported findings on the
understanding of users' needs, preferences, and challenges in handling dark
patterns and investigated the feedback and reactions to users' awareness of and
action on dark patterns being empowered in a realistic in-situ setting.Comment: Conditionally Accepted at CSCW 202
Shaping the Emerging Norms of Using Large Language Models in Social Computing Research
The emergence of Large Language Models (LLMs) has brought both excitement and
concerns to social computing research. On the one hand, LLMs offer
unprecedented capabilities in analyzing vast amounts of textual data and
generating human-like responses, enabling researchers to delve into complex
social phenomena. On the other hand, concerns are emerging regarding the
validity, privacy, and ethics of the research when LLMs are involved. This SIG
aims at offering an open space for social computing researchers who are
interested in understanding the impacts of LLMs to discuss their current
practices, perspectives, challenges when engaging with LLMs in their everyday
work and collectively shaping the emerging norms of using LLMs in social
computing research
An Empathy-Based Sandbox Approach to Bridge Attitudes, Goals, Knowledge, and Behaviors in the Privacy Paradox
The "privacy paradox" describes the discrepancy between users' privacy
attitudes and their actual behaviors. Mitigating this discrepancy requires
solutions that account for both system opaqueness and users' hesitations in
testing different privacy settings due to fears of unintended data exposure. We
introduce an empathy-based approach that allows users to experience how privacy
behaviors may alter system outcomes in a risk-free sandbox environment from the
perspective of artificially generated personas. To generate realistic personas,
we introduce a novel pipeline that augments the outputs of large language
models using few-shot learning, contextualization, and chain of thoughts. Our
empirical studies demonstrated the adequate quality of generated personas and
highlighted the changes in privacy-related applications (e.g., online
advertising) caused by different personas. Furthermore, users demonstrated
cognitive and emotional empathy towards the personas when interacting with our
sandbox. We offered design implications for downstream applications in
improving user privacy literacy and promoting behavior changes
- …