3 research outputs found
PromptInfuser: How Tightly Coupling AI and UI Design Impacts Designers' Workflows
Prototyping AI applications is notoriously difficult. While large language
model (LLM) prompting has dramatically lowered the barriers to AI prototyping,
designers are still prototyping AI functionality and UI separately. We
investigate how coupling prompt and UI design affects designers' workflows.
Grounding this research, we developed PromptInfuser, a Figma plugin that
enables users to create semi-functional mockups, by connecting UI elements to
the inputs and outputs of prompts. In a study with 14 designers, we compare
PromptInfuser to designers' current AI-prototyping workflow. PromptInfuser was
perceived to be significantly more useful for communicating product ideas, more
capable of producing prototypes that realistically represent the envisioned
artifact, more efficient for prototyping, and more helpful for anticipating UI
issues and technical constraints. PromptInfuser encouraged iteration over
prompt and UI together, which helped designers identify UI and prompt
incompatibilities and reflect upon their total solution. Together, these
findings inform future systems for prototyping AI applications
Recommended from our members
Designing Exploratory Search Systems that Stimulate Memory and Reduce Cognitive Load
From music fans finding new songs in a genre, graphic designers brainstorming ways to depict a message, and journalists scrutinizing documents for angles, people often conduct exploratory searches to understand complex topics. In contrast to traditional search, which is done to quickly answer simple questions, exploratory search is an iterative learning process that involves understanding an information space in order to find useful pieces of information.
Exploratory search is composed of two, closely-related sub-processes: (1) information foraging, choosing sources and collecting information, and (2) sensemaking, organizing this information into a mental framework. Both of these sub-processes are cognitively taxing and heavily rely on our memory. For information foraging, users need to read long, complex resources and recognize useful pieces of information. For sensemaking, as users encounter more information, it becomes harder to relate new information to their current knowledge.
The spreading activation theory of memory purports that the information we encounter materializes in our working memory, which spreads activation into our long-term memory, enabling us to recall related semantic information to make sense of newly found information. From this theory, this thesis introduces three strategies for creating organizations that better stimulate memory: (1) constructing overviews that are association networks that mimic our memory's structure, (2) incorporating our prior knowledge in these overviews, and (3) providing concrete information to help us make sense of abstract ideas. This thesis demonstrates how to employ these strategies through three exploratory search systems across three domains: (A) SymbolFinder helps graphic designers explore visual symbols for abstract concepts, (B) TastePaths helps music fans explore artists within a genre, and (C) AngleKindling supports journalists explore story angles for a press release. Through this body of work, I demonstrate that by designing exploratory search systems to stimulate our memory, we can make acquiring and making sense of knowledge less cognitively demanding
ConstitutionMaker: Interactively Critiquing Large Language Models by Converting Feedback into Principles
Large language model (LLM) prompting is a promising new approach for users to
create and customize their own chatbots. However, current methods for steering
a chatbot's outputs, such as prompt engineering and fine-tuning, do not support
users in converting their natural feedback on the model's outputs to changes in
the prompt or model. In this work, we explore how to enable users to
interactively refine model outputs through their feedback, by helping them
convert their feedback into a set of principles (i.e. a constitution) that
dictate the model's behavior. From a formative study, we (1) found that users
needed support converting their feedback into principles for the chatbot and
(2) classified the different principle types desired by users. Inspired by
these findings, we developed ConstitutionMaker, an interactive tool for
converting user feedback into principles, to steer LLM-based chatbots. With
ConstitutionMaker, users can provide either positive or negative feedback in
natural language, select auto-generated feedback, or rewrite the chatbot's
response; each mode of feedback automatically generates a principle that is
inserted into the chatbot's prompt. In a user study with 14 participants, we
compare ConstitutionMaker to an ablated version, where users write their own
principles. With ConstitutionMaker, participants felt that their principles
could better guide the chatbot, that they could more easily convert their
feedback into principles, and that they could write principles more
efficiently, with less mental demand. ConstitutionMaker helped users identify
ways to improve the chatbot, formulate their intuitive responses to the model
into feedback, and convert this feedback into specific and clear principles.
Together, these findings inform future tools that support the interactive
critiquing of LLM outputs