Collaborative storytelling with an embodied conversational agent

Thesis (M.Eng.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2003.Leaves not numbered.Includes bibliographical references (leaves [60]-[64]).This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.When children tell stories to their peers, they naturally collaborate with each other: coauthoring stories, corroborating when in doubt, and acting as active listeners. Their reliance on each other during, as well as the creative process itself, benefits their literacy development. If an interactive system were to engage a child in collaborative narrative, it would be able to exert greater influence over the child's language processes, without becoming overly intrusive as to obstruct his/her natural behaviors. However, due to the spontaneous nature of improvisational play, the problem becomes a challenging one from both a technical, and a behavioral standpoint. This thesis studies children's collaborative behaviors during storytelling and presents a model of the participants' roles, and how to initiate and participate in collaboration with appropriate speech acts and turn-taking cues. Furthermore, it demonstrates how technologies such as speech recognition, natural language processing with commonsense reasoning, multimodal interfaces, and floor management are critical to realizing a real-time collaborative interaction between children and an embodied conversational agent.bu Austin J. Wang.M.Eng

CALYPSO: LLMs as Dungeon Masters' Assistants

The role of a Dungeon Master, or DM, in the game Dungeons & Dragons is to perform multiple tasks simultaneously. The DM must digest information about the game setting and monsters, synthesize scenes to present to other players, and respond to the players' interactions with the scene. Doing all of these tasks while maintaining consistency within the narrative and story world is no small feat of human cognition, making the task tiring and unapproachable to new players. Large language models (LLMs) like GPT-3 and ChatGPT have shown remarkable abilities to generate coherent natural language text. In this paper, we conduct a formative evaluation with DMs to establish the use cases of LLMs in D&D and tabletop gaming generally. We introduce CALYPSO, a system of LLM-powered interfaces that support DMs with information and inspiration specific to their own scenario. CALYPSO distills game context into bite-sized prose and helps brainstorm ideas without distracting the DM from the game. When given access to CALYPSO, DMs reported that it generated high-fidelity text suitable for direct presentation to players, and low-fidelity ideas that the DM could develop further while maintaining their creative agency. We see CALYPSO as exemplifying a paradigm of AI-augmented tools that provide synchronous creative assistance within established game worlds, and tabletop gaming more broadly.Comment: 11 pages, 4 figures. AIIDE 202

Advancing Computational Models of Narrative

Report of a Workshop held at the Wylie Center, Beverly, MA, Oct 8-10 2009Sponsored by the AFOSR under MIT-MURI contract #FA9550-05-1-032

Exploring Improvisational Approaches to Social Knowledge Acquisition

To build agents that can engage user in more open-ended social contexts, more and more attention has been focused on data-driven approaches to reduce the requirement of extensive, hand-authored behavioral content creation. However, one fundamental challenge of data-driven approaches, is acquiring human social interaction data with sufficient variety to capture more open-ended social interactions, as well as their coherency. Previous work has attempted to extract such social knowledge using crowdsourced narratives. This paper proposes an approach to acquire the knowledge of social interaction by integrating an improvisational theatre training technique into a crowdsourcing task aimed at collecting social narratives. The approach emphasizes theory of mind concepts, through an iterative prompting process about the mental states of characters in the narrative and paired writing, in order to encourage the authoring of diverse social interactions. To assess the effectiveness of integrating prompting and two-worker improvisation to the knowledge acquisition process, we systematically compare alternative ways to design the crowdsourcing task, including a) single worker vs. two workers authoring interaction between different characters in a given social context, and b) with or without prompts. Findings from 175 participants across two different social contexts show that the prompts and two-workers collaboration could significantly improve the diversity and the objective coherency of the narratives. The results presented in this paper can provide a rich set of diverse and coherent action sequences to inform the design of socially intelligent agents

Exploring Improvisational Approaches to Social Knowledge Acquisition

To build agents that can engage user in more open-ended social contexts, more and more attention has been focused on data-driven approaches to reduce the requirement of extensive, hand-authored behavioral content creation. However, one fundamental challenge of data-driven approaches, is acquiring human social interaction data with sufficient variety to capture more open-ended social interactions, as well as their coherency. Previous work has attempted to extract such social knowledge using crowdsourced narratives. This paper proposes an approach to acquire the knowledge of social interaction by integrating an improvisational theatre training technique into a crowdsourcing task aimed at collecting social narratives. The approach emphasizes theory of mind concepts, through an iterative prompting process about the mental states of characters in the narrative and paired writing, in order to encourage the authoring of diverse social interactions. To assess the effectiveness of integrating prompting and two-worker improvisation to the knowledge acquisition process, we systematically compare alternative ways to design the crowdsourcing task, including a) single worker vs. two workers authoring interaction between different characters in a given social context, and b) with or without prompts. Findings from 175 participants across two different social contexts show that the prompts and two-workers collaboration could significantly improve the diversity and the objective coherency of the narratives. The results presented in this paper can provide a rich set of diverse and coherent action sequences to inform the design of socially intelligent agents