Detecting and Grounding Important Characters in Visual Stories

Characters are essential to the plot of any story. Establishing the characters before writing a story can improve the clarity of the plot and the overall flow of the narrative. However, previous work on visual storytelling tends to focus on detecting objects in images and discovering relationships between them. In this approach, characters are not distinguished from other objects when they are fed into the generation pipeline. The result is a coherent sequence of events rather than a character-centric story. In order to address this limitation, we introduce the VIST-Character dataset, which provides rich character-centric annotations, including visual and textual co-reference chains and importance ratings for characters. Based on this dataset, we propose two new tasks: important character detection and character grounding in visual stories. For both tasks, we develop simple, unsupervised models based on distributional similarity and pre-trained vision-and-language models. Our new dataset, together with these models, can serve as the foundation for subsequent work on analysing and generating stories from a character-centric perspective.Comment: AAAI 202

“Game over, man. Game over”:looking at the Alien in film and videogames

In this article we discuss videogame adaptations of the Alien series of films, in particular Alien: Colonial Marines (2013) and Alien: Isolation (2014). In comparing critical responses and developer commentary across these texts, we read the very different affective, aesthetic and socio-political readings of the titular alien character in each case. The significant differences in what it means to ‘look’ at this figure can be analyzed in terms of wider storytelling techniques that stratify remediation between film and games. Differing accounts of how storytelling techniques create intensely ‘immersive’ experiences such as horror and identification—as well as how these experiences are valued—become legible across this set of critical contexts. The concept of the ‘look’ is developed as a comparative series that enables the analysis of the affective dynamics of film and game texts in terms of gender-normative ‘technicity’, moving from the ‘mother monster’ of the original film to the ‘short controlled burst’ of the colonial marines and finally to the ‘psychopathic serendipity’ of Alien: Isolation

Visual Storytelling: Captioning of Image Sequences

In the space of automated captioning, the task of visual storytelling is a dimension. Given sequences of images as inputs, visual storytelling (VIST) is about automatically generating textual narratives as outputs. Automatically producing stories for an order of pictures or video frames have several potential applications in diverse domains ranging from multimedia consumption to autonomous systems. The task has evolved over recent years and is moving into adolescence. The availability of a dedicated VIST dataset for the task has mainstreamed research for visual storytelling and related sub-tasks. This thesis work systematically reports the developments of standard captioning as a parent task with accompanying facets like dense captioning and gradually delves into the domain of visual storytelling. Existing models proposed for VIST are described by examining respective characteristics and scope. All the methods for VIST adapt from the typical encoder-decoder style design, owing to its success in addressing the standard image captioning task. Several subtle differences in the underlying intentions of these methods for approaching the VIST are subsequently summarized. Additionally, alternate perspectives around the existing approaches are explored by re-modeling and modifying their learning mechanisms. Experiments with different objective functions are reported with subjective comparisons and relevant results. Eventually, the sub-field of character relationships within storytelling is studied and a novel idea called character-centric storytelling is proposed to account for prospective characters in the extent of data modalities

Narrative Generation in Entertainment: Using Artificial Intelligence Planning

From the field of artificial intelligence (AI) there is a growing stream of technology capable of being embedded in software that will reshape the way we interact with our environment in our everyday lives. This ‘AI software’ is often used to tackle more mundane tasks that are otherwise dangerous or meticulous for a human to accomplish. One particular area, explored in this paper, is for AI software to assist in supporting the enjoyable aspects of the lives of humans. Entertainment is one of these aspects, and often includes storytelling in some form no matter what the type of media, including television, films, video games, etc. This paper aims to explore the ability of AI software to automate the story-creation and story-telling process. This is part of the field of Automatic Narrative Generator (ANG), which aims to produce intuitive interfaces to support people (without any previous programming experience) to use tools to generate stories, based on their ideas of the kind of characters, intentions, events and spaces they want to be in the story. The paper includes details of such AI software created by the author that can be downloaded and used by the reader for this purpose. Applications of this kind of technology include the automatic generation of story lines for ‘soap operas’

Emotional characters for automatic plot creation

The Virtual Storyteller is a multi-agent framework for automatic story generation. In this paper we describe how plots emerge from the actions of semi-autonomous character agents, focusing on the influence of the characters’ emotions on plot development

Learning in a Flash

Text is no longer the primary means of learning transfer. Character-based simulation, in which animated characters provide a social context that motivates learners, can improve cognition and recall and bodes well for high-impact e-learning

Clarifying creative nonfiction through the personal essay

In a recent issue of TEXT, Matthew Ricketson sought to clarify the &lsquo;boundaries between fiction and nonfiction&rsquo;.&nbsp;In his capacity as a teacher of the creative nonfiction form he writes, &lsquo;I have lost count of the number of times, in classes and in submitted work, that students have described a piece of nonfiction as a novel&rsquo;. The confusion thus highlighted is not restricted to Ricketson&rsquo;s journalism students. In our own university&rsquo;s creative writing cohort, students also struggle with difficulties in melding the research methodology of the journalist with the language and form of creative writing required to produce nonfiction stories for a 21st century readership.Currently in Australia creative nonfiction is enthusiastically embraced by publishers and teaching institutions. Works of memoir proliferate in the lists of mainstream publishers, as do anthologies of the essay form. During a time of increasing competition and desire for differentiation between institutions, when graduate outcomes form a basis for marketing university degrees, it is hardly surprising that, increasingly, tertiary writing teachers focus on this genre in their writing programs. A second tension has arisen in higher education more generally, which affects our writing students&rsquo; approaches to tertiary study. The student writers of the 21st century emerge from a digitally literate and socially collaborative generation: the NetGen(eration). From a learner-centric viewpoint, they could be described as time-poor, and motivated by work-integrated learning with its perceived close links to workplace contexts and to writing genres. They seek just-in-time learning to meet their immediate employment needs, which inhibits the development of their capacity to adapt their researching and writing to various genres and audiences. This article examines issues related to moving these NetGen student writers into the demanding and rapidly expanding creative nonfiction market. It is form rather than genre that denotes creative nonfiction and, we argue, it is the unique features of the personal essay, based as it is on doubt, discovery and the writer&rsquo;s personal voice that can be instrumental in teaching creative nonfiction writing to our digitally and socially literate cohort of students.<br /

Narratology

This essay provides an overview of the different types of study that can be conducted when considering the narrative aspects of video game play. It contextualizes this research among the larger movements of narratology, particularly concerning the structuralist roots of the discipline and the parallels between gameplay and narrative structures. A brief overview of the key points of the ludology/narratology debate is made, followed by an introduction to the three domains of narrative in video game studies: story content, story structures, and narration as the discursive mode that games use to relay the game-state