AI-generated Content for Various Data Modalities: A Survey

AI-generated content (AIGC) methods aim to produce text, images, videos, 3D assets, and other media using AI algorithms. Due to its wide range of applications and the demonstrated potential of recent works, AIGC developments have been attracting lots of attention recently, and AIGC methods have been developed for various data modalities, such as image, video, text, 3D shape (as voxels, point clouds, meshes, and neural implicit fields), 3D scene, 3D human avatar (body and head), 3D motion, and audio -- each presenting different characteristics and challenges. Furthermore, there have also been many significant developments in cross-modality AIGC methods, where generative methods can receive conditioning input in one modality and produce outputs in another. Examples include going from various modalities to image, video, 3D shape, 3D scene, 3D avatar (body and head), 3D motion (skeleton and avatar), and audio modalities. In this paper, we provide a comprehensive review of AIGC methods across different data modalities, including both single-modality and cross-modality methods, highlighting the various challenges, representative works, and recent technical directions in each setting. We also survey the representative datasets throughout the modalities, and present comparative results for various modalities. Moreover, we also discuss the challenges and potential future research directions

The Child in Games: From the Meek, to the Mighty, to the Monstrous

Drawing across game studies, childhood studies, and children’s literature studies, this thesis catalogues and critiques the representation of children in contemporary video games. It poses two questions: 1) How are children represented in contemporary video games? 2) In what ways do the representations of children in video games affirm or challenge dominant Western beliefs about the figure of the child? To answer these questions, I combine a large-scale content analysis of over 500 games published between 2009 and 2019 with a series of autoethnographic close readings. My content analysis is designed to provide a quantitative snapshot of the representation of children in games. I use statistical analysis to assemble data points as meaningful constellations. I use the axes of race, gender, and age, as well as genre, age-rating, and publication year, to identify patterns in representation. I distil my findings as a set of seven archetypes: The Blithe Child, The Heroic Child, The Human Becoming, The Child Sacrifice, The Side Kid, The Waif, and The Little Monster. This typology is not intended to work against the granular detail of the information recorded in the dataset, but to draw attention to patterns of coherence and divergence that occur between particular examples, as well as to intersections with representational tropes about children identified in other media. I select four of these seven archetypes to structure my autoethnographic close readings. While content analysis is a useful tool for documenting the presence, absence, and dominant function of child-characters in games, close reading allows for a more intersectional approach that can attend to the nuances of representation across identity markers, creating opportunities to examine internal contradictions, ironies, and the polysemy generated through interpretive gaps. I develop my own close reading method building on the autoethnographic approaches of Carr (2019), Vossen (2020), McArthur (2018), and Jennings (2021), which I call critical ekphrasis. Chapter one argues that the Blithe Child triangulates ‘children’, ‘toys’, and ‘paidia’. It suggests that both childhood and play can be conceptualised as a ‘magic circle’, and that the immateriality of the Blithe Child implies childhood can be a mode of being unconnected to anatomical markers or chronological age. Chapter two explores how the Heroic Child challenges the apparent affinity between video games and traditional hero 2 narratives. It argues that the dependence of the childly protagonist undermines dualistic thinking and instead celebrates cooperation, compromise, and connection. Chapter three compares the Child Sacrifice to the woman-in-the-refrigerator trope, arguing that it functions to justify aggressive, hypermasculine, militarised violence. The final chapter compares the Little Monster and the Waif to examine how the uncanny child raises metareferential questions about autonomy in interactive media and agency in intergenerational relationships. My research project concludes by suggesting that virtual children in simulated worlds point to the active construction and delimitation of ‘the child’ in society and can reveal that much of what is assumed to be natural, obvious, and universal about the figure of ‘the child’ is in fact ideological. It hints at the possibility that just as virtual children are used as rhetorical figures to explain and justify the rules, mechanics, and moral systems of a digital game, so too is the figure of ‘the child’ used to routinise and vindicate the rules, workings, and moral systems of Euro-American culture.AHR

Paratextualizing Games

Gaming no longer only takes place as a ›closed interactive experience‹ in front of TV screens, but also as broadcast on streaming platforms or as cultural events in exhibition centers and e-sport arenas. The popularization of new technologies, forms of expression, and online services has had a considerable influence on the academic and journalistic discourse about games. This anthology examines which paratexts gaming cultures have produced - i.e., in which forms and formats and through which channels we talk (and write) about games - as well as the way in which paratexts influence the development of games. How is knowledge about games generated and shaped today and how do boundaries between (popular) criticism, journalism, and scholarship have started to blur? In short: How does the paratext change the text

Paratextualizing Games

Gaming no longer only takes place as a ›closed interactive experience‹ in front of TV screens, but also as broadcast on streaming platforms or as cultural events in exhibition centers and e-sport arenas. The popularization of new technologies, forms of expression, and online services has had a considerable influence on the academic and journalistic discourse about games. This anthology examines which paratexts gaming cultures have produced - i.e., in which forms and formats and through which channels we talk (and write) about games - as well as the way in which paratexts influence the development of games. How is knowledge about games generated and shaped today and how do boundaries between (popular) criticism, journalism, and scholarship have started to blur? In short: How does the paratext change the text

Predicting Head Pose From Speech

Speech animation, the process of animating a human-like model to give the impression it is talking, most commonly relies on the work of skilled animators, or performance capture. These approaches are time consuming, expensive, and lack the ability to scale. This thesis develops algorithms for content driven speech animation; models that learn visual actions from data without semantic labelling, to predict realistic speech animation from recorded audio. We achieve these goals by _rst forming a multi-modal corpus that represents the style of speech we want to model; speech that is natural, expressive and prosodic. This allows us to train deep recurrent neural networks to predict compelling animation. We _rst develop methods to predict the rigid head pose of a speaker. Predicting the head pose of a speaker from speech is not wholly deterministic, so our methods provide a large variety of plausible head pose trajectories from a single utterance. We then apply our methods to learn how to predict the head pose of the listener while in conversation, using only the voice of the speaker. Finally, we show how to predict the lip sync, facial expression, and rigid head pose of the speaker, simultaneously, solely from speec

From digital creations of space to analogous experiences of places :living in second life and acting in Flash Mob

PhD ThesisThis dissertation aims to raise the question of how individuals and groups become placed – or take up place – in the contemporary environment and to consider what forms the need for situatedness takes today, by examining the phenomena of the Flash Mob and Second Life. In a Flash Mob, an email activates a virtual community and converts it into a physical performance in the city, challenging a new cognition of place, where place is constituted by the event. On the other hand, Second Life takes the form of a digitally constructed world, which opens the possibility of a “virtual place” that enables users to establish connections not only with each other, but also with the [virtual] environment itself. The two case studies together question place in its materiality and its symbolism, and it is argued that they act as media to re-code “groundedness”. Thus we reach a paradoxical conclusion: although the contemporary world suggests a dynamic and more flexible existence on the earth, the need for “situatedness” and the demand for “well-grounded claims” remain stronger than ever. The structure of this research reflects a double set of conditions that, although not new, have intensified due to the emergence of new technologies: first, the expansion of the human body beyond its corporeal limits and second, the augmentation of the perceived world beyond the mere materiality of any kind of environment. Therefore the thesis studies how, on the one hand, bodies, communities and crowds transform within digitisation, and, on the other, how the world develops as a consequence of the digital reconstruction of grounds. It examines the way in which individuals detach from their “real-world groundedness” by forming bonds-connections to these digitised grounds, which display – as generators of endless possibilities – a kind of utopian openendedness. Finally, it explores the phenomenon of “virtualisation” to raise the question of whether the contemporary world is infused by information and thus augmented in terms of meanings, connections, and attachments, or is instead made of a series of projections, transforming reality into an idealised version of itself.Panayiotis Triantafillidis Foundation,The Greek Ministry of Education: Newcastle University

Environments of Intelligence

What is the role of the environment, and of the information it provides, in cognition? More specifically, may there be a role for certain artefacts to play in this context? These are questions that motivate "4E" theories of cognition (as being embodied, embedded, extended, enactive). In his take on that family of views, Hajo Greif first defends and refines a concept of information as primarily natural, environmentally embedded in character, which had been eclipsed by information-processing views of cognition. He continues with an inquiry into the cognitive bearing of some artefacts that are sometimes referred to as 'intelligent environments'. Without necessarily having much to do with Artificial Intelligence, such artefacts may ultimately modify our informational environments. With respect to human cognition, the most notable effect of digital computers is not that they might be able, or become able, to think but that they alter the way we perceive, think and act. The Open Access version of this book, available at http://www.tandfebooks.com/doi/view/10.4324/9781315401867, has been made available under a Creative Commons CC-BY licenc

The gesturing screen : art and screen agency within postmedia assemblages

This thesis investigates screens as key elements in postmedia assemblages where multiple technical devices and media platforms function relationally to activate new capacities. I characterize the agency of screens as a gesturality that re-arranges and sustains medial relations with and between other components. The gestures of screens reformulate mediality, but also shift experiences and open elements to novel formations and affects. In each re- organisation of the postmedial assemblage, the function and relations of screens are not pre-defined but emerge in process. I develop this dynamic conception of screens and assemblages to account for their diverse manifestations in postmedia. Here I employ an agential realist framework found in the work of Karen Barad and draw on the concept of gestures set out by Giorgio Agamben. This research contributes to a new understanding of postmediality in conjunction with a new conception of the agency of screens. The thesis focuses on mainly digital screen-oriented artworks, repositioning these as heralding or firmly engaging with the postmedial condition. By challenging an understanding of screens that limits them to mere casings for images, this thesis expands the scope and role of screens in postmedia art practices stretching as far back as three decades. It argues that such art works and practices foreground the gesturality of screens and offers in-depth studies of works by Shilpa Gupta, Ulrike Gabriel, Natalie Bookchin, Blast Theory, Ragnar Kjartansson, Sandra Mujinga and Sondra Perry. Such works highlight how screens come to be relationally enacted in postmedia and how that enactment occurs through their performance of medial gestures. I identify two kinds of gestures of postmedia screens that support and connect the technical, aesthetic and in some cases political components of an assemblage. I turn to the multiplicity of frames both on-screen and distributed across screens observed by theorists of media such as Lev Manovich and Anne Friedberg. But the postmedial frame is consistently accompanied by what is out-of-frame – scrolling, swiping and ‘pinching’ continually calls on the out-of-frame to be moved on-screen. The out-of-frame is a postmedial screen gesture, then, that maintains an ongoing relation to the ‘inside’ of the frame, supporting and conditioning it. The multiple temporalities of postmedia assemblages – such as that of images, participants and software – allow an ‘out-of-frame’ to endure beyond the framed image. The second gesture is observed in the pervasiveness of chroma screens — blue and green screens used for compositing other images in postproduction. I suggest that this now ubiquitous technique suspends images from screens. Through a relational analysis of colour, and focusing on Perry’s work, I draw upon ways in which the blankness of chroma screens can be made to gesture a different enactment of race – ‘blackness’ as productive difference. Such gesture in postmedia entails the circulation and transference of social and cultural setting. These two gestures of screens highlight multiple dimensions of the relations of postmedial screens beyond that of framed images, offering us ways to be attentive to enactments of screens as they continue to gather relevance in our expanded visual setting. As screens multiply, this research suggests alternate ways of conceiving the aesthetics and experience of screens’ persistent medial configurations

Discourse in Action

From emails relating to adoption over the Internet to discussions in the airline cockpit, the spoken or written texts we produce can have significant social consequences. The area of Mediated Discourse Analysis considers texts in their social and cultural contexts to explore the actions individuals take with texts - and the consequences of those actions. Discourse in Action: brings together leading scholars from around the world in the area of Mediated Discourse Analysis reveals ways in which its theory and methodology can be used in research into contemporary social situations explores real situations and draws on real data in each chapter shows how analysis of texts in their social contexts broadens our understanding of the real world. Taken together, the chapters provide a comprehensive overview to the field and present a range of current studies that address some of the most important questions facing students and researchers in linguistics, education, communication studies and other fields