Expressive movement generation with machine learning

Movement is an essential aspect of our lives. Not only do we move to interact with our physical environment, but we also express ourselves and communicate with others through our movements. In an increasingly computerized world where various technologies and devices surround us, our movements are essential parts of our interaction with and consumption of computational devices and artifacts. In this context, incorporating an understanding of our movements within the design of the technologies surrounding us can significantly improve our daily experiences. This need has given rise to the field of movement computing – developing computational models of movement that can perceive, manipulate, and generate movements. In this thesis, we contribute to the field of movement computing by building machine-learning-based solutions for automatic movement generation. In particular, we focus on using machine learning techniques and motion capture data to create controllable, generative movement models. We also contribute to the field by creating datasets, tools, and libraries that we have developed during our research. We start our research by reviewing the works on building automatic movement generation systems using machine learning techniques and motion capture data. Our review covers background topics such as high-level movement characterization, training data, features representation, machine learning models, and evaluation methods. Building on our literature review, we present WalkNet, an interactive agent walking movement controller based on neural networks. The expressivity of virtual, animated agents plays an essential role in their believability. Therefore, WalkNet integrates controlling the expressive qualities of movement with the goal-oriented behaviour of an animated virtual agent. It allows us to control the generation based on the valence and arousal levels of affect, the movement’s walking direction, and the mover’s movement signature in real-time. Following WalkNet, we look at controlling movement generation using more complex stimuli such as music represented by audio signals (i.e., non-symbolic music). Music-driven dance generation involves a highly non-linear mapping between temporally dense stimuli (i.e., the audio signal) and movements, which renders a more challenging modelling movement problem. To this end, we present GrooveNet, a real-time machine learning model for music-driven dance generation

The State of the (CHI)Art

We are all researchers, practitioners, and educators - but many of us are also artists, makers, curators. Our arts practice is part of what makes up our sense of self, but also influences our interests and directions in digital and technological enquiry. There exist spaces where the traditional lives alongside the computational, or where the two are blended, no less valid in purpose or value. We seek to investigate this liminal environment, and explore the current state of art in HCI, computer science and other related fields, shifting boundaries as to what "art"is in these spaces. By bringing together like-minded and creative individuals, this workshop aims to both inspire and legitimise our diverse practices, present viewpoints, create meaningful outputs, host discussions, and work toward the future of this plurality

Assessing the reliability of the Laban Movement Analysis system

The Laban Movement Analysis system (LMA) is a widely used system for the description of human movement. Here we present results of an empirical analysis of the reliability of the LMA system. Firstly, we developed a directed graph-based representation for the formalization of LMA. Secondly, we implemented a custom video annotation tool for stimulus presentation and annotation of the formalized LMA. Using these two elements, we conducted an experimental assessment of LMA reliability. In the experimental assessment of the reliability, experts–Certified Movement Analysts (CMA)–were tasked with identifying the differences between a “neutral” movement and the same movement executed with a specific variation in one of the dimensions of the LMA parameter space. The videos represented variations on the pantomimed movement of knocking at a door or giving directions. To be as close as possible to the annotation practice of CMAs, participants were given full control over the number of times and order in which they viewed the videos. The LMA annotation was captured by means of the video annotation tool that guided the participants through the LMA graph by asking them multiple-choice questions at each node. Participants were asked to first annotate the most salient difference (round 1), and then the second most salient one (round 2) between a neutral and gesture and the variation. To quantify the overall reliability of LMA, we computed Krippendorff’s α. The quantitative data shows that the reliability, depending on how the two rounds are integrated, ranges between a weak and an acceptable reliability of LMA. The analysis of viewing behavior showed that, despite relatively large differences at the inter-individual level, there is no simple relationship between viewing behavior and individual performance (quantified as the level of agreement of the individual with the dominant rating). This research advances the state of the art in formalizing and implementing a reliability measure for the Laban Movement Analysis system. The experimental study we conducted allows identifying some of the strengths and weaknesses of the widely used movement coding system. Additionally, we have gained useful insights into the assessment procedure itself

Carrying others: A feminist materialist approach to research-creation

Everyone is connected and operates with or alongside a maternal structure. As psychologist Bracha L. Ettinger states, we all hold within us an imprint or memory of being carried — carried across landscapes, across time, into destinations unknown (Ettinger, 2006). This doctoral dissertation takes up these poetics through an interdisciplinary investigation of Feminist Materialist Research-Creation practices and strategies. Referencing recent traditions of Art Intervention, Performance Art, Land Art, and the canon of feminist art history, this research mirrors, connects with, and critiques digital imaginaries and considers how the maternal body responds to the agency of things in the world. This research makes a unique contribution to the humanities, feminist scholarship, and Research-Creation practices by exploring strategies and subjectivities, new positions of theorization, and analyses that unsettle contemporary approaches to artistic research. This includes a series of theoretical texts, experimental framing, and a portfolio of eight artworks that were individually and collaboratively created and produced between 2016–2019: Traces of Motherhood; Domestic Cupboards; Magical Beast: The Space Within, Out and In-Between, Hunting Self; Mothering Bacteria: The Body as an Interface; Floating in the In-Between; Carrying Others; and Nostalgic Geography: Mama and Papa have Trains, Orchards and Mountains in their Backyard. Showcased with the artwork are digital and technological ephemera, including curatorial conversations, exhibition and submission text, process documentation, links, posters, and other preparatory information. This document also introduces a series of interludes and refections that construct and demonstrate alternative ways of approaching the central ideas, themes, and methodological and theoretical ideas explored in the thesis. Cumulatively, these creative articulations foreground the complexities, process, and nuances of Feminist Materialist approaches to Research-Creation. This document also presents the three main themes which include: 1) Materiality; 2) the Optical Unconscious; and 3) the Technological Unconscious; and, take up the three salient concepts and theories: 1) Carriance; 2) Feminist Materialism; and, 3) Research-Creation. In particular, I argue that Carriance aligns with ideas of care, co-production and becomes a creative way of thinking about connection. Each of the eight artworks demonstrate aspects of Carriance, collaboration, and connection and present emergent ways to consider creative methods, methodologies, and expanded feminist expressions. By discussing a variety of projects and creative forms, this dissertation is a speculative art-making investigation that foregrounds human and non-human relationships, ecofeminist perspectives, and mothering, opening up the term Carriance in a variety of ways to show how it can be more than one method, form, or approach with much potential to challenge, encourage and elicit embodied ways of knowing

La tecnología como extensión de las facultades sensoriales en las artes escénicas

Programa de Doctorado en Humanidades por la Universidad Carlos III de MadridPresidente: Ricardo Iglesias García.- Secretario: Eduardo Blázquez Mateos.- Vocal: Victoria Pérez Arroy

Border Experiences in Europe. Everyday Life - Working Life - Communication - Languages

For a decade now, borders in Europe have been back on the political agenda. Border research has responded and is breaking new ground in thinking about and exploring borders. This book follows this development and strengthens a perspective that is interested in life realities and that focuses on everyday cultural experiences of borders. The authors reconstruct such experiences in the context of different forms of migration and mobility as well as language contact situations and are sensitive to the freedom of the participants. In this way, they empirically identify everyday cultural usage or appropriation strategies of borders as vastly different experiences of borders. The readers of this volume will gain insights into current developments in border research and life realities in Europe where borders are (made) relevant. With contributions by Christian Wille, Birte Nienaber, Carsten Yndigegn, Isabelle Pigeron-Piroth, Rachid Belkacem, Ursula Roos, Elisabeth Boesen, Ariela House, Ignacy Jóźwiak, Corinne Martin, Erika Kalocsányiová, Xosé-Afonso Álvarez, Konstanze Jungbluth, Florian Dost, Nicole Richter, Dominik Gerst.Seit einem Jahrzehnt stehen Grenzen in Europa wieder auf der politischen Agenda. Die Grenzforschung hat darauf reagiert und schlägt neue Wege ein, um Grenzen zu denken und zu untersuchen. Das Buch folgt dieser Entwicklung und macht eine Perspektive stark, die sich für Lebenswirklichkeiten interessiert und die alltagskulturelle Erfahrung der Grenze in den Blick rückt. Die Autor_innen rekonstruieren solche Erfahrungen im Kontext verschiedener Migrations- und Mobilitätsformen sowie Sprachkontaktsituationen und sind sensibel für die Gestaltungsspielräume der Akteure. Auf diese Weise werden alltagskulturelle Gebrauchs- bzw. Aneignungsstrategien von Grenzen als höchst unterschiedliche Erfahrungen der Grenze empirisch herausgearbeitet. Die Leser des Bands bekommen Einblicke in aktuelle Entwicklungen der Grenzforschung und in Lebenswirklichkeiten in Europa, in denen Grenzen relevant (gemacht) werden. Mit Beiträgen von Christian Wille, Birte Nienaber, Carsten Yndigegn, Isabelle Pigeron-Piroth, Rachid Belkacem, Ursula Roos, Elisabeth Boesen, Ariela House, Ignacy Jóźwiak, Corinne Martin, Erika Kalocsányiová, Xosé-Afonso Álvarez, Konstanze Jungbluth, Florian Dost, Nicole Richter, Dominik Gers

Transnational Ties: Australian Lives in the World

Australian lives are intricately enmeshed with the world, bound by ties of allegiance and affinity, intellect and imagination. In Transnational Ties: Australian Lives in the World, an eclectic mix of scholars—historians, literary critics, and museologists—trace the flow of people that helped shape Australia’s distinctive character and the flow of ideas that connected Australians to a global community of thought. It shows how biography, and the study of life stories, can contribute greatly to our understanding of such patterns of connection and explores how transnationalism can test biography’s limits as an intellectual, professional and commercial practice

ENCOUNTERS BETWEEN DANCE AND DIGITAL MEANING DISCOVERING POTENTIAL IN THE QUESTION OF MOVEMENT

This thesis explores how a series of artistic and philosophical enquiries opened up through dancing with Tools that Propel (TTP), the choreographic improvisation system that was developed (partly) in response to the question ‘how might we create real-time interaction between dance and technology that catalyses growth of new embodied knowledge?’ The difficulty of delineating movements – determining where one movement begins and one ends – underpins the functionality of the TTP; its multiple affordances born of, or affected by, different (digital and human) perceptions of movement are products of recursive causality and make the system (which includes the dancer) self-maintaining (Simondon 2012). This thesis examines these affordances and how problematizing the question ‘what is a movement?’ foregrounds a series of interrelated investigations into how and in what form new embodied knowledge arises with TTP. Incorporating text, video, recorded interview and dialogue this thesis is a record of relational encounters of practices: talking (about technical mentality, agency, choreography, improvisation, dramaturgy, archives, time, ontologies and not knowing); co-developing TTP; moving bodies (material, representational, virtual); reading; and writing. It interrogates TTP’s use by dancers in improvisation and performance over two years and navigates an exploded framework of theory to understand the emergent enquiries. Building on the practice of Merce Cunningham, William Forsythe, and Wayne McGregor amongst others, and Mark Coniglio’s concept of ‘digital intervention’, this thesis contributes to knowledge concerning how technological interventions in choreography produce new physical thinking. Drawing on Karen Barad’s agential realism and Jane Bennett’s vital materiality, Andy Clark and David Chalmers’ extended mind theory, and Brian Rotman’s discussions of the para-self and gesturalhaptic writing, this thesis raises questions concerning bio-eco-technological-becoming as it explores how new choreographic language evolves relationally across human and digital perception and how creative potential is found in a recursive and regenerative process of looking within newly perceived movement. Overall, it suggests that an ecological aesthetics (Stern 2018; Yang 2015) approach to dance/technology collaborations can lead not only to new choreographic thinking, attention, and intention in dancers, but to shifted perceptual awareness and enactment of the dancing self in the world