Understanding the role of Interactive Machine Learning in Movement Interaction Design

Interaction based on human movement has the potential to become an important new paradigm of human computer interaction, but for it to become mainstream there need to be effective tools and techniques to support designers. A promising approach to movement interaction design is Interactive Machine Learning, in which designing is done by physically performing . This paper brings together many different perspectives on understand human movement knowledge and movement interaction. This understanding shows that the embodied knowledge involved in movement interaction is very different from the representational knowledge involved in a traditional interface so a very different approach to design is needed. We apply this knowledge to understanding why interactive machine learning is an effective tool for motion interaction designers and to make a number of suggestions for future development of the techniqu

Programming by Moving: Interactive Machine Learning for Embodied Interaction Design

Interactive Machine Learning is a promising approach for designing movement interaction because it allows creators to implement even complex movement designs by simply performing them with their bodies. We introduce a new tool, InteractML and accompanying ideation method, being developed to make movement interaction design faster, adaptable and accessible to creators of varying experience and backgrounds. By closing the gap between ideation and implementation stages of designing movement interaction, we hope to apply embodied sketching all the way through the creation process, supporting the design of more expressive and reflective range of movement interaction

Emergent interfaces: vague, complex, bespoke and embodied interaction between humans and computers

Most human-computer interfaces are built on the paradigm of manipulating abstract representations. This can be limiting when computers are used in artistic performance or as mediators of social connection, where we rely on qualities of embodied thinking: intuition, context, resonance, ambiguity, fluidity. We explore an alternative approach to designing interaction that we call the emergent interface: interaction leveraging unsupervised machine learning to replace designed abstractions with contextually-derived emergent representations. The approach offers opportunities to create interfaces bespoke to a single individual, to continually evolve and adapt the interface in line with that individual’s needs and affordances, and to bridge more deeply with the complex and imprecise interaction that defines much of our non-digital communication. We explore this approach through artistic research rooted in music, dance and AI with the partially emergent system Sonified Body. The system maps the moving body into sound using an emergent representation of the body derived from a corpus of improvised movement from the first author. We explore this system in a residency with three dancers. We reflect on the broader implications and challenges of this alternative way of thinking about interaction, and how far it may help users avoid being limited by the assumptions of a system’s designer

Co-Creating with the Senses: Towards an Embodiment Grammar for Conceptualising Virtual Reality (VR) Narrative Design

This creative practice thesis comprises two components, a dissertation titled Co-Creating with the Senses: Towards an Embodiment Grammar for Conceptualising Virtual Reality (VR) Narrative Design and a creative work, The Recluse, a fictional VR script written in the Maria Vargas Immersive Play template, available through Final Draft. The advent of publicly available virtual reality (VR) technologies has led to the emergence of a new genre of storytelling, henceforth referred to as ‘VR narratives’. There has therefore been a need to articulate its defining grammar and to contribute insights born out of artistic experimentation in a scholarly field which until recently was dominated by scientific points of view. Employing a somaesthetics approach outlined by researcher Kristina Höök, the dissertation draws on a qualitative study into 10 VR narrative works in order to propose an embodiment grammar through which the art form may be conceptualised. The study’s findings, a group of eight embodied states organised into a framework, urge for the relenting of authorial control in order to instead frame affective potential, thus echoing a Deleuzian concept of the assemblage. In particular, the framework draws attention to the way that VR’s deeper affective dimensions may be elucidated by framing co-creation through the medium’s distinct sensory possibilities. As interest gathers in a future metaverse, the insights raised by the study are significant, with potential applications in a range of affective design contexts. The Recluse is my original contribution to this emerging art form and a case study through which to interrogate the framework’s findings. A mystery with supernatural elements, the VR script aims to communicate an experience that transports the participant to the world of Alma Cohen, a famous artist turned recluse, where they are invited to experience the strange occurrences in Alma’s life through their own embodied actions. The VR script explores the potential for intimate encounters with virtual characters and the sensory, co-creational and affective possibilities which arise through these dynamics. It highlights, following the framework, the way that more open structuring approaches are required in order to access the medium’s deeper affective possibilities and also the present technological constraints in achieving this