Design Resources in Movement-based Design Methods:A Practice-based Characterization

Movement-based design methods are increasingly adopted to help design rich embodied experiences. While there are well-known methods in the field, there is no systematic overview to help designers choose among them, adapt them, or create their own. We collected 41 methods used by movement design researchers and employed a practice-based, bottom-up approach to analyze and characterize their properties. We found 17 categories and arranged them into five main groups: Design Resources, Activities, Delivery, Framing, and Context. In this paper, we describe these groups in general and then focus on Design Resources containing the categories of Movement, Space, and Objects. We ground the characterization with examples from empirical material provided by the design researchers and references to previous work. Additionally, we share recommendations and action points to bring these into practice. This work can help novice and seasoned design researchers who want to employ movement-based design methods in their practice.This research was supported by the EU Erasmus+ project Method Cardsfor Movement-based Interaction Design (2020-1-DK01-KA203-075164) IO4: Gathering movement-frst methods for the design of movement-based experiences. This work was also partially supported by the Madrid Government (Comunidad de Madrid) under the Multiannual Agreement with UC3M in the line of Research Funds for Beatriz Galindo Fellowships (MovIntPlayLab-CM-UC3M 2021/00050/001) in the context of the V PRICIT (Regional Programme of Research and Technological Innovation), and by the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 101002711; project BODYinTRANSIT). We would like to collectively acknowledge all the people and institutions that have made possible the individual projects featured in this paper: ACHIEVE, KOMPAN Workshop, Astaire, Super Trouper (Vetenskapsrådet grant number 2017-04880), Magic outFit (Spanish Agencia Estatal de Investigación, PID2019-105579RBI00), Sense2makeSense (SpanishAgencia Estatal de Investigación, PID2019-109388GB-I00), LearnSPORTtech, Tangibles, DigiFys (Sweden Innovation Agency Vinnova grant number 2016-03777), Diverging Squash, GIFT, and Online Course in Embodied Interaction

Body x Materials: A workshop exploring the role of material-enabled body-based multisensory experiences

Over the last 15 years, HCI and Interaction Design have experienced a “material turn” characterized by a growing interest in the materiality of technology and computation, and in methods that support exploring, envisioning, and crafting with and through materials. The community has experienced a similar turn focused on the body, on how to best design for and from a first-person, lived experience, and the moving and sensual body. In this workshop, we focus on the intersection of these two turns. The emerging developments in multimodal interfaces open opportunities to bring in materiality to the digital world as well as to transform the materiality of objects and bodies in the real-world, including the materiality of our own body. The different sensory qualities of (touchable and untouchable, physical and digital) objects and bodies, including our own, can be brought into the design of digital technologies to enrich, augment, and transform embodied experiences. In this “materials revolution” [15], what are the current theories, approaches, methods, and tools that emphasize the critical role of materiality to body-based interactions with technology? To explore this, in this workshop we will focus on five related themes: material enabling expression, material as a catalyst for human action, material enabling reflection and awareness, material enabling transformation and material supporting the design process for the re-creation of the existing and the yet-to-exist. This workshop with technology presentations, panel sessions with experts, and multidisciplinary discussions will: (i) bring together researchers who work on (re)creating sensory properties of materials through technology with those who investigate experiential effects of materials and materialenabled interactions, (ii) discuss methods, opportunities, difficulties in designing materiality and material-enabled interactions, and (iii) form a multidisciplinary community to build synergies and collaborations

Exploring the Design Space for Body Transformation Wearables to Support Physical Activity through Sensitizing and Bodystorming

Negative or disturbed body perceptions are often interwoven with people's physical inactivity. While wearables can support body perception changes (body transformation), the design space of body transformation wearables supporting physical activity remains narrow. To expand this design space, we conducted an embodied co-design workshop with users. Using conceptual and tangible sensitizing tools, we explored/reflected on bodily sensations at three moments of movement execution (before/during/after). Conceptual tools were used to evoke/reflect/capture past lived experiences, while tangible tools were used as ideation probes for sensory bodystorming. Two design concepts emerged, reflecting diverging approaches to body transformation wearables: one focused on reminders and movement correction; the other on sensory augmentation and facilitation. We reflect on how each facilitates useful representations of body sensations during movement, and present methodological recommendations for designing technology for sensory augmentation in this area. Finally, we propose a preliminary prototype based on our design concepts and discuss future steps

Body x materials: A workshop exploring the role of material-enabled body-based multisensory experiences

Over the last 15 years, HCI and Interaction Design have experienced a “material turn” characterized by a growing interest in the materi- ality of technology and computation, and in methods that support exploring, envisioning, and crafting with and through materials. The community has experienced a similar turn focused on the body, on how to best design for and from a first-person, lived experience, and the moving and sensual body. In this workshop, we focus on the intersection of these two turns. The emerging developments in mul- timodal interfaces open opportunities to bring in materiality to the digital world as well as to transform the materiality of objects and bodies in the real-world, including the materiality of our own bod- ies. The different sensory qualities of (touchable and untouchable, physical and digital) objects and bodies, including our own, can be brought into the design of digital technologies to enrich, augment, and transform embodied experiences. In this “materials revolution” [15], what are the current theories, approaches, methods, and tools that emphasize the critical role of materiality to body-based interac- tions with technology? To explore this, in this workshop we will fo- cus on five related themes: material enabling expression, material as a catalyst for human action, material enabling reflection and aware- ness, material enabling transformation and material supporting the design process for the re-creation of the existing and the yet-to- exist. This workshop with technology presentations, panel sessions with experts, and multidisciplinary discussions will: (i) bring to- gether researchers who work on (re)creating sensory properties of materials through technology with those who investigate expe- riential effects of materials and material-enabled interactions, (ii)discuss methods, opportunities, difficulties in designing materiality and material-enabled interactions, and (iii) form a multidisciplinary community to build synergies and collaborations

Sensory probes:An exploratory design research method for Human-Food Interaction

Designing interactions with food holds potential for rich multisensory experiences but their pervasiveness can challenge our understanding of them. This paper presents the design and evaluation of Sensory probes, a novel, exploratory design research method aimed to sensitize participants towards their food experiences. We report on workshops with 8 participants for co-designing the probes, followed by iterative revision through two-week diary studies with 18 participants. Findings indicate strong engagement with the sensory probes and how they brought forward the bodily and sensory aspects of these experiences, alongside emotional and social ones. We highlight the design rationale for the sensory probes which has been both empirically- and theoretically-grounded, provide reflections on the value of these probes for enabling novel perspectives on food experiences, and on probes’ ability to capture what we called sensory fragments of participants’ experience reflecting distinct sensory aspects form both internal and external senses

Amplifying Actions - Towards Enactive Sound Design

Recently, artists and designers have begun to use digital technologies in order to stimulate bodily interaction, while scientists keep revealing new findings about sensorimotor contingencies, changing the way in which we understand human knowledge. However, implicit knowledge generated in artistic projects can become difficult to transfer and scientific research frequently remains isolated due to specific disciplinary languages and methodologies. By mutually enriching holistic creative approaches and highly specific scientific ways of working, this doctoral dissertation aims to set the foundation for Enactive Sound Design. It is focused on sound that engages sensorimotor experience that has been neglected within the existing design practices. The premise is that such a foundation can be best developed if grounded in transdisciplinary methods that bring together scientific and design approaches. The methodology adopted to achieve this goal is practice-based and supported by theoretical research and project analysis. Three different methodologies were formulated and evaluated during this doctoral study, based on a convergence of existing methods from design, psychology and human-computer interaction. First, a basic design approach was used to engage in a reflective creation process and to extend the existing work on interaction gestalt through hands-on activities. Second, psychophysical experiments were carried out and adapted to suit the needed shift from reception-based tests to a performance-based quantitative evaluation. Last, a set of participatory workshops were developed and conducted, within which the enactive sound exercises were iteratively tested through direct and participatory observation, questionnaires and interviews. A foundation for Enactive Sound Design developed in this dissertation includes novel methods that have been generated by extensive explorations into the fertile ground between basic design education, psychophysical experiments and participatory design. Combining creative practices with traditional task analysis further developed this basic design approach. The results were a number of abstract sonic artefacts conceptualised as the experimental apparatuses that can allow psychologists to study enactive sound experience. Furthermore, a collaboration between designers and scientists on a psychophysical study produced a new methodology for the evaluation of sensorimotor performance with tangible sound interfaces.These performance experiments have revealed that sonic feedback can support enactive learning. Finally, participatory workshops resulted in a number of novel methods focused on a holistic perspective fostered through a subjective experience of self-producing sound. They indicated the influence that such an approach may have on both artists and scientists in the future. The role of designer, as a scientific collaborator within psychological research and as a facilitator of participatory workshops, has been evaluated. Thus, this dissertation recommends a number of collaborative methods and strategies that can help designers to understand and reflectively create enactive sound objects. It is hoped that the examples of successful collaborations between designers and scientists presented in this thesis will encourage further projects and connections between different disciplines, with the final goal of creating a more engaging and a more aware sonic future.European Commission 6th Framework and European Science Foundation (COST Action

Running to Your Own Beat:An Embodied Approach to Auditory Display Design

Personal fitness trackers represent a multi-billion-dollar industry, predicated on devices for assisting users in achieving their health goals. However, most current products only offer activity tracking and measurement of performance metrics, which do not ultimately address the need for technique related assistive feedback in a cost-effective way. Addressing this gap in the design space for assistive run training interfaces is also crucial in combating the negative effects of Forward Head Position, a condition resulting from mobile device use, with a rapid growth of incidence in the population. As such, Auditory Displays (AD) offer an innovative set of tools for creating such a device for runners. ADs present the opportunity to design interfaces which allow natural unencumbered motion, detached from the mobile or smartwatch screen, thus making them ideal for providing real-time assistive feedback for correcting head posture during running. However, issues with AD design have centred around overall usability and user-experience, therefore, in this thesis an ecological and embodied approach to AD design is presented as a vehicle for designing an assistive auditory interface for runners, which integrates seamlessly into their everyday environments

Exploring and designing for multisensory interactions with 3D printed food

Experience of food is as varied as it is widespread, part of mundane activities but also embedded in rituals and celebrations. Despite its pervasive richness it has yet to be fully exploited to support embodied and multisensory experiences within Human-Computer Interaction. This thesis addresses this shortcoming, drawing on the unique qualities of food experience in combination with novel technology to design rich, affective, and embodied interactions through food. This work approaches 3D printed food as a material to design emotion- and memory-based experiences with food, and 3D printing of food as a technology for crafting multisensory user experiences in everyday contexts. These perspectives are integrated through the design and evaluation of novel interactions with 3D printed food, following a Research through Design approach combined with material approaches. Through this enquiry, novel research tools for HCI were also created for working with food, flavour, and taste. The thesis comprises seven studies that advance knowledge, based on gaps identified, and novel theoretical framings in a systematic literature review. Through a survey of user perceptions of 3D printed food, opportunities for user experience-based applications were highlighted. An identified opportunity for affective interactions through taste was considered through lab-based studies and interviews with chefs and food designers on using 3D printed food. This was extended through a co-design study with couples in romantic relationships to create flavours of 3D printed food to support emotional expression and coregulation. The use of flavours to cue experience was then explored in relation to self-defining memories with older adults. Through both co-design studies, a multisensory probe kit was built and evaluated to support designing with the senses in HCI and to further explore ideas from the study into food and memory and an app prototype designed for creating personalised flavour-based memory cues. Collectively, these studies support applications of the 3D printing of food for emotional and memory-based applications in HCI, as well as theoretical and methodological contributions to multisensory design and design with food and the body in HCI

Landscapes of Affective Interaction: Young Children's Enactive Engagement with Body Metaphors

Empirical research into embodied meaning making suggests specific sensorimotor experiences can support children’s understanding of abstract science ideas. This view is aligned with enactive and grounded cognition perspectives, both centred in the view that our ability to conceptualise emerges from our experiences of interaction with our environment. While much of this research has focused on understanding action and action processes in individual children or children in pairs, less attention has been paid to affective dimensions of young children’s group interaction, and how this relates to meaning making with body metaphors. Indeed, Gallagher describes how no action exists in a vacuum, but rather revolves around a complex web of affective-pragmatic features comprising a ‘Landscape of Interaction’ (2020, p.42). This research project addresses gaps in research in understanding young children’s affective engagement from an enactivist cognition perspective. It takes a Design-Based Research approach with an iterative design orientation to examine young children’s interaction with multisensory body-based metaphors through an embodied participation framework. A series of empirical studies with young children, aged 2-7 years, comprising of experiential workshops, build iteratively upon each other. A novel theoretically informed method, Affective Imagination in Motion, is developed involving several purpose-built multisensory body metaphors prompts to enable access to dimensions of young children’s affective engagement. This research makes theoretical and methodological contributions. It extends the theoretical notion of ‘affect’ from enactive and grounded cognition perspectives through identifying key interactive processes in young children’s engagement with multisensory action metaphors. In addition, the novel method offers a contribution as a way of ‘looking’ at affect within a group situation from affective-pragmatic and social embodiment perspectives. Finally, the research contributes to embodied learning design frameworks offering a guideline for designers wishing to inform their work from enactive cognition perspective

Towards Balancing Fun and Exertion in Exergames: Exploring the Impact of Movement-Based Controller Devices, Exercise Concepts, Game Adaptivity and Player Modes on Player Experience and Training Intensity in Different Exergame Settings

Physical inactivity remains one of the biggest societal challenges of the 21st century. The gaming industry and the fitness sector have responded to this alarming fact by introducing game-based or gamified training scenarios and thus established the promising trend of exergaming. Exergames – games controlled by active (whole) body movements – have been extolled as potential attractive and effective training tools. However, the majority of the exergames do not meet the required intensity or effectiveness, nor do they induce the intended training adherence or long-term motivation. One reason for this is that the evaluated exergames were often not co-designed with the user group to meet their specific needs and preferences, nor were they co-designed with an interdisciplinary expert team of game designers (to ensure a good gaming experience) and sports scientists (for a great training experience). Accordingly, the research results from studies with these exergames are rather limited. To fully exploit the potential of these innovative movement tools and to establish them as attractive and effective training approach, it is necessary to understand and explore both the underlying interdisciplinary theories and concepts as well as possible design approaches and their impact on the game and training experience. This dissertation aims to contribute to a better understanding of well-balanced exergame design. It explores and evaluates how different movement-based control devices, exercise concepts, game adaptations, and player modes influence the attractiveness and effectiveness of exergames. The work provides theoretical and practical contributions to the problem area of effective and attractive exergames. For this purpose, a research and development (R&D) approach with iterative phases was followed. As preliminary work for the contributions of this dissertation, exergames were approached from a theoretical perspective. Underlying multidisciplinary theories and concepts of exergames from relevant fields were analyzed and a generic framework was built, which structured the findings based on three interdependent dimensions: the player, the game controller, and the virtual game scenario. Some commercially available exergames were explored to verify the theory-based assumption that the interposition of technology brings specific transformations in the coupling of perception and action that do not occur in real sports situations. Among other things, the comparative pilot study showed that two different controllers (one gesture-based and one haptic device), which allowed for different physical input, were likely to induce diverse gameplay experiences (e.g., higher feeling of flow and self-location when playing with the haptic device) with differently skilled players. However, certain design-specific differences in the two exergame conditions meant that these results could only be interpreted as a first trend. To overcome the limitations of this preliminary study approach (e.g., unequal game design of the commercial exergames and very sports-specific movement concept), Plunder Planet, an adaptive exergame environment, was iteratively designed with and for children and allowed for a single- and cooperative multiplayer experience with two different controller devices. The user-centered design was further informed by insights from the growing body of related R&D work in the field of exergames. The first study presented in this dissertation compared the subjectively experienced attractiveness and effectiveness of Plunder Planet when played with different motion-based controllers. Besides a generally great acceptance of the exergame, it was found that the haptic full-body motion controller provided physical guidance and a more cognitively and coordinatively challenging workout, which was more highly rated by experienced gamers with fewer athletic skills. The gesture-based Kinect sensor felt more natural, allowed more freedom of movement, and provided a rather physically intense but cognitively less challenging workout, which was more highly rated by athletic players with less gameplay experience. Furthermore, experiments were made with an exploratory adaptive algorithm that enabled the cognitive and the physical challenge of the exergame to be manually adapted in real-time based on the player’s fitness and gaming skills. The first and the second study also compared an adaptive with a non-adaptive single player version of Plunder Planet. It could be shown that the (well-balanced) adaptive version of the exergame was better valued than the non-adaptive version with regard to the experienced and measured attractiveness (motivation, game flow, spatial presence experience, balance of cognitive and physical challenge) and effectiveness (heart rate, physical exertion, balance of cognitive and physical challenge) by differently skilled players. Finally, and contrary to the findings from related work, the results of the third study proved that the specifically designed controller technology could be used as an “enabler”, “supporter” and “shaper” of bodily interplay in social exergaming. Based on these promising findings, the goal became to further explore the effectiveness of exergames, refine the adaptive game difficulty algorithm, and explore further attractiveness- and motivation-boosting design approaches. Therefore, the ExerCube, a physically immersive and adaptive fitness game setting, was developed. It was iteratively designed with and for adults and allowed for cooperatively and competitive exergame experiences. With its physically immersive game setup, the ExerCube combines a mixed version of the advantages of both previously tested controllers. A coordinatively and cognitively challenging functional workout protocol with scalable intensity (moderate to high) was developed and the subjective experience of the ExerCube training was compared with a conventional functional training with a personal trainer. The fourth study showed that the game-based training gave signs of reaching a similar intensity to the personal training, but was more highly rated for flow, motivation, and enjoyment. Based on this exploratory comparison of the ExerCube with a personal trainer session, valuable avenues for further design could be identified. Among other things, it could be proved that the player’s focus during the ExerCube session was more on the game than on the own body. Players experienced stronger physical exertion and social pressure with the personal trainer and a stronger cognitive exertion and involvement with the ExerCube. Furthermore, a refined version of the previously tested adaptive game difficulty algorithm was implemented and automated for the first time for purpose of this study. Again it was shown that the adaptive version had benefits with regard to subjectively experienced attractiveness (motivation, game flow, balance of cognitive and physical challenge) and effectiveness (physical exertion, balance of cognitive and physical challenge) compared to the non-adaptive version. In order to further enhance the gaming experience, experiments were also conducted with sound designs and an adaptive audio design with adaptive background music and sound feedback was implemented. It was found to be a promising and beneficial add-on for a user-centered attractive exergame design. To inform the design of a multiplayer version of the ExerCube, different social play mechanics were explored in the fifth study. This resulted in differently balanced experiences of fun, and in physical as well as cognitive exertion. As the preliminary comparative evaluation of the subjectively experienced effectiveness and attractiveness of an ExerCube session and a personal trainer session could prove the general feasibility of the concept and revealed the first indications of the intensity of the ExerCube’s training concept, the objectively measured effectiveness of a single ExerCube session with a functional high-intensity interval training (fHIIT) with a personal trainer was compared in a final sixth study, and after another design iteration. Again, the subjectively experienced attractiveness of both conditions was assessed. It could be shown that the ExerCube is a feasible training device for training at fHIIT-level. While physical exertion was slightly lower than in the conventional fHIIT condition, the ExerCube condition’s average heart rate values reached the fHIIT threshold and also yielded significantly better results for flow, enjoyment, and motivation. The ExerCube training also resulted in a subjectively experienced higher cognitive load (dual-domain training). To sum up, it can be stated that this dissertation provides valuable and fundamental research contributions to the promising field of exergames as attractive and effective training tools. Furthermore, important contributions to design questions in this field could be developed. Since this field is still relatively unexplored, the work presented creates a sound basis for future R&D work in this area