Design-led approach for transferring the embodied skills of puppet stop-motion animators into haptic workspaces

This design-led research investigates the transfer of puppet stop-motion animators’ embodied skills from the physical workspace into a digital environment. The approach is to create a digital workspace that evokes an embodied animating experience and allows puppet stop-motion animators to work in it unencumbered. The insights and outcomes of the practical explorations are discussed from the perspective of embodied cognition. The digital workspace employs haptic technology, an advanced multi-modal interface technology capable of invoking the tactile, kinaesthetic and proprioceptive senses. The overall aim of this research is to contribute, to the Human-Computer Interaction design community, design considerations and strategies for developing haptic workspaces that can seamlessly transfer and accommodate the rich embodied knowledge of non-digital skillful practitioners. Following an experiential design methodology, a series of design studies in collaboration with puppet stop-motion animators led to the development of a haptic workspace prototype for producing stop-motion animations. Each design study practically explored the transfer of different aspects of the puppet stop-motion animation practice into the haptic workspace. Beginning with an initial haptic workspace prototype, its design was refined in each study with the addition of new functionalities and new interaction metaphors which were always developed with the aim to create and maintain an embodied animating experience. The method of multiple streams of reflection was proposed as an important design tool for identifying, understanding and articulating design insights, empirical results and contextual considerations throughout the design studies. This thesis documents the development of the haptic workspace prototype and discusses the collected design insights and empirical results from the perspective of embodied cognition. In addition, it describes and reviews the design methodology that was adopted as an appropriate approach towards the design of the haptic workspace prototype

An aesthetics of touch: investigating the language of design relating to form

How well can designers communicate qualities of touch? This paper presents evidence that they have some capability to do so, much of which appears to have been learned, but at present make limited use of such language. Interviews with graduate designer-makers suggest that they are aware of and value the importance of touch and materiality in their work, but lack a vocabulary to fully relate to their detailed explanations of other aspects such as their intent or selection of materials. We believe that more attention should be paid to the verbal dialogue that happens in the design process, particularly as other researchers show that even making-based learning also has a strong verbal element to it. However, verbal language alone does not appear to be adequate for a comprehensive language of touch. Graduate designers-makers’ descriptive practices combined non-verbal manipulation within verbal accounts. We thus argue that haptic vocabularies do not simply describe material qualities, but rather are situated competences that physically demonstrate the presence of haptic qualities. Such competencies are more important than groups of verbal vocabularies in isolation. Design support for developing and extending haptic competences must take this wide range of considerations into account to comprehensively improve designers’ capabilities

Tangible user interfaces : past, present and future directions

In the last two decades, Tangible User Interfaces (TUIs) have emerged as a new interface type that interlinks the digital and physical worlds. Drawing upon users' knowledge and skills of interaction with the real non-digital world, TUIs show a potential to enhance the way in which people interact with and leverage digital information. However, TUI research is still in its infancy and extensive research is required in or- der to fully understand the implications of tangible user interfaces, to develop technologies that further bridge the digital and the physical, and to guide TUI design with empirical knowledge. This paper examines the existing body of work on Tangible User In- terfaces. We start by sketching the history of tangible user interfaces, examining the intellectual origins of this field. We then present TUIs in a broader context, survey application domains, and review frame- works and taxonomies. We also discuss conceptual foundations of TUIs including perspectives from cognitive sciences, phycology, and philoso- phy. Methods and technologies for designing, building, and evaluating TUIs are also addressed. Finally, we discuss the strengths and limita- tions of TUIs and chart directions for future research

St. George and the Dragon: Design and production of a cultural heritage museum installation using media archaeology

Media archaeology is a field of media research investigating new media cultures through material manifestations. Although often recognized as an approach to art, its use as an approach to design has not been fully explored. Media archaeology can be valuable because it offers alternative qualities of mediation, as a design palette, to that of prescriptive common media devices. This thesis describes a media archaeological approach towards the design of a cultural heritage media installation, exhibited at Häme Castle between April–December 2017, and produced as a collaboration between the National Museum of Finland (Kansallismuseo) and the Systems of Representation research group in the Department of Media at Aalto University in Finland. The installation displayed a multi-view stereoscopic (3D) digital reconstruction of a medieval sculptural scene of St. George and the Dragon, based on preserved, fragmented medieval sculptures from the museum’s archives. Four stereoscopic video viewers were synchronized to a rotating central physical display, affording visitors an effect of augmented reality, without the need for a mainstream augmented reality implementation. Though the work was time-limited and project-driven, the design approach achieved a well-integrated installation that was sensitive to the aims of an exhibition of sculpture within a cultural heritage museum: artistry, materiality, interpretation. This thesis therefore seeks to argue that media archaeological approaches to design can identify historical ideas that can be remediated into relevancy for new contexts, and, in spite of their historical connotations, foster engaging technological experiences for the contemporary audience, that are sensitive to the aims of an exhibition of cultural heritage

Material agency and performative dynamics in the practices of media art

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel UniversityThis dissertation identifies a strategy of artistic inquiry within contemporary media art practice. It applies the concept of material that acts in an agential capacity, generating performative acts. It argues that the emergent potentials of materials and their interconnectedness with the compositional layers of a work can facilitate modes of effecting change in the artistic system. Through the theoretical investigation of the production processes of physical structures and environments, the thesis focuses on the compositional dynamics within which materials actively perform. It examines how Lars Spuybroek’s architectural design method of Material Machines (2004), and both the tactile potential as well as tactical uses of materials as generators to the formtaking process, might describe an open and active artistic strategy for employing the experimental capacities of such materialization processes. Building on philosophical and conceptual arguments that trace concepts of agency (Bruno Latour’s Actant-Network theory) and enactment (Karen Barad’s concept of intra-acting), the thesis introduces the two installation works ANI_MATE (described as a performative pneumatic stage machine) and ON TRACK (described as a mechanic-robotic installation). These apply the introduced artistic strategies. The analyses of these two artworks traces the particular capacities of the materials involved (respectively, their elasticity or viscosity) to negotiate forces of physical movement, which effect the system to transiently or irreversibly transform. ANI_MATE is a machine that is artist-operated and that explores the relationship between liveanimation procedures and the transformability and flexibility of its material environment. In contrast, ON TRACK’s performative machine ecology removes human agency. The machines act autonomously, giving rise to chance in the artistic system and allowing agency to emerge from the dynamic interconnectivity between materials, parts, and processes, eventually producing an entropic scenario of spilling resources. The thesis concludes that, in the context of a post digital paradigm in-development, such artistic practice offers a new strategy for an emergent aesthetics within contemporary physical-digital performance

To Affinity and Beyond: Interactive Digital Humans as a Human Computer Interface

The field of human computer interaction is increasingly exploring the use of more natural, human-like user interfaces to build intelligent agents to aid in everyday life. This is coupled with a move to people using ever more realistic avatars to represent themselves in their digital lives. As the ability to produce emotionally engaging digital human representations is only just now becoming technically possible, there is little research into how to approach such tasks. This is due to both technical complexity and operational implementation cost. This is now changing as we are at a nexus point with new approaches, faster graphics processing and enabling new technologies in machine learning and computer vision becoming available. I articulate the issues required for such digital humans to be considered successfully located on the other side of the phenomenon known as the Uncanny Valley. My results show that a complex mix of perceived and contextual aspects affect the sense making on digital humans and highlights previously undocumented effects of interactivity on the affinity. Users are willing to accept digital humans as a new form of user interface and they react to them emotionally in previously unanticipated ways. My research shows that it is possible to build an effective interactive digital human that crosses the Uncanny Valley. I directly explore what is required to build a visually realistic digital human as a primary research question and I explore if such a realistic face provides sufficient benefit to justify the challenges involved in building it. I conducted a Delphi study to inform the research approaches and then produced a complex digital human character based on these insights. This interactive and realistic digital human avatar represents a major technical undertaking involving multiple teams around the world. Finally, I explored a framework for examining the ethical implications and signpost future research areas

Aesthetic Animism: Digital Poetry as Ontological Probe

This thesis is about the poetic edge of language and technology. It inter-relates both computational creation and poetic reception by analysing typographic animation softwares and meditating (speculatively) on a future malleable language that possesses the quality of being (and is implicitly perceived as) alive. As such it is a composite document: a philosophical and practice-based exploration of how computers are transforming literature, an ontological meditation on life and language, and a contribution to software studies. Digital poetry introduces animation, dimensionality and metadata into literary discourse. This necessitates new terminology; an acronym for Textual Audio-Visual Interactivity is proposed: Tavit. Tavits (malleable digital text) are tactile and responsive in ways that emulate living entities. They can possess dimensionality, memory, flocking, kinematics, surface reflectivity, collision detection, and responsiveness to touch, etc…. Life-like tactile tavits involve information that is not only semantic or syntactic, but also audible, imagistic and interactive. Reading mediated language-art requires an expanded set of critical, practical and discourse tools, and an awareness of the historical continuum that anticipates this expansion. The ontological and temporal design implications of tavits are supported with case-studies of two commercial typographic-animation softwares and one custom software (Mr Softie created at OBX Labs, Concordia) used during a research-creation process

Skyler and Bliss

Hong Kong remains the backdrop to the science fiction movies of my youth. The city reminds me of my former training in the financial sector. It is a city in which I could have succeeded in finance, but as far as art goes it is a young city, and I am a young artist. A frustration emerges; much like the mould, the artist also had to develop new skills by killing off his former desires and manipulating technology. My new series entitled HONG KONG surface project shows a new direction in my artistic research in which my technique becomes ever simpler, reducing the traces of pixelation until objects appear almost as they were found and photographed. Skyler and Bliss presents tectonic plates based on satellite images of the Arctic. Working in a hot and humid Hong Kong where mushrooms grow ferociously, a city artificially refrigerated by climate control, this series provides a conceptual image of a imaginary typographic map for survival. (Laurent Segretier