Design of Body-Grounded Tactile Actuators for Playback of Human Physical Contact

We present four wearable tactile actuators capable of recreating physical sensations commonly experienced in human interactions, including tapping on, dragging across, squeezing, and twisting an individual’s wrist. In seeking to create tactile signals that feel natural and are easy to understand, we developed movement control interfaces to play back each of these forms of actual human physical contact. Through iterative design, prototyping, programming, and testing, each of these servo-motor-based mechanisms produces a signal that is gradable in magnitude, can be played in a variety of temporal patterns, is localizable to a small area of skin, and, for three of the four actuators, has an associated direction. Additionally, we have tried to design toward many of the characteristics that have made high frequency vibration the most common form of wearable tactile feedback, including low cost, light weight, comfort, and small size. Bolstered by largely positive comments from naive users during an informal testing session, we plan to continue improving these devices for future use in tactile motion guidance

Master of Science

thesisHaptic interactions with smartphones are generally restricted to vibrotactile feedback that offers limited distinction between delivered tactile cues. The lateral movement of a small, high-friction contactor at the fingerpad can be used to induce skin stretch tangent to the skin's surface. This method has been demonstrated to reliably communicate four cardinal directions with 1 mm translations of the device's contactor, when finger motion is properly restrained. While earlier research has used a thimble to restrain the finger, this interface has been made portable by incorporating a simple conical hole as a finger restraint. An initial portable device design used RC hobby servos and the conical hole finger restraint, but the shape and size of this portable device wasn't compatible with smartphone form factors. This design also had significant compliance and backlash that must be compensated for with additional control schemes. In contrast, this thesis presents the design, fabrication, and testing of a low-profile skin-stretch display (LPSSD) with a novel actuation design for delivering complex tactile cues with minimal backlash or hysteresis of the skin contactor or "tactor." This flatter mechanism features embedded sensors for fingertip cursor control and selection. This device's nonlinear tactor motions are compensated for using table look-up and high-frequency open-loop control to create direction cues with 1.8 mm radial tactor displacements in 16 directions (distributed evenly every 22.5°) before returning to center. Two LPSSDs are incorporated into a smartphone peripheral and used in single-handed and bimanual tests to identify 16 directions. Users also participated in "relative" identification tests where they were first provided a reference direction cue in the forward/north direction followed by the cue direction that they were to identify. Tests were performed with the user's thumbs oriented in the forward direction and with thumbs angled inward slightly, similar to the angledthumb orientation console game controllers. Users are found to have increased performance with an angled-thumb orientation. They performed similarly when stimuli were delivered to their right or left thumbs, and had significantly better performance judging direction cues with both thumbs simultaneously. Participants also performed slightly better in identifying the relative direction cues than the absolute

Haptic Media Scenes

The aim of this thesis is to apply new media phenomenological and enactive embodied cognition approaches to explain the role of haptic sensitivity and communication in personal computer environments for productivity. Prior theory has given little attention to the role of haptic senses in influencing cognitive processes, and do not frame the richness of haptic communication in interaction design—as haptic interactivity in HCI has historically tended to be designed and analyzed from a perspective on communication as transmissions, sending and receiving haptic signals. The haptic sense may not only mediate contact confirmation and affirmation, but also rich semiotic and affective messages—yet this is a strong contrast between this inherent ability of haptic perception, and current day support for such haptic communication interfaces. I therefore ask: How do the haptic senses (touch and proprioception) impact our cognitive faculty when mediated through digital and sensor technologies? How may these insights be employed in interface design to facilitate rich haptic communication? To answer these questions, I use theoretical close readings that embrace two research fields, new media phenomenology and enactive embodied cognition. The theoretical discussion is supported by neuroscientific evidence, and tested empirically through case studies centered on digital art. I use these insights to develop the concept of the haptic figura, an analytical tool to frame the communicative qualities of haptic media. The concept gauges rich machine- mediated haptic interactivity and communication in systems with a material solution supporting active haptic perception, and the mediation of semiotic and affective messages that are understood and felt. As such the concept may function as a design tool for developers, but also for media critics evaluating haptic media. The tool is used to frame a discussion on opportunities and shortcomings of haptic interfaces for productivity, differentiating between media systems for the hand and the full body. The significance of this investigation is demonstrating that haptic communication is an underutilized element in personal computer environments for productivity and providing an analytical framework for a more nuanced understanding of haptic communication as enabling the mediation of a range of semiotic and affective messages, beyond notification and confirmation interactivity

Understanding Hand Interactions and Mid-Air Haptic Responses within Virtual Reality and Beyond.

Hand tracking has long been seen as a futuristic interaction, firmly situated into the realms of sci-fi. Recent developments and technological advancements have brought that dream into reality, allowing for real-time interactions by naturally moving and positioning your hand. While these developments have enabled numerous research projects, it is only recently that businesses and devices are truly starting to implement and integrate the technology into their different sectors. Numerous devices are shifting towards a fully self- contained ecosystem, where the removal of controllers could significantly help in reducing barriers to entry. Prior studies have focused on the effects or possible areas for implementation of hand tracking, but rarely focus on the direct comparisons of technologies, nor do they attempt to reproduce lost capabilities. With this prevailing background, the work presented in this thesis aims to understand the benefits and negatives of hand tracking when treated as the primary interaction method within virtual reality (VR) environments. Coupled with this, the implementation and usage of novel mid-air ultrasound-based haptics attempt to reintroduce feedback that would have been achieved through conventional controller interactions. Two unique user studies were undertaken, testing core underlying interactions within VR that represent common instances found throughout simulations. The first study focuses on the interactions presented within 3D VR user interfaces, with a core topic of buttons. While the second study directly compares input and haptic modalities within two different fine motor skill tasks. These studies are coupled with the development and implementation of a real-time user study recording toolkit, allowing for significantly heightened user analysis and visual evaluation of interactions. Results from these studies and developments make valuable contributions to the research and business knowledge of hand tracking interactions, as well as providing a uniquely valuable open-source toolkit for other researchers to use. This thesis covers work undertaken at Ultraleap over varying projects between 2018 and 2021

Expressing Tacit Material Sensations from a Robo-Sculpting Process by Communicating Shared Haptic Experiences

A sculptor's sense of touch is paramount because we experience sculpting in the iterative process of making new objects. Making sculpture is a process of expressing the inner 'tacit-self' by way of tangible material interactions that become shared artefacts. The existence of tacit- tactile awareness indicates a natural world of personal haptic experience that this thesis will attempt to unpack. Tele-haptic solutions are presented in the form of two robotic sculptures, Touchbot #1 and Touchbot #2. Touchbots (collectively) are the study objects that this practice- based art-research thesis produced, to ask the question: Is it possible to create a machine that could capture and retransmit tacit-tactile experiences within the artistic act of sculpting, through material engagement, from a sculptor's hand to a non-sculptor's hand? Research, conducted and presented, aims to demonstrate that robotic haptic feedback is a vehicle for communicating 'touch' messages through mechanical transmission during sculptural actions (demonstrated through participant interviews and video observation analysis). Additionally, an epistemological context for exploring 'hands-on' knowledge and practice deficits in machine-assisted object modelling is presented including: Michael Polanyi's Tacit Dimension (Polanyi, 2009), David Gooding's Thing Knowledge (Gooding, 2004, p. 1) and Lambros Malafouris' "Material Agency" and material culture (Malafouris, 2008, pp. 19-36). Intersecting bodies of knowledge weave a common thread to support developing a method of communicating tacit sculptural information using haptic touch experience. Unfortunately, there exists more tele-haptics and telerobotics technology for industrial applications than artworks using the same technology. For instance, 'rapid prototyping' technology—such as 3D printers—is removing human tactile-material interaction from object making altogether. In response to the technological obstacle of expanding contemporary interactive sculpture, haptics is applied to include real-time, iterative, robotically assisted object modelling. A review of contemporary haptic technology demonstrates a gap in our understanding iii of embodied knowledge transference. A shortlist of contemporary artists and their works that address the communication of tacit-haptic experiences is also offered, highlighting the importance of exploring embodied knowledge transfer

"Arte Factus" : estudo e co-design socialmente consciente de artefatos digitais socioenativos

Orientador: Maria Cecília Calani BaranauskasTese (doutorado) - Universidade Estadual de Campinas, Instituto de ComputaçãoResumo: Atualmente, a tecnologia computacional tornou-se cada vez mais pervasiva por meio de computadores de diferentes tamanhos, formas e capacidades. Mas avanços tecnológicos, embora necessários, não são suficientes para tornar a interação com tecnologia computacional mais transparente, como preconizado pela computação ubíqua. Sistemas computacionais atuais ainda exigem um vocabulário técnico de entradas e saídas para serem utilizados. No campo da Interação Humano-Computador (IHC), a adoção da teoria da cognição enativa pode lançar luz sobre um novo paradigma de interação que preenche a lacuna entre ação e percepção. Sistemas computacionais enativos são um promissor tema de pesquisa, mas seu design e avaliação ainda são pouco explorados. Além disso, sistemas enativos, como já proposto na literatura, carecem de consideração do contexto social. O objetivo desta tese de doutorado é contribuir para o design de tecnologia computacional dentro de uma abordagem da cognição enativa, além de também sensível à aspectos sociais. Portanto, esta tese investiga os conceitos de sistemas enativos e socioenativos por meio do co-design de arte interativa e instalações. Para atingir esse objetivo, é proposto um arcabouço teórico-metodológico chamado "Arte Factus" para apoiar o estudo e o co-design socialmente consciente de artefatos digitais. O arcabouço "Arte Factus" foi utilizado em três estudos de design relatados nesta tese: InterArt, InstInt e InsTime. Esses estudos envolveram a participação de 105 estudantes de graduação e pós-graduação em Ciência da Computação e Engenharia de Computação no co-design de 19 instalações. O processo envolveu o uso de tecnologia pervasiva do tipo Faça-Você-Mesmo ("Do-It-Yourself, DIY"), e algumas dessas instalações foram estudadas em oficinas de prática situada que ocorreram em cenários educacionais (escola e museu exploratório de ciências). O arcabouço "Arte Factus", como a principal contribuição desta tese de doutorado, mostrou-se eficaz no apoio ao co-design socialmente consciente de instalações interativas que materializam o conceito de artefatos digitais socioenativos. Além disso, através do estudo dos artefatos criados no contexto desta investigação, esta tese também contribui para a construção teórica do conceito de sistemas socioenativosAbstract: Currently, computational technology has become more and more pervasive with computers of different sizes, shapes, and capacities. But technological advancements, although necessary, are not enough to make the interaction with computational technology more transparent, as preconized by the ubiquitous computing. Current computational systems still require a technical vocabulary of inputs and outputs to be interacted with. Within the field of Human-Computer Interaction (HCI), the adoption of the enactive cognition theory can shed light on a new interaction paradigm that bridges the gap between action and perception. Enactive computational systems are a promising subject of research, but their design and evaluation are still hardly explored. Furthermore, enactive systems as already proposed in the literature lack a social context consideration. The objective of this doctoral thesis is to contribute towards the design of computational technology within an enactive approach to cognition, while also being sensitive to social aspects. Therefore, this thesis investigates the concepts of enactive and socioenactive systems by enabling the co-design of interactive art installations. To achieve this objective, a theoretical-methodological framework named "Arte Factus" is proposed to support the study and socially aware co-design of digital artifacts. The "Arte Factus" framework was used in three design studies reported in this thesis: InterArt, InstInt, and InsTime. These studies involved the participation of 105 Computer Science and Computer Engineering undergraduate and graduate students in the co-design of 19 installations. The process involved the use of pervasive "Do-It-Yourself" (DIY) technology, and some of these installations were further studied in workshops of situated practice that took place in educational scenarios (school and exploratory science museum). The "Arte Factus" framework, as the main contribution of this doctoral thesis, has shown effective in supporting the socially aware co-design of interactive installations that materialize the concept of socioenactive digital artifacts. Moreover, through the study of the artifacts created in the context of this investigation, this thesis also contributes towards the theoretical construction of the concept of socioenactive systemsDoutoradoCiência da ComputaçãoDoutor em Ciência da Computação2017/06762-0FAPESPCAPE

The Making of Meaning through Dyadic Haptic Affective Touch

Despite the importance of touch in human-human relations, research in affective tactile practices is in its infancy, lacking in-depth understanding needed to inform the design of remote digital touch communication. This paper reports two qualitative studies that explore tactile affective communication in specific social contexts, and the bi-directional creation, sending and interpretation of digital touch messages using a purpose-built research tool, the Tactile Emoticon. The system comprises a pair of remotely connected mitts which enable users in different locations to communicate through tactile messages, by orchestrating duration and level of three haptic sensations: vibration, pressure and temperature. Qualitative analysis shows the nuanced ways in which 68 participants configured these elements to make meaning from touch messages they sent and received. It points to the affect and emotion of touch, its sensoriality and ambiguity, the significance of context, social norms and expectations of touch participants. Findings suggest key design considerations for digital touch communication, where the emphasis shifts from generating ‘recognizable touches’ to tools that allow people to shape their touches and establish common understanding about their meaning

Expressing Tacit Material Sensations from a Robo-Sculpting Process by Communicating Shared Haptic Experiences

A sculptor's sense of touch is paramount because we experience sculpting in the iterative process of making new objects. Making sculpture is a process of expressing the inner 'tacit-self' by way of tangible material interactions that become shared artefacts. The existence of tacit- tactile awareness indicates a natural world of personal haptic experience that this thesis will attempt to unpack. Tele-haptic solutions are presented in the form of two robotic sculptures, Touchbot #1 and Touchbot #2. Touchbots (collectively) are the study objects that this practice- based art-research thesis produced, to ask the question: Is it possible to create a machine that could capture and retransmit tacit-tactile experiences within the artistic act of sculpting, through material engagement, from a sculptor's hand to a non-sculptor's hand? Research, conducted and presented, aims to demonstrate that robotic haptic feedback is a vehicle for communicating 'touch' messages through mechanical transmission during sculptural actions (demonstrated through participant interviews and video observation analysis). Additionally, an epistemological context for exploring 'hands-on' knowledge and practice deficits in machine-assisted object modelling is presented including: Michael Polanyi's Tacit Dimension (Polanyi, 2009), David Gooding's Thing Knowledge (Gooding, 2004, p. 1) and Lambros Malafouris' "Material Agency" and material culture (Malafouris, 2008, pp. 19-36). Intersecting bodies of knowledge weave a common thread to support developing a method of communicating tacit sculptural information using haptic touch experience. Unfortunately, there exists more tele-haptics and telerobotics technology for industrial applications than artworks using the same technology. For instance, 'rapid prototyping' technology—such as 3D printers—is removing human tactile-material interaction from object making altogether. In response to the technological obstacle of expanding contemporary interactive sculpture, haptics is applied to include real-time, iterative, robotically assisted object modelling. A review of contemporary haptic technology demonstrates a gap in our understanding iii of embodied knowledge transference. A shortlist of contemporary artists and their works that address the communication of tacit-haptic experiences is also offered, highlighting the importance of exploring embodied knowledge transfer