Modeling Emotional Valence Integration From Voice and Touch

In the context of designing multimodal social interactions for Human–Computer Interaction and for Computer–Mediated Communication, we conducted an experimental study to investigate how participants combine voice expressions with tactile stimulation to evaluate emotional valence (EV). In this study, audio and tactile stimuli were presented separately, and then presented together. Audio stimuli comprised positive and negative voice expressions, and tactile stimuli consisted of different levels of air jet tactile stimulation performed on the arm of the participants. Participants were asked to evaluate communicated EV on a continuous scale. Information Integration Theory was used to model multimodal valence perception process. Analyses showed that participants generally integrated both sources of information to evaluate EV. The main integration rule was averaging rule. The predominance of a modality over the other modality was specific to each individual

Exploration of mid-air haptics experience design

Ultrasonic Mid-air Haptics (UMH) is a novel technology that uses the mechanical properties of sound waves to create a pressure point in mid-air. This pressure point, called focal point, can slightly bend the skin and be felt in mid-air without any attachment to the body. This thesis focuses on both studying how to integrate this technology with other senses (i.e. vision and audition) and exploring the range of tactile sensations it can provide. The first two projects presented in this document present the integration of ultrasonic mid-air haptics with audio-visual content. The first project describes the process of creating a unique haptic experience that was part of a six-weeks multisensory exhibition in a museum. The second project moved from the museum to a controlled environment and explored the creation of haptic experiences based on physiologic measurements for six short films. Both studies showed the positive value of adding ultrasonic mid-air haptics to traditional media through higher reported arousal and participants’ high enthusiasm for multisensory content. In the two latter projects of this thesis, it was explored how we could extend the range of possible tactile sensations provided by UMHs. We introduced a new technique called Spatio-Temporal Modulation (STM). It enabled the creation of brand-new tactile experiences, including more salient shapes and wider range of textures. We also provided some guidelines on how to control some of the tactile properties of the sensation, including strength,roughness,or regularity. The findings of those four projects contribute to the growing body of knowledge of UMHs. A summary of the key contributions is provided at the end of the thesis as well as several leads for future works

On the Correlation Between Tactile Stimulation and Pleasantness

: several studies in the affective haptics research field showed the potential of using haptic technology to convey emotions in remote communications. In this context, it is of interest to simplify the haptic feedback without altering the informative content of the stimulus, with a two-fold advantage. on one side, it would allow the development of affective haptic devices whose technological complexity is limited, hence more compatible with wearability and portability requirements. On the other side, having a simplified set of stimuli would decrease the amount of data to be transmitted, thus improving the overall quality of remote haptic interactions. In this work, we investigated the correlation between the parameters regulating a caress-like stimulation and the perceived pleasantness. This was done by means of two experiments, in which we asked subjects to adjust the temperature and the motion velocity of a set of stimuli in order to find the most pleasant combination. results indicated that subjects preferred different values of temperature and velocity of the stimulus depending on the proposed tactile stimulation. a small difference in the pleasantness ratings was observed between caresses provided with linear movements and those given as discrete sequences of taps. In particular, participants preferred linear movements set at 34.5 °C and 3.4 cms-1. As regards caress-like stimuli provided with discrete sequences of taps, the preferred temperature and velocity were 33.2 °C and 2.9 cms-1, respectively. the presence of vibration had a little effect on the perceived pleasantness

Investigating Social Haptic Illusions for Tactile Stroking (SHIFTS)

A common and effective form of social touch is stroking on the forearm. We seek to replicate this stroking sensation using haptic illusions. This work compares two methods that provide sequential discrete stimulation: sequential normal indentation and sequential lateral skin-slip using discrete actuators. Our goals are to understand which form of stimulation more effectively creates a continuous stroking sensation, and how many discrete contact points are needed. We performed a study with 20 participants in which they rated sensations from the haptic devices on continuity and pleasantness. We found that lateral skin-slip created a more continuous sensation, and decreasing the number of contact points decreased the continuity. These results inform the design of future wearable haptic devices and the creation of haptic signals for effective social communication.Comment: To be published in IEEE Haptics Symposium 202

Social touch in human–computer interaction

Touch is our primary non-verbal communication channel for conveying intimate emotions and as such essential for our physical and emotional wellbeing. In our digital age, human social interaction is often mediated. However, even though there is increasing evidence that mediated touch affords affective communication, current communication systems (such as videoconferencing) still do not support communication through the sense of touch. As a result, mediated communication does not provide the intense affective experience of co-located communication. The need for ICT mediated or generated touch as an intuitive way of social communication is even further emphasized by the growing interest in the use of touch-enabled agents and robots for healthcare, teaching, and telepresence applications. Here, we review the important role of social touch in our daily life and the available evidence that affective touch can be mediated reliably between humans and between humans and digital agents. We base our observations on evidence from psychology, computer science, sociology, and neuroscience with focus on the first two. Our review shows that mediated affective touch can modulate physiological responses, increase trust and affection, help to establish bonds between humans and avatars or robots, and initiate pro-social behavior. We argue that ICT mediated or generated social touch can (a) intensify the perceived social presence of remote communication partners and (b) enable computer systems to more effectively convey affective information. However, this research field on the crossroads of ICT and psychology is still embryonic and we identify several topics that can help to mature the field in the following areas: establishing an overarching theoretical framework, employing better research methodologies, developing basic social touch building blocks, and solving specific ICT challenges

Touch Technology in Affective Human, Robot, Virtual-Human Interactions: A Survey

Given the importance of affective touch in human interactions, technology designers are increasingly attempting to bring this modality to the core of interactive technology. Advances in haptics and touch-sensing technology have been critical to fostering interest in this area. In this survey, we review how affective touch is investigated to enhance and support the human experience with or through technology. We explore this question across three different research areas to highlight their epistemology, main findings, and the challenges that persist. First, we review affective touch technology through the human–computer interaction literature to understand how it has been applied to the mediation of human–human interaction and its roles in other human interactions particularly with oneself, augmented objects/media, and affect-aware devices. We further highlight the datasets and methods that have been investigated for automatic detection and interpretation of affective touch in this area. In addition, we discuss the modalities of affective touch expressions in both humans and technology in these interactions. Second, we separately review how affective touch has been explored in human–robot and real-human–virtual-human interactions where the technical challenges encountered and the types of experience aimed at are different. We conclude with a discussion of the gaps and challenges that emerge from the review to steer research in directions that are critical for advancing affective touch technology and recognition systems. In our discussion, we also raise ethical issues that should be considered for responsible innovation in this growing area

Touch Technology in Affective Human-, Robot-, and Virtual-Human Interactions:A Survey

Given the importance of affective touch in human interactions, technology designers are increasingly attempting to bring this modality to the core of interactive technology. Advances in haptics and touch-sensing technology have been critical to fostering interest in this area. In this survey, we review how affective touch is investigated to enhance and support the human experience with or through technology. We explore this question across three different research areas to highlight their epistemology, main findings, and the challenges that persist. First, we review affective touch technology through the human-computer interaction literature to understand how it has been applied to the mediation of human-human interaction and its roles in other human interactions particularly with oneself, augmented objects/media, and affect-aware devices. We further highlight the datasets and methods that have been investigated for automatic detection and interpretation of affective touch in this area. In addition, we discuss the modalities of affective touch expressions in both humans and technology in these interactions. Second, we separately review how affective touch has been explored in human-robot and real-human-virtual-human interactions where the technical challenges encountered and the types of experience aimed at are different. We conclude with a discussion of the gaps and challenges that emerge from the review to steer research in directions that are critical for advancing affective touch technology and recognition systems. In our discussion, we also raise ethical issues that should be considered for responsible innovation in this growing area.</p

Social Touch

Interpersonal or social touch is an intuitive and powerful way to express and communicate emotions, comfort a friend, bond with teammates, comfort a child in pain, and soothe someone who is stressed. If there is one thing that the current pandemic is showing us, it is that social distancing can make some people crave physical interaction through social touch. The notion of “skin-hunger” has become tangible for many.Social touch differs at a functional and anatomical level from discriminative touch, and has clear effects at physiological, emotional, and behavioural levels. Social touch is a topic in psychology (perception, emotion, behaviour), neuroscience (neurophysiological pathways), computer science (mediated touch communication), engineering (haptic devices), robotics (social robots that can touch), humanities (science and technology studies), and sociology (the social implications of touch). Our current scientific knowledge of social touch is scattered across disciplines and not yet adequate for the purpose of meeting today's challenges of connecting human beings through the mediating channel of technology