Emotional self-regulation of individuals with autism spectrum disorders: smartwatches for monitoring and interaction

In this paper, we analyze the needs of individuals with Autism Spectrum Disorders (ASD) to have a pervasive, feasible and non-stigmatizing form of assistance in their emotional self-regulation, in order to ease certain behavioral issues that undermine their mental health throughout their life. We argue the potential of recent widespread wearables, and more specifically smartwatches, to achieve this goal. Then, a smartwatch system that implements a wide range of self-regulation strategies and infers outburst patterns from physiological signals and movement is presented, along with an authoring tool for smartphones that is to be used by caregivers or family members to create and edit these strategies, in an adaptive way. We conducted an intensive experiment with two individuals with ASD who showed varied, representative behavioral responses to their emotional dysregulation. Both users were able to employ effective, customized emotional self-regulation strategies by means of the system, recovering from the majority of mild stress episodes and temper tantrums experienced in the nine days of experiment in their classroomThis work has been partially funded by the projects “e-Training y e-Coaching para la integración socio—laboral” (TIN2013-44586-R) and “eMadrid-CM: Investigación y Desarrollo de Tecnologías Educativas en la Comunidad de Madrid” (S2013/ICE-2715). It has been also funded by Fundación Orange during the early stages of the project “Tic-Tac-TEA: Sistema de asistencia para la autorregulación emocional en momentos de crisis para personas con TEA mediante smartwatches

Creative Haptic Interface Design for the Aging Population

Audiovisual human-computer-interfaces still make up the majority of content to the public; however, haptic interfaces offer unique advantage over the dominant information infrastructure, particularly for users with a disability or diminishing cognitive and physical skills like the elderly. The tactile sense allows users to integrate new, unobstructive channels for digital information into their sensorium, one that is less likely to be overwhelmed compared to vision and audition. Haptics research focus on the development of hardware, improving resolution, modality, and fidelity of the actuators. Despite the technological limitations, haptic interfaces are shown to reinforce physical skill acquisition, therapy, and communication. This chapter will present key characteristics intuitive tactile interfaces should capture for elderly end-users; sample projects will showcase unique applications and designs that identify the limitations of the UI

Social Touch

Interpersonal or social touch is an intuitive and powerful way to express and communicate emotions, comfort a friend, bond with teammates, comfort a child in pain, and soothe someone who is stressed. If there is one thing that the current pandemic is showing us, it is that social distancing can make some people crave physical interaction through social touch. The notion of “skin-hunger” has become tangible for many.Social touch differs at a functional and anatomical level from discriminative touch, and has clear effects at physiological, emotional, and behavioural levels. Social touch is a topic in psychology (perception, emotion, behaviour), neuroscience (neurophysiological pathways), computer science (mediated touch communication), engineering (haptic devices), robotics (social robots that can touch), humanities (science and technology studies), and sociology (the social implications of touch). Our current scientific knowledge of social touch is scattered across disciplines and not yet adequate for the purpose of meeting today's challenges of connecting human beings through the mediating channel of technology

Social touch in human–computer interaction

Touch is our primary non-verbal communication channel for conveying intimate emotions and as such essential for our physical and emotional wellbeing. In our digital age, human social interaction is often mediated. However, even though there is increasing evidence that mediated touch affords affective communication, current communication systems (such as videoconferencing) still do not support communication through the sense of touch. As a result, mediated communication does not provide the intense affective experience of co-located communication. The need for ICT mediated or generated touch as an intuitive way of social communication is even further emphasized by the growing interest in the use of touch-enabled agents and robots for healthcare, teaching, and telepresence applications. Here, we review the important role of social touch in our daily life and the available evidence that affective touch can be mediated reliably between humans and between humans and digital agents. We base our observations on evidence from psychology, computer science, sociology, and neuroscience with focus on the first two. Our review shows that mediated affective touch can modulate physiological responses, increase trust and affection, help to establish bonds between humans and avatars or robots, and initiate pro-social behavior. We argue that ICT mediated or generated social touch can (a) intensify the perceived social presence of remote communication partners and (b) enable computer systems to more effectively convey affective information. However, this research field on the crossroads of ICT and psychology is still embryonic and we identify several topics that can help to mature the field in the following areas: establishing an overarching theoretical framework, employing better research methodologies, developing basic social touch building blocks, and solving specific ICT challenges

Connecting Couples in Long-Distance Relationships : Towards Unconventional Computer-Mediated Emotional Communication Systems

The number of couples who find themselves in a long-distance relationship (LDR) is increasing for a wide range of reasons, such as overseas employment, academic pursuits, military duty, and similar circumstances. With the myriad of communication channels enabled by the low cost and ubiquity of computer-mediated communication technologies, couples in LDRs are able to stay in touch with each other around the globe. However, recent studies have revealed that the mainstream communication tools are inadequate to support the full spectrum of communication needed in intimate relationships. Emotional communication is one of the fundamental needs in close relationships, as it forms an important part of intimacy. This dissertation argues that there is a gap between what is known about LDR couples’ needs in research and what has been implemented for them in practice. The aim of this work is to bridge this gap by mediating emotional communication through unconventional user interfaces that use interaction solutions outside of the scope of their conventional use, with a particular focus on couples who sustain a committed LDR. Here, taking research through design as a core approach, a variety of qualitative methods were employed to seek answers to the research questions. This dissertation includes eight case studies, each of which is dedicated to answering its corresponding research question(s). Study I presents a systematic literature review which explored the current state of the art and identified the design opportunities. Study II introduces a series of co-design activities with five couples in LDRs to reveal the needs and challenges of users in an LDR. Studies III and IV propose two functional prototypes for unconventional communication systems to connect couples in LDRs. Study V showcases 12 design concepts of wearables created by the participants to support their own LDR. Study VI describes how four low-resolution prototypes created for mediating LDRs by the participants in the workshop would be used in real-world contexts. Studies VII and VIII each present a novel design tool to be used as a scaffold when designing communication systems for supporting LDRs: specifically, a conceptual design framework and a card-based design toolkit. This dissertation contributes new knowledge to the field of human-computer interaction through design interventions. It showcases a spectrum of practices which can be seen as a first step towards mediating emotional communication for couples in LDRs using unconventional communication systems. The findings comprise theoretical and empirical insights—derived from the eight case studies in which the author identified design opportunities and design considerations—relating to how couples in LDRs can be better supported by unconventional computer-mediated emotional communication systems

Touch Technology in Affective Human-, Robot-, and Virtual-Human Interactions:A Survey

Given the importance of affective touch in human interactions, technology designers are increasingly attempting to bring this modality to the core of interactive technology. Advances in haptics and touch-sensing technology have been critical to fostering interest in this area. In this survey, we review how affective touch is investigated to enhance and support the human experience with or through technology. We explore this question across three different research areas to highlight their epistemology, main findings, and the challenges that persist. First, we review affective touch technology through the human-computer interaction literature to understand how it has been applied to the mediation of human-human interaction and its roles in other human interactions particularly with oneself, augmented objects/media, and affect-aware devices. We further highlight the datasets and methods that have been investigated for automatic detection and interpretation of affective touch in this area. In addition, we discuss the modalities of affective touch expressions in both humans and technology in these interactions. Second, we separately review how affective touch has been explored in human-robot and real-human-virtual-human interactions where the technical challenges encountered and the types of experience aimed at are different. We conclude with a discussion of the gaps and challenges that emerge from the review to steer research in directions that are critical for advancing affective touch technology and recognition systems. In our discussion, we also raise ethical issues that should be considered for responsible innovation in this growing area.</p

Towards disappearing user interfaces for ubiquitous computing: human enhancement from sixth sense to super senses

The enhancement of human senses electronically is possible when pervasive computers interact unnoticeably with humans in Ubiquitous Computing. The design of computer user interfaces towards “disappearing” forces the interaction with humans using a content rather than a menu driven approach, thus the emerging requirement for huge number of non-technical users interfacing intuitively with billions of computers in the Internet of Things is met. Learning to use particular applications in Ubiquitous Computing is either too slow or sometimes impossible so the design of user interfaces must be naturally enough to facilitate intuitive human behaviours. Although humans from different racial, cultural and ethnic backgrounds own the same physiological sensory system, the perception to the same stimuli outside the human bodies can be different. A novel taxonomy for Disappearing User Interfaces (DUIs) to stimulate human senses and to capture human responses is proposed. Furthermore, applications of DUIs are reviewed. DUIs with sensor and data fusion to simulate the Sixth Sense is explored. Enhancement of human senses through DUIs and Context Awareness is discussed as the groundwork enabling smarter wearable devices for interfacing with human emotional memories

Digitally-mediated parent–baby touch and the formation of subjectivities

This article examines how the use of emergent smart baby monitors re-mediates parent–baby touch, notions of connection, parental sensing and the interpretation of babies’ bodies, and contributes to the formation of subjectivities. Domestic baby monitors are a mid 20th-century phenomenon which normalizes parental anxieties. While baby monitoring is not new, the ‘next generation’ of wearable bio-sensing baby monitors offers a different relationship to the body via the physiological tracking of babies, and the sending of information or alerts to parents’ via connected mobile apps. These devices have been associated with creating unnecessary parental anxiety and the digital ‘replacement’ of parental touch, although little research exists on their use in the context of parent–infant interaction or touch. The authors present a qualitative case study of one such technology, Owlet, to explore how parents experienced, understood and negotiated the discourses of parent–infant touch that circulate around and through Owlet, with particular attention to the relationship between visual and tactile resources. The study focuses on both its multimodal design and take-up by parents through analysis of interviews with the Owlet designer, Owlet as a product, focus groups with parents and families’ home experiences of Owlet. Data is analysed through a tri-part lens, which first combines multimodal social semiotic and sensory ethnographic approaches, and then the analytical concept of governmentality. The findings are discussed in relation to four analytical themes: (1) creating a desire for digitally mediated touch; (2) spatiality of digitally mediated connection; (3) formulating the ‘right kind’ of touch; and (4) reconfiguring ‘knowing touch’. The authors discuss multimodal discourses pertinent to the shaping of parent–baby touch practices including: rationality and efficiency; individualism, autonomy and freedom; and self-improvement and empowerment. They conclude that the discourses that coalesce around Owlet contribute to the reconfiguration of parent–baby touch and the formation of neoliberal subjectivities

The Making of Meaning through Dyadic Haptic Affective Touch

Despite the importance of touch in human-human relations, research in affective tactile practices is in its infancy, lacking in-depth understanding needed to inform the design of remote digital touch communication. This paper reports two qualitative studies that explore tactile affective communication in specific social contexts, and the bi-directional creation, sending and interpretation of digital touch messages using a purpose-built research tool, the Tactile Emoticon. The system comprises a pair of remotely connected mitts which enable users in different locations to communicate through tactile messages, by orchestrating duration and level of three haptic sensations: vibration, pressure and temperature. Qualitative analysis shows the nuanced ways in which 68 participants configured these elements to make meaning from touch messages they sent and received. It points to the affect and emotion of touch, its sensoriality and ambiguity, the significance of context, social norms and expectations of touch participants. Findings suggest key design considerations for digital touch communication, where the emphasis shifts from generating ‘recognizable touches’ to tools that allow people to shape their touches and establish common understanding about their meaning