What does touch tell us about emotions in touchscreen-based gameplay?

This is the post-print version of the Article. The official published version can be accessed from the link below - Copyright @ 2012 ACM. It is posted here by permission of ACM for your personal use. Not for redistribution.Nowadays, more and more people play games on touch-screen mobile phones. This phenomenon raises a very interesting question: does touch behaviour reflect the player’s emotional state? If possible, this would not only be a valuable evaluation indicator for game designers, but also for real-time personalization of the game experience. Psychology studies on acted touch behaviour show the existence of discriminative affective profiles. In this paper, finger-stroke features during gameplay on an iPod were extracted and their discriminative power analysed. Based on touch-behaviour, machine learning algorithms were used to build systems for automatically discriminating between four emotional states (Excited, Relaxed, Frustrated, Bored), two levels of arousal and two levels of valence. The results were very interesting reaching between 69% and 77% of correct discrimination between the four emotional states. Higher results (~89%) were obtained for discriminating between two levels of arousal and two levels of valence

The affective body argument in technology design

In this paper, I argue that the affective body is underused in the design of interactive technology despite what it has to offer. Whilst the literature shows it to be a powerful affective communication channel, it is often ignored in favor of the more commonly studied facial and vocal expression modalities. This is despite it being as informative and in some situations even more reliable than the other affective channels. In addition, due to the proliferation of increasingly cheaper and ubiquitous movement sensing technologies, the regulatory affective functions of the body could open new possibilities in various application areas. In this paper, after presenting a brief summary of the opportunities that the affective body offers to technology designers, I will use the case of physical rehabilitation to discuss how its use could lead to interesting new solutions and more effective therapies

Musical Expectancy in Squat Sonification For People Who Struggle With Physical Activity

Physical activity is important for a healthy lifestyle. However, it can be hard to stay engaged with exercise and this can often lead to avoidance. Sonification has been used to support physical activity through the optimisation/correction of movement. Though previous work has shown how sonification can improve movement execution and motivation, the specific mechanisms of motivation have yet to be investigated in the context of challenging exercises. We investigate the role of music expectancy as a way to leverage people’s implicit and embodied understanding of music within movement sonification to provide information on technique while also motivating continuation of movement and rewarding its completion. The paper presents two studies showing how this musically informed sonification can be used to support the squat movement. The results show how musical expectancy impacted people’s perception of their own movement, in terms of reward, motivation and movement behaviour and the way in which they moved

RealPen: Providing Realism in Handwriting Tasks on Touch Surfaces using Auditory-Tactile Feedback

We present RealPen, an augmented stylus for capacitive tablet screens that recreates the physical sensation of writing on paper with a pencil, ball-point pen or marker pen. The aim is to create a more engaging experience when writing on touch surfaces, such as screens of tablet computers. This is achieved by regenerating the friction-induced oscillation and sound of a real writing tool in contact with paper. To generate realistic tactile feedback, our algorithm analyzes the frequency spectrum of the friction oscillation generated when writing with traditional tools, extracts principal frequencies, and uses the actuator's frequency response profile for an adjustment weighting function. We enhance the realism by providing the sound feedback aligned with the writing pressure and speed. Furthermore, we investigated the effects of superposition and fluctuation of several frequencies on human tactile perception, evaluated the performance of RealPen, and characterized users' perception and preference of each feedback type

MeTA: Mediated Touch and Affect

The main aim of this first workshop on Mediated Touch and Affect (MeTA) is to bring together researchers from diverse communities, such as affective computing, hap tics, augmented reality, communication, design, psychology, human-robot interaction, and telepresence. The goal is to discuss the current state of research on mediated touch and affect and to formulate a research agenda for future directions in research on aspects of the touch-technology-affect triangle

Pain level recognition using kinematics and muscle activity for physical rehabilitation in chronic pain

People with chronic musculoskeletal pain would benefit from technology that provides run-time personalized feedback and help adjust their physical exercise plan. However, increased pain during physical exercise, or anxiety about anticipated pain increase, may lead to setback and intensified sensitivity to pain. Our study investigates the possibility of detecting pain levels from the quality of body movement during two functional physical exercises. By analyzing recordings of kinematics and muscle activity, our feature optimization algorithms and machine learning techniques can automatically discriminate between people with low level pain and high level pain and control participants while exercising. Best results were obtained from feature set optimization algorithms: 94% and 80% for the full trunk flexion and sit-to-stand movements respectively using Support Vector Machines. As depression can affect pain experience, we included participants' depression scores on a standard questionnaire and this improved discrimination between the control participants and the people with pain when Random Forests were used. / Note: As originally published there is an error in the document. The following information was omitted by the authors: "The project was funded by the EPSRC grant Emotion & Pain Project EP/H017178/1 and Olugbade was supported by the 2012 Nigerian PRESSID PhD funding." The article PDF remains unchanged

Investigating nuanced sensory experiences in textiles selection

In this study, research tools are used to investigate designers' textile selection activities and uncover the sensorial experiences that underpin those activities. Such tools were purposely introduced to disrupt the way designers generally interact with textiles and generate conversations around it. The study was conducted in a textile fair during two consecutive years with an expert audience who were in the mind-set of sourcing. This study resulted in four main themes that reveal the importance of the multisensory experience to textile selection, and the complexity of remembering and communicating such experiences in the design process, given how tacit such experiences are

Action Sounds Modulate Arm Reaching Movements

Our mental representations of our body are continuously updated through multisensory bodily feedback as we move and interact with our environment. Although it is often assumed that these internal models of body-representation are used to successfully act upon the environment, only a few studies have actually looked at how body-representation changes influence goal-directed actions, and none have looked at this in relation to body-representation changes induced by sound. The present work examines this question for the first time. Participants reached for a target object before and after adaptation periods during which the sounds produced by their hand tapping a surface were spatially manipulated to induce a representation of an elongated arm. After adaptation, participants' reaching movements were performed in a way consistent with having a longer arm, in that their reaching velocities were reduced. These kinematic changes suggest auditory-driven recalibration of the somatosensory representation of the arm morphology. These results provide support to the hypothesis that one's represented body size is used as a perceptual ruler to measure objects' distances and to accordingly guide bodily actions

Roles for Personal Informatics in Chronic Pain

Self-management of chronic pain is a complex and demanding activity. Multidisciplinary pain management programs are designed to provide patients with the skills to improve, maintain functioning and self-manage their pain but gains diminish in the long-term due to lack of support from clinicians. Sensing technology can be a cost-effective way to extend support for self-management outside clinical settings but they are currently under-explored. In this paper, we report studies carried out to investigate how Personal Informatics Systems (PIS) based on wearable body sensing technology could facilitate pain self-management and functioning. Five roles for PIS emerged from a qualitative study with people with chronic pain and physiotherapists: (i) assessment, planning and prevention (ii) a direct supervisory and co-management role, (iii) facilitating deeper understanding, (iv) managing emotional states, and (v) sharing for social acceptability. A web-based survey was conducted to understand the parameters that should be tracked to support self-management and what tracked information should be shared with others. Finally, we suggest an extension to previous PIS models and propose design implications to address immediate, short-term and long-term information needs for personal use of people with chronic pain and for sharing with others. / Note: As originally published there is an error in the document. The following information was omitted by the authors: "The project was funded by the EPSRC grant Emotion & Pain Project EP/H017178/1 rather than the EPSRC grant EP/G043507/1: Pain rehabilitation: E/Motion-based automated coaching.." The article PDF remains unchanged