The affective body argument in technology design

In this paper, I argue that the affective body is underused in the design of interactive technology despite what it has to offer. Whilst the literature shows it to be a powerful affective communication channel, it is often ignored in favor of the more commonly studied facial and vocal expression modalities. This is despite it being as informative and in some situations even more reliable than the other affective channels. In addition, due to the proliferation of increasingly cheaper and ubiquitous movement sensing technologies, the regulatory affective functions of the body could open new possibilities in various application areas. In this paper, after presenting a brief summary of the opportunities that the affective body offers to technology designers, I will use the case of physical rehabilitation to discuss how its use could lead to interesting new solutions and more effective therapies

Human Observer and Automatic Assessment of Movement Related Self-Efficacy in Chronic Pain: from Exercise to Functional Activity

Clinicians tailor intervention in chronic pain rehabilitation to movement related self-efficacy (MRSE). This motivates us to investigate automatic MRSE estimation in this context towards the development of technology that is able to provide appropriate support in the absence of a clinician. We first explored clinical observer estimation, which showed that body movement behaviours, rather than facial expressions or engagement behaviours, were more pertinent to MRSE estimation during physical activity instances. Based on our findings, we built a system that estimates MRSE from bodily expressions and bodily muscle activity captured using wearable sensors. Our results (F1 scores of 0.95 and 0.78 in two physical exercise types) provide evidence of the feasibility of automatic MRSE estimation to support chronic pain physical rehabilitation. We further explored automatic estimation of MRSE with a reduced set of low-cost sensors to investigate the possibility of embedding such capabilities in ubiquitous wearable devices to support functional activity. Our evaluation for both exercise and functional activity resulted in F1 score of 0.79. This result suggests the possibility of (and calls for more studies on) MRSE estimation during everyday functioning in ubiquitous settings. We provide a discussion of the implication of our findings for relevant areas

Head movement differs for positive and negative emotions in video recordings of sitting individuals

Individuals tend to approach positive stimuli and avoid negative stimuli. Furthermore, emotions influence whether individuals freeze or move more. These two kinds of motivated behavior refer to the approach/avoidance behavior and behavioral freezing/activation. Previous studies examined (e.g., using forced platforms) whether individuals' behavior depends on stimulus' valence; however, the results were mixed. Thus, we aimed to test whether emotions' effects on spontaneous whole-body behavior of standing individuals also occur in the seated position. We used a computer vision method to measure the head sway in video recordings that offers ease of use, replicability, and unobtrusiveness for the seated research participant. We analyzed behavior recorded in the laboratory during emotion manipulations across five studies totaling 932 participants. We observed that individuals leaned more forward and moved more when watching positive stimuli than when watching negative stimuli. However, individuals did not behave differently when watching positive or negative stimuli than in the neutral condition. Our results indicate that head movements extracted from seated individuals' video recordings can be useful in detecting robust differences in emotional behavior (positive vs. negative emotions)

Unsupervised domain adaptation under label space mismatch for speech classification

Unsupervised domain adaptation using adversarial learning has shown promise in adapting speech models from a labeled source domain to an unlabeled target domain. However, prior works make a strong assumption that the label spaces of source and target domains are identical, which can be easily violated in real-world conditions. We present AMLS, an end-to-end architecture that performs Adaptation under Mismatched Label Spaces using two weighting schemes to separate shared and private classes in each domain. An evaluation on three speech adaptation tasks, namely gender, microphone, and emotion adaptation, shows that AMLS provides significant accuracy gains over baselines used in speech and vision adaptation tasks. Our contribution paves the way for applying UDA to speech models in unconstrained settings with no assumptions on the source and target label spaces

Automated Inference of Cognitive Stress in-the-Wild

We aim to build technology that combines mobile sensing systems to automatically infer a person’s cognitive stress to provide better and continuous stress management support. Our main innovation is the use of low-cost mobile thermal camera integrated in smartphone or other devices to produce new stress measures. We have developed a robust mobile based tracking system that tracks a person’s breathing pattern by measuring temperature changes around a person’s nostrils region while the person is facing the smartphone. Stress levels are automatically assessed by capturing breathing pattern dynamics through a novel signature based on time and frequency values and using convolutional neural networks to reduce the need to hand craft higher level features. We are now extending the system to integrate multiple sensors (e.g., PPG and GSR) and behavioural information (context). The system is being also adapted to be applied in the context of industry workfloor within the EU H2020 HUMAN research project to support workers during stress inducing tasks. Evaluations are being conducted both in the laboratory and in-the-wild (e.g., industry workfloor)

MeTA: Mediated Touch and Affect

The main aim of this first workshop on Mediated Touch and Affect (MeTA) is to bring together researchers from diverse communities, such as affective computing, hap tics, augmented reality, communication, design, psychology, human-robot interaction, and telepresence. The goal is to discuss the current state of research on mediated touch and affect and to formulate a research agenda for future directions in research on aspects of the touch-technology-affect triangle

Pain level recognition using kinematics and muscle activity for physical rehabilitation in chronic pain

People with chronic musculoskeletal pain would benefit from technology that provides run-time personalized feedback and help adjust their physical exercise plan. However, increased pain during physical exercise, or anxiety about anticipated pain increase, may lead to setback and intensified sensitivity to pain. Our study investigates the possibility of detecting pain levels from the quality of body movement during two functional physical exercises. By analyzing recordings of kinematics and muscle activity, our feature optimization algorithms and machine learning techniques can automatically discriminate between people with low level pain and high level pain and control participants while exercising. Best results were obtained from feature set optimization algorithms: 94% and 80% for the full trunk flexion and sit-to-stand movements respectively using Support Vector Machines. As depression can affect pain experience, we included participants' depression scores on a standard questionnaire and this improved discrimination between the control participants and the people with pain when Random Forests were used. / Note: As originally published there is an error in the document. The following information was omitted by the authors: "The project was funded by the EPSRC grant Emotion & Pain Project EP/H017178/1 and Olugbade was supported by the 2012 Nigerian PRESSID PhD funding." The article PDF remains unchanged

RealPen: Providing Realism in Handwriting Tasks on Touch Surfaces using Auditory-Tactile Feedback

We present RealPen, an augmented stylus for capacitive tablet screens that recreates the physical sensation of writing on paper with a pencil, ball-point pen or marker pen. The aim is to create a more engaging experience when writing on touch surfaces, such as screens of tablet computers. This is achieved by regenerating the friction-induced oscillation and sound of a real writing tool in contact with paper. To generate realistic tactile feedback, our algorithm analyzes the frequency spectrum of the friction oscillation generated when writing with traditional tools, extracts principal frequencies, and uses the actuator's frequency response profile for an adjustment weighting function. We enhance the realism by providing the sound feedback aligned with the writing pressure and speed. Furthermore, we investigated the effects of superposition and fluctuation of several frequencies on human tactile perception, evaluated the performance of RealPen, and characterized users' perception and preference of each feedback type