Tactile modulation of emotional speech samples

Copyright © 2012 Katri Salminen et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly citedTraditionally only speech communicates emotions via mobile phone. However, in daily communication the sense of touch mediates emotional information during conversation. The present aim was to study if tactile stimulation affects emotional ratings of speech when measured with scales of pleasantness, arousal, approachability, and dominance. In the Experiment 1 participants rated speech-only and speech-tactile stimuli. The tactile signal mimicked the amplitude changes of the speech. In the Experiment 2 the aim was to study whether the way the tactile signal was produced affected the ratings. The tactile signal either mimicked the amplitude changes of the speech sample in question, or the amplitude changes of another speech sample. Also, concurrent static vibration was included. The results showed that the speech-tactile stimuli were rated as more arousing and dominant than the speech-only stimuli. The speech-only stimuli were rated as more approachable than the speech-tactile stimuli, but only in the Experiment 1. Variations in tactile stimulation also affected the ratings. When the tactile stimulation was static vibration the speech-tactile stimuli were rated as more arousing than when the concurrent tactile stimulation was mimicking speech samples. The results suggest that tactile stimulation offers new ways of modulating and enriching the interpretation of speech.Peer reviewe

Predicting Daytime Sleepiness from Electrocardiography Based Respiratory Rate Using Deep Learning

Daytime sleepiness impairs the activities of daily living, especially in chronic disease patients. Typically, daytime sleepiness is measured with subjective patient reported outcomes (PROs), which could be prone to recall bias. Objective measures of daytime sleepiness, which are sensitive to change, would benefit disease state assessment and novel therapies that impact the quality of life. The presented study aimed to predict daytime sleepiness from two hours of continuously measured respiratory rate using a 1-dimensional convolutional neural network. A wearable biosensor was used to continuously measure electrocardiography (ECG) based respiratory rate, while the participants (N=82) were asked to fill in Karolinska Sleepiness Scale three times a day. Considering the need for a sleepiness measure for chronic diseases, neurodegenerative disease (NDD, N=14) patients, immune-mediated inflammatory disease (IMID, N=42) patients, as well as healthy participants (N=26) were included in the study. The diseaseagnostic model achieved an accuracy of 63% between nonsleepy and sleepy states. The result demonstrates the potential of using respiratory rate with deep learning for an objective measure of daytime sleepiness.acceptedVersionPeer reviewe

Fatigue-Related Changes of Daily Function: Most Promising Measures for the Digital Age

Background: Fatigue is a prominent symptom in many diseases and is strongly associated with impaired daily function. The measurement of daily function is currently almost always done with questionnaires, which are subjective and imprecise. With the recent advances of digital wearable technologies, novel approaches to evaluate daily function quantitatively and objectively in real-life conditions are increasingly possible. This also creates new possibilities to measure fatigue-related changes of daily function using such technologies. Summary: This review examines which digitally assessable parameters in immune-mediated inflammatory and neurodegenerative diseases may have the greatest potential to reflect fatigue-related changes of daily function. Key Messages: Results of a standardized analysis of the literature reporting about perception-, capacity-, and performance-evaluating assessment tools indicate that changes of the following parameters: physical activity, independence of daily living, social participation, working life, mental status, cognitive and aerobic capacity, and supervised and unsupervised mobility performance have the highest potential to reflect fatigue-related changes of daily function. These parameters thus hold the greatest potential for quantitatively measuring fatigue in representative diseases in real-life conditions, e.g., with digital wearable technologies. Furthermore, to the best of our knowledge, this is a new approach to analysing evidence for the design of performance-based digital assessment protocols in human research, which may stimulate further systematic research in this area

Assessing fatigue and sleep in chronic diseases using physiological signals from wearables : A pilot study

Problems with fatigue and sleep are highly prevalent in patients with chronic diseases and often rated among the most disabling symptoms, impairing their activities of daily living and the health-related quality of life (HRQoL). Currently, they are evaluated primarily via Patient Reported Outcomes (PROs), which can suffer from recall biases and have limited sensitivity to temporal variations. Objective measurements from wearable sensors allow to reliably quantify disease state, changes in the HRQoL, and evaluate therapeutic outcomes. This work investigates the feasibility of capturing continuous physiological signals from an electrocardiography-based wearable device for remote monitoring of fatigue and sleep and quantifies the relationship of objective digital measures to self-reported fatigue and sleep disturbances. 136 individuals were followed for a total of 1,297 recording days in a longitudinal multi-site study conducted in free-living settings and registered with the German Clinical Trial Registry (DRKS00021693). Participants comprised healthy individuals (N = 39) and patients with neurodegenerative disorders (NDD, N = 31) and immune mediated inflammatory diseases (IMID, N = 66). Objective physiological measures correlated with fatigue and sleep PROs, while demonstrating reasonable signal quality. Furthermore, analysis of heart rate recovery estimated during activities of daily living showed significant differences between healthy and patient groups. This work underscores the promise and sensitivity of novel digital measures from multimodal sensor time-series to differentiate chronic patients from healthy individuals and monitor their HRQoL. The presented work provides clinicians with realistic insights of continuous at home patient monitoring and its practical value in quantitative assessment of fatigue and sleep, an area of unmet need.publishedVersionPeer reviewe

Dynaaminen tuntopalaute ihmisen ja koneen välisessä vuorovaikutuksessa

Vibrotactile stimulation is an efficient way to provide feedback when a user is interacting with a mobile device. Because the device is held in hand, the user's input, such as touch and gesturing, can be easily measured with force- and motion sensors. In addition, the feedback can be provided into the same device with a vibrotactile actuator. This thesis introduces a set of methods to provide the vibrotactile feedback dynamically i.e. proportionally to the applied input in real-time during the interaction. The effects of dynamic tactile feedback in gesture and touch interaction are investigated in five studies reported in the thesis. The first two studies introduce the method for providing the dynamic feedback based on wavetable synthesis and assess the perceptual dimensions of the feedback when coupled with gesture movements. The studies indicated that the method is a powerful tool for creating an illusion of real textures with a simple vibrotactile actuator and motion sensor. Modifications in texture ridge length and spatial density lead into large variation in subjective evaluations of texture roughness, bumpiness, stickiness and pleasantness. In study III, mobile device was used to search for targets in the environment. Tactile feedback turned out to be an efficient way to indicate target directions. The finding hints that vibration could be efficiently used as a private and transparent signal to indicate directions and target locations in augmented reality interaction. In study IV, dynamic tactile feedback was provided when playing a gesture controlled virtual musical instrument. The feedback reflecting the distance to a virtual control surface assisted the temporal accuracy of the playing. In study V, vibrotactile feedback was provided proportionally to the applied force on a rigid surface. In a force repetition task, all the dynamic feedback models assisted the accuracy of the force control. On contrast, in a force hold task, the feedback did not assist the performance. Dynamic tactile feedback appeared to be a powerful tool to create illusions of real textures and it improved the motor accuracy in repetitive gesture- and force based interaction. The findings make a contribution to research focusing on touch and gesture based user interfaces as well as psychophysics research dealing with augmented feedback. In order to apply the findings in future user interfaces of mobile devices, more studies are needed. In further work, system latency should be minimized and tactile feedback should be design in parallel with visual and auditory feedback.Ihmisen tuntoaistia voidaan hyödyntää tehokkasti mobiililaitteiden käyttöliittymissä käyttämällä laitteen värinäaktuaattoria palautteen antamiseen. Tämän lisäksi käyttäjän syötteitä, kuten eleitä ja kosketusta, voidaan helposti mitata laitteeseen asennetuilla liike- ja voima-antureilla. Tässä työssä esitellään erilaisia tapoja tuottaa dynaamista värinäpalautetta joka syntetisoidaan reaaliajassa käyttäjän antaman syötteen perusteella. Näiden palautetapojen hyödyllisyyttä ele- ja kosketusvuorovaikutuksessa tutkittiin viidessä eri tutkimuksessa. Kahdessa ensimmäisessä tutkimuksessa tarkasteltiin aaltomuotosynteesin avulla tuotettua palautetta, joka on suhteutettu käyttäjän käden liikenopeuteen. Tutkimukset osoittivat, että tällä tavalla voidaan tehokkaasti synnyttää illuusio virtuaalisesta tekstuurista yksinkertaista värinäaktuaattoria ja liikeanturia hyväksikäyttäen. Tekstuurin tiheyttä ja kuvioinnin leveyttä muuttamalla voidaan saada aikaan erilaisia vaikutelmia tekstuurin karkeudesta, epätasaisuudesta, tahmeudesta ja miellyttävyydestä. Kolmannessa tutkimuksessa mobiililaitetta käytettiin virtuaalisten kohteiden etsintään ympäristöstä. Värinäpalaute todettiin tehokkaaksi tavaksi osoittaa kohteen suunta. Tässä vahvistetun todellisuuden käyttöskenaariossa tuntopalaute toimii siis oivallisesti henkilökohtaisena signaalina kohteiden sijainnin paljastamiseen, eikä se sido käyttäjän näkö- ja kuuloaistia käyttötilanteessa. Neljännessä tutkimuksessa dynaamista palautetta annettiin eleohjattua virtuaalirumpua soitettaessa. Värinäpalaute, jonka amplitudi voimistui virtuaalista rummunkalvoa lähestyttäessä paransi merkittävästi suoritusta rytmistä tarkkuutta vaativassa tehtävässä. Viidennessä tutkimuksessa dynaaminen palaute suhteutettiin voimaan, jolla sormi painoi laitteen pintaa. Voimatasojen toistotehtävässä kaikki dynaamiset palautetavat auttoivat käyttäjiä voiman kontrolloinnissa. Toisaalta, dynaaminen palaute ei auttanut voimatason ylläpitotehtävässä. Dynaaminen palaute osoittautui tehokkaaksi työkaluksi illuusioiden muodostamiseen oikeista tekstuureista ja se paransi motorista tarkkuutta toistoa vaativissa ele- ja voimainteraktiossa. Löydökset ovat merkittäviä sekä ele- ja kosketuskäyttöliittymien kehittämisen että vahvistettuun palautteeseen keskittyvän psykofysiologisen tutkimuksen kannalta. Jotta tuloksia voitaisiin hyödyntää tulevaisuuden käyttöliittymissä, tarvitaan lisätutkimuksia. Niissä tulisi minimoida systeemin viiveet ja tuntopalaute tulisi suunnitella yhdessä visuaalisen ja auditorisen palautteen kanssa

Squeeze vs. tilt : A comparative study using continuous tactile feedback

This paper presents an investigation into the performance of squeezing as a manipulative interaction technique in comparison to tilting with an aim to answer two questions: is squeezing an effective input technique for mobile devices and can tactile feedback improve performance? The experiment results show that both input methods are viable but squeezing is significantly faster and more sustainable than tilting (with and without tactile feedback)

Tactile Modulation of Emotional Speech Samples

Traditionally only speech communicates emotions via mobile phone. However, in daily communication the sense of touch mediates emotional information during conversation. The present aim was to study if tactile stimulation affects emotional ratings of speech when measured with scales of pleasantness, arousal, approachability, and dominance. In the Experiment 1 participants rated speech-only and speech-tactile stimuli. The tactile signal mimicked the amplitude changes of the speech. In the Experiment 2 the aim was to study whether the way the tactile signal was produced affected the ratings. The tactile signal either mimicked the amplitude changes of the speech sample in question, or the amplitude changes of another speech sample. Also, concurrent static vibration was included. The results showed that the speech-tactile stimuli were rated as more arousing and dominant than the speech-only stimuli. The speech-only stimuli were rated as more approachable than the speech-tactile stimuli, but only in the Experiment 1. Variations in tactile stimulation also affected the ratings. When the tactile stimulation was static vibration the speech-tactile stimuli were rated as more arousing than when the concurrent tactile stimulation was mimicking speech samples. The results suggest that tactile stimulation offers new ways of modulating and enriching the interpretation of speech