Tunteiden Havaitseminen Arkielämässä Koneoppimisen ja Puettavien Laitteiden Avulla

Tavoitteet. Tämän tutkimuksen tavoitteena on arvioida tunteiden havaitsemisen mahdollisuutta arkielämässä puettavien laitteiden ja koneoppimismallien avulla. Tunnetiloilla on tärkeä rooli päätöksenteossa, havaitsemisessa ja käyttäytymisessä, mikä tekee objektiivisesta tunnetilojen havaitsemisesta arvokkaan tavoitteen, sekä mahdollisten sovellusten että tunnetiloja koskevan ymmärryksen syventämisen kannalta. Tunnetiloihin usein liittyy mitattavissa olevia fysiologisia ja käyttäymisen muutoksia, mikä mahdollistaa koneoppimismallien kouluttamisen muutoksia aiheuttaneen tunnetilan havaitsemiseksi. Suurin osa tunteiden havaitsemiseen liittyvästä tutkimuksesta on toteutettu laboratorio-olosuhteissa käyttämällä tunteita herättäviä ärsykkeitä tai tehtäviä, mikä herättää kysymyksen siitä että yleistyvätkö näissä olosuhteissa saadut tulokset arkielämään. Vaikka puettavien laitteiden ja kännykkäkyselyiden kehittyminen on helpottanut aiheen tutkimista arkielämässä, tutkimusta tässä ympäristössä on vielä niukasti. Tässä tutkimuksessa itseraportoituja tunnetiloja ennustetaan koneoppimismallien avulla arkielämässä havaittavissa olevien tunnetilojen selvittämiseksi. Lisäksi tutkimuksessa käytetään mallintulkintamenetelmiä mallien hyödyntämien yhteyksien tunnistamiseksi. Metodit. Aineisto tätä tutkielmaa varten on peräisin tutkimuksesta joka suoritettiin osana Helsingin Yliopiston ja VTT:n Sisu at Work projektia, missä 82:ta tietotyöläistä neljästä suomalaisesta organisaatiosta tutkittiin kolmen viikon ajan. Osallistujilla oli jakson aikana käytettävissään mittalaitteet jotka mittasivat fotoplethysmografiaa (PPG), ihon sähkönjohtavuutta (EDA) ja kiihtyvyysanturi (ACC) signaaleita, lisäksi heille esitettiin kysymyksiä koetuista tunnetiloista kolmesti päivässä puhelinsovelluksen avulla. Signaalinkäsittelymenetelmiä sovellettiin signaaleissa esiintyvien liikeartefaktien ja muiden ongelmien korjaamiseksi. Sykettä (HR) ja sykevälinvaihtelua (HRV) kuvaavia piirteitä irroitettiin PPG signaalista, fysiologista aktivaatiota kuvaavia piirteitä EDA signaalista, sekä liikettä kuvaavia piirteitä ACC signaalista. Seuraavaksi koneoppimismalleja koulutettiin ennustamaan raportoituja tunnetiloja irroitetujen piirteiden avulla. Mallien suoriutumista vertailtiin suhteessa odotusarvoihin havaittavissa olevien tunnetilojen määrittämiseksi. Lisäksi permutaatiotärkeyttä sekä Shapley additive explanations (SHAP) arvoja hyödynnettiin malleille tärkeiden yhteyksien selvittämiseksi. Tulokset ja johtopäätökset. Mallit tunnetiloille virkeä, keskittynyt ja innostunut paransivat suoriutumistaan yli odotusarvon, joista mallit tunnetilalle virkeä paransivat suoriutumista tilastollisesti merkitsevästi. Permutaatiotärkeys korosti liike- ja HRV-piirteiden merkitystä, kun SHAP arvojen tarkastelu nosti esiin matalan liikkeen, matalan EDA:n, sekä korkean HRV:n merkityksen mallien ennusteille. Nämä tulokset ovat lupaavia korkean aktivaation positiivisten tunnetilojen havaitsemiselle arkielämässä, sekä nostavat esiin mahdollisia yhteyksiä jatkotutkimusta varten.Objectives. This study aims to evaluate feasibility of affect detection in daily life using wearable devices and machine learning models. Affective states play an important role in decision making, perception and behaviour, making objective detection of affective states a desirable goal both for potential applications and as a way to gain insight into affective phenomena. Affective states have been found to have measurable physiological and behavioral changes, which allows training of machine learning models for detecting the underlying affects. Majority of affect detection studies have been conducted in laboratory conditions using affect elicitation stimuli or tasks, raising the question whether results from these studies will generalize to daily life. Although development of wearable devices and mobile surveys have facilitated evaluation in the context of daily life, research here remains sparse. In this study, self-reported affective states are predicted using machine learning models to identify which affective states can be detected in daily life. Additionally, model interpretation methods will be used to identify which relationships the models found important for their predictions. Methods. Data for this thesis came from a study conducted as a part of Sisu at Work project between University of Helsinki and VTT, where 82 knowledge workers from four Finnish organizations were studied for a period of three weeks. During this period, the participants were queried by mobile surveys about their affective states thrice a day, while they also used wearable devices to record photoplethysmography (PPG), electrodermal activity (EDA) and accelerometry (ACC) signals. A signal processing pipeline was implemented to deal with movement artefacts and other issues with the data. Features describing heart rate (HR) and heart rate variation (HRV) were extraced from PPG, physiological activation from EDA and movement from ACC signals. Models were then fitted to predict the reported affective states using the extracted features. Model performance was compared against a baseline to identify which affects could be reliably detected, while permutation importance and Shapley additive explanations (SHAP) values were used to identify important relationships established by the models. Results and conclusions. Models for affective state vigor showed improvements over baseline with statistical significance, while improvements were also noted for affects focused and enthusiastic. Permutation importance highlighted the significance of movement and HRV features, while examination of SHAP values indicated that low movement, low EDA and high HRV impacted model predictions the most. These results indicate potential for detecting high activation affective states in daily life and propose potential relationships for future research

EDAMUX : A method for measuring User Experience

Background: User experience (UX) is seen as an important quality of a successful product and software companies are becoming increasingly interested in the field of UX. As UX has the goal to improve the experience of users, there is a need for better methods in measuring the actual experience. One aspect of UX is to understand the emotional aspect of experience. Psychophysiology studies the relations between emotions and physiology and electrodermal activity (EDA) has been found to be a physiological measurement of emotional arousal. Aims: The aim of this thesis is researching the utility of measuring EDA to identify moments of emotional arousal during human-computer interaction. By studying peaks in EDA during software interaction we expect to find issues in the software that work as triggers or stimuli for the peaks. Method: We used the design science methodology to develop EDAMUX. EDAMUX is a method to unobtrusively observe users, while gathering significant interaction moments through self reporting and EDA. A qualitative single-case study was conducted to evaluate the utility of EDAMUX. Results: We found that we can discover causes of bad user experience with EDAMUX. Moments of emotional arousal, derived from EDA, was found in conjunction with performance issues, usability issues and bugs. Emotional arousal was also observed during software interaction where the user was blaming themself. Conclusions: EDAMUX shows potential in discovering issues in software that are difficult to find with methods that rely on subjective self-reporting. Having the potential to objectively study emotional reactions is seen as valuable in complementing existing methods of measuring user experience

A health-oriented emotion-centred origami-based PSS concept. A product-service system concept aimed to help users manage and reduce their stress more tangibly to improve their health and well-being.

Emotions have a fundamental role in the experience, perception, cognition, and development of people (Barrett et al., 2016; Plutchik, 2001). Negative emotions such as stress, if not managed appropriately, may be a risk factor in developing diseases such as dementia, cardiovascular problems, and depression, amongst others (Dum et al., 2016; Sandi et al., 2001). This interdisciplinary research presents OrigamEase, a health-oriented emotion-centred origami-based PSS concept to help adults manage their stress more tangibly, in response to its main research question: How can engineering design research contribute to improving people’s emotional health and wellbeing? OrigamEase was designed throughout this research, and its design and testing served to develop a health & well-being oriented, emotion-centred engineering design methodology (HWOEED). Therefore, the design of this methodology is the result of the structuring and ordering of the developmental process of exploring, conducting and streamlining the research, design and testing of OrigamEase. The design of OrigamEase (both the product and the service part) is based on the cognitive research stream of emotions and stress. Therefore, this research also aims to broaden the knowledge about the implications and management of the emotional experience of stress from an engineering and design point of view, complementing the available solutions for stress management through a structured, measurable, and tangible tool. The HWOEED methodology is influenced by Kansei Engineering and Design Thinking methodologies and integrates various engineering design research methods. However, this methodology proposes a different application of the integration of emotional considerations into engineering design processes., it seeks to improve the emotional experience of users to preserve and promote their health and well-being through specifically designed products, services or PSS. Therefore, these designs become the means and not the end of the engineering design efforts. Also, this methodology can be transferrable to other engineering design solutions. OrigamEase was tested with 114 adults between 18 and 70 years old through three pilot tests (n=43) and six trial tests (n=71) using a concurrent triangulation mixed methods design. Then the results from these tests were contrasted with two control tests (n=22). The results show that using OrigamEase reduced the measured stress levels of participants (self-reported, heart rate and electrodermal activity) significantly, supporting the experiment hypothesis. Stress levels were recorded before and after using OrigamEase; then, a repeated-measures t-test was applied to find if these differences were significant or not. After using OrigamEase, 73.2% of participants reported feeling less stressed (mean reduction=13.94%), 85.5% experienced a reduction in their heart rate (mean reduction=9.8 bpm), and 78.9% had a lower electrodermal activity (mean reduction= 10.8 points). The testing of OrigamEase served as an initial application validation of the HWOEED methodology. This research demonstrates that engineering and design fields not only can but need to contribute to research on emotions through interdisciplinary research. Emotions are a fundamental part of all human experiences, impacting a person’s health and well-being profoundly

The design & fabrication of an epileptic seizure detection watch

Capstone submitted to the Department of Engineering, Ashesi University in partial fulfilment of the requirements for the award of Bachelor of Science degree in Electrical & Electronic Engineering, April 2019.This paper describes the design and fabrication of an epileptic seizure detection watch for the timely detection of Generalized Tonic-Clonic (GTC) seizures; using skin conductance (SC) signals. The watch’s circuit was designed in EasyEDA and implemented on a Breadboard to showcase the dispatch of a seizure event alert to a phone via a Bluetooth module; in the event of an ongoing seizure and vice versa. Due to the unavailability of SC signal databases, Electroencephalography (EEG) signals, acquired from a physiological database known as PhysioNet were used in showcasing the signal processing of incoming SC signals, temporal and spectral feature extraction of these signals, and the classification of these signals using a trained machine learning algorithm. Twenty-five machine learning algorithms provided by the MATLAB Classification Learner App were trained using 80 EEG signals (both seizure and non-seizure) and only two algorithms, namely the Medium Tree and Linear Support Vector Machine (SVM) had the highest training prediction accuracy. However, in determining their prediction accuracy with two different data sets, the Medium Tree model had the highest cumulative prediction accuracy of 76.7%; as compared to the Linear SVM model which had a cumulative prediction accuracy of 73.3%. Based on these results, the Medium Tree model was recommended as a good seizure detection algorithm to prevent fatal and non-fatal injuries; and even Sudden Unexpected Death in Epilepsy (SUDEP).Ashesi Universit

Physiopad: development of a non-invasive game controller toolkit to study physiological responses for Game User Research

Os jogos afectivos usam as respostas fisiológicas do jogador para criar um ambiente adequado ao estado emocional do utilizador. A investigação destes jogos tem sido explorada nos últimos anos. Estas experiências, contudo, ainda requerem sistemas complexos e difíceis de utilizar. Nesta dissertação, é proposta a construção de um dispositivo capaz de ler dados fisiológicos de forma não invasiva e que seja de fácil utilização. Este aparelho faz a leitura do ritmo cardíaco e dos níveis de excitação do jogador, além disso foi criado um software para interligar com o dispositivo. Utilizando um comando da PlayStation 3 e um BITalino, o dispositivo é capaz de fazer a aquisição do sinal PPG e sinal EDA durante o jogo. O software analisa os sinais do comando, calcula o ritmo cardíaco e mede os níveis de excitação em tempo real. Foi realizada uma experiência utilizando biofeedback positivo e negativo, com o objectivo de testar a integração entre o software e o hardware. Não será no imediato que os dispositivos deste género sejam disponibilizados comercialmente. Os resultados são, no entanto, promissores. O cálculo do ritmo cardíaco em tempo real tem apenas uma diferença de 5 batimentos por minuto em relação ao ritmo cardíaco real do jogador. Apesar de os testes com o EDA serem inconclusivos, pode-se verificar que foi possível construir um sistema para ler os dados fisiológicos sendo mais económico do que os seus pares, sem comprometer a fiabilidade dos dados.Affective games are a genre of games that use the physiological responses from the player to adapt the gameplay to a more enjoyable emotional state and experience. Physiological responses and affective games have been studied vastly over the years. However, the setups used in these interventions are very intrusive and are complex to set up. In this project, it is purposed to build a non-invasive and easy-to-set-up toolkit that records physiological data. This toolkit records the player's heart rate and arousal levels and was decomposed into software and hardware. Using a PS3 game controller replica and a BITalino, a physiological game controller which can record heart rate and arousal during gameplay was built. The software interfaces with the gamepad, processes the physiological signals and sends this information to the game. An experiment with a positive biofeedback condition and negative biofeedback condition was conducted. This experiment showed that even though more work must be done until these type of devices could be commercially available, the results are promising. This toolkit’s heart rate values, when compared with other more traditional acquisition devices, were very similar, being on average only 5 BMP lower than the actual heart rate, proving that is possible to build more affordable non-invasive physiological hardware without compromising the signal's accuracy

Wearable Computing for Health and Fitness: Exploring the Relationship between Data and Human Behaviour

Health and fitness wearable technology has recently advanced, making it easier for an individual to monitor their behaviours. Previously self generated data interacts with the user to motivate positive behaviour change, but issues arise when relating this to long term mention of wearable devices. Previous studies within this area are discussed. We also consider a new approach where data is used to support instead of motivate, through monitoring and logging to encourage reflection. Based on issues highlighted, we then make recommendations on the direction in which future work could be most beneficial

Aerospace Medicine and Biology: A continuing supplement 180, May 1978

This special bibliography lists 201 reports, articles, and other documents introduced into the NASA scientific and technical information system in April 1978

Utilising Emotion Monitoring for Developing Music Interventions for People with Dementia:A State-of-the-Art Review

The demand for smart solutions to support people with dementia (PwD) is increasing. These solutions are expected to assist PwD with their emotional, physical, and social well-being. At the moment, state-of-the-art works allow for the monitoring of physical well-being; however, not much attention is delineated for monitoring the emotional and social well-being of PwD. Research on emotion monitoring can be combined with research on the effects of music on PwD given its promising effects. More specifically, knowledge of the emotional state allows for music intervention to alleviate negative emotions by eliciting positive emotions in PwD. In this direction, the paper conducts a state-of-the-art review on two aspects: (i) the effect of music on PwD and (ii) both wearable and non-wearable sensing systems for emotional state monitoring. After outlining the application of musical interventions for PwD, including emotion monitoring sensors and algorithms, multiple challenges are identified. The main findings include a need for rigorous research approaches for the development of adaptable solutions that can tackle dynamic changes caused by the diminishing cognitive abilities of PwD with a focus on privacy and adoption aspects. By addressing these requirements, advancements can be made in harnessing music and emotion monitoring for PwD, thereby facilitating the creation of more resilient and scalable solutions to aid caregivers and PwD