Mobile devices for the remote acquisition of physiological and behavioral biomarkers in psychiatric clinical research

Psychiatric disorders are linked to a variety of biological, psychological, and contextual causes and consequences. Laboratory studies have elucidated the importance of several key physiological and behavioral biomarkers in the study of psychiatric disorders, but much less is known about the role of these biomarkers in naturalistic settings. These gaps are largely driven by methodological barriers to assessing biomarker data rapidly, reliably, and frequently outside the clinic or laboratory. Mobile health (mHealth) tools offer new opportunities to study relevant biomarkers in concert with other types of data (e.g., self-reports, global positioning system data). This review provides an overview on the state of this emerging field and describes examples from the literature where mHealth tools have been used to measure a wide array of biomarkers in the context of psychiatric functioning (e.g., psychological stress, anxiety, autism, substance use). We also outline advantages and special considerations for incorporating mHealth tools for remote biomarker measurement into studies of psychiatric illness and treatment and identify several specific opportunities for expanding this promising methodology. Integrating mHealth tools into this area may dramatically improve psychiatric science and facilitate highly personalized clinical care of psychiatric disorders

Exploring students’ emotional state during a test-related task using wearable electroencephalogram

Using wireless sensors for brain activity, brain signals associated with the mood states of engineering students have been captured before and during the taking of a mathematics exam. The characterization of brain lobule activity related to arousal/valence states was analyzed from reports on the literature of the horizontal dimensions of pleasure-displeasure and vertical dimensions representing arousal-sleep. The results showed a direct relationship of the level of students’ arousal with the event of taking an exam as well as feelings of negative emotions during the exam presentation. The development of this research can lead to the implementation of controlled spaces for the presentation of students’ exams in which arousal/valence states can be controlled so that they do not affect their performance and the fulfillment of the goals, achievements or objectives established in a program or subject

Wearable feedback systems for rehabilitation

In this paper we describe LiveNet, a flexible wearable platform intended for long-term ambulatory health monitoring with real-time data streaming and context classification. Based on the MIT Wearable Computing Group's distributed mobile system architecture, LiveNet is a stable, accessible system that combines inexpensive, commodity hardware; a flexible sensor/peripheral interconnection bus; and a powerful, light-weight distributed sensing, classification, and inter-process communications software architecture to facilitate the development of distributed real-time multi-modal and context-aware applications. LiveNet is able to continuously monitor a wide range of physiological signals together with the user's activity and context, to develop a personalized, data-rich health profile of a user over time. We demonstrate the power and functionality of this platform by describing a number of health monitoring applications using the LiveNet system in a variety of clinical studies that are underway. Initial evaluations of these pilot experiments demonstrate the potential of using the LiveNet system for real-world applications in rehabilitation medicine

Healthcare in the Smart Home: A Study of Past, Present and Future

Open Access journalUbiquitous or Pervasive Computing is an increasingly used term throughout the technology industry and is beginning to enter the consumer electronics space in its most recent form under the umbrella term: “Internet of Things”. One area of focus is in augmenting the home with intelligent, networked sensors and computers to create a Smart Home which opens a host of possibilities for the role of tomorrow’s dwelling. As the world’s population continues to live longer and consequently experience more medical-related ailments, at the same time institutional healthcare is struggling to cope, the role of the Smart Home becomes paramount to monitoring a dweller’s health and providing any necessary intervention. This study looks at the history of Smart Home Healthcare, current research areas, and potential areas of future investigation. Unique categorisations are presented in Activities of Daily Living (ADL) and Personal Sensors, and a thorough look at the application of Smart Home Healthcare is presented. Technology can augment traditional methods of healthcare delivery and in some cases completely replace it. Costs can be reduced and medical adherence can be increased, all of which contribute to a more sustainable and effective model of care

Logging Stress and Anxiety Using a Gamified Mobile-based EMA Application, and Emotion Recognition Using a Personalized Machine Learning Approach

According to American Psychological Association (APA) more than 9 in 10 (94 percent) adults believe that stress can contribute to the development of major health problems, such as heart disease, depression, and obesity. Due to the subjective nature of stress, and anxiety, it has been demanding to measure these psychological issues accurately by only relying on objective means. In recent years, researchers have increasingly utilized computer vision techniques and machine learning algorithms to develop scalable and accessible solutions for remote mental health monitoring via web and mobile applications. To further enhance accuracy in the field of digital health and precision diagnostics, there is a need for personalized machine-learning approaches that focus on recognizing mental states based on individual characteristics, rather than relying solely on general-purpose solutions. This thesis focuses on conducting experiments aimed at recognizing and assessing levels of stress and anxiety in participants. In the initial phase of the study, a mobile application with broad applicability (compatible with both Android and iPhone platforms) is introduced (we called it STAND). This application serves the purpose of Ecological Momentary Assessment (EMA). Participants receive daily notifications through this smartphone-based app, which redirects them to a screen consisting of three components. These components include a question that prompts participants to indicate their current levels of stress and anxiety, a rating scale ranging from 1 to 10 for quantifying their response, and the ability to capture a selfie. The responses to the stress and anxiety questions, along with the corresponding selfie photographs, are then analyzed on an individual basis. This analysis focuses on exploring the relationships between self-reported stress and anxiety levels and potential facial expressions indicative of stress and anxiety, eye features such as pupil size variation and eye closure, and specific action units (AUs) observed in the frames over time. In addition to its primary functions, the mobile app also gathers sensor data, including accelerometer and gyroscope readings, on a daily basis. This data holds potential for further analysis related to stress and anxiety. Furthermore, apart from capturing selfie photographs, participants have the option to upload video recordings of themselves while engaging in two neuropsychological games. These recorded videos are then subjected to analysis in order to extract pertinent features that can be utilized for binary classification of stress and anxiety (i.e., stress and anxiety recognition). The participants that will be selected for this phase are students aged between 18 and 38, who have received recent clinical diagnoses indicating specific stress and anxiety levels. In order to enhance user engagement in the intervention, gamified elements - an emerging trend to influence user behavior and lifestyle - has been utilized. Incorporating gamified elements into non-game contexts (e.g., health-related) has gained overwhelming popularity during the last few years which has made the interventions more delightful, engaging, and motivating. In the subsequent phase of this research, we conducted an AI experiment employing a personalized machine learning approach to perform emotion recognition on an established dataset called Emognition. This experiment served as a simulation of the future analysis that will be conducted as part of a more comprehensive study focusing on stress and anxiety recognition. The outcomes of the emotion recognition experiment in this study highlight the effectiveness of personalized machine learning techniques and bear significance for the development of future diagnostic endeavors. For training purposes, we selected three models, namely KNN, Random Forest, and MLP. The preliminary performance accuracy results for the experiment were 93%, 95%, and 87% respectively for these models

A multimodal dataset of real world mobility activities in Parkinson’s disease

Parkinson’s disease (PD) is a neurodegenerative disorder characterised by motor symptoms such as gait dysfunction and postural instability. Technological tools to continuously monitor outcomes could capture the hour-by-hour symptom fluctuations of PD. Development of such tools is hampered by the lack of labelled datasets from home settings. To this end, we propose REMAP (REal-world Mobility Activities in Parkinson’s disease), a human rater-labelled dataset collected in a home-like setting. It includes people with and without PD doing sit-to-stand transitions and turns in gait. These discrete activities are captured from periods of free-living (unobserved, unstructured) and during clinical assessments. The PD participants withheld their dopaminergic medications for a time (causing increased symptoms), so their activities are labelled as being “on” or “off” medications. Accelerometry from wrist-worn wearables and skeleton pose video data is included. We present an open dataset, where the data is coarsened to reduce re-identifiability, and a controlled dataset available on application which contains more refined data. A use-case for the data to estimate sit-to-stand speed and duration is illustrated

Tunteiden Havaitseminen Arkielämässä Koneoppimisen ja Puettavien Laitteiden Avulla

Tavoitteet. Tämän tutkimuksen tavoitteena on arvioida tunteiden havaitsemisen mahdollisuutta arkielämässä puettavien laitteiden ja koneoppimismallien avulla. Tunnetiloilla on tärkeä rooli päätöksenteossa, havaitsemisessa ja käyttäytymisessä, mikä tekee objektiivisesta tunnetilojen havaitsemisesta arvokkaan tavoitteen, sekä mahdollisten sovellusten että tunnetiloja koskevan ymmärryksen syventämisen kannalta. Tunnetiloihin usein liittyy mitattavissa olevia fysiologisia ja käyttäymisen muutoksia, mikä mahdollistaa koneoppimismallien kouluttamisen muutoksia aiheuttaneen tunnetilan havaitsemiseksi. Suurin osa tunteiden havaitsemiseen liittyvästä tutkimuksesta on toteutettu laboratorio-olosuhteissa käyttämällä tunteita herättäviä ärsykkeitä tai tehtäviä, mikä herättää kysymyksen siitä että yleistyvätkö näissä olosuhteissa saadut tulokset arkielämään. Vaikka puettavien laitteiden ja kännykkäkyselyiden kehittyminen on helpottanut aiheen tutkimista arkielämässä, tutkimusta tässä ympäristössä on vielä niukasti. Tässä tutkimuksessa itseraportoituja tunnetiloja ennustetaan koneoppimismallien avulla arkielämässä havaittavissa olevien tunnetilojen selvittämiseksi. Lisäksi tutkimuksessa käytetään mallintulkintamenetelmiä mallien hyödyntämien yhteyksien tunnistamiseksi. Metodit. Aineisto tätä tutkielmaa varten on peräisin tutkimuksesta joka suoritettiin osana Helsingin Yliopiston ja VTT:n Sisu at Work projektia, missä 82:ta tietotyöläistä neljästä suomalaisesta organisaatiosta tutkittiin kolmen viikon ajan. Osallistujilla oli jakson aikana käytettävissään mittalaitteet jotka mittasivat fotoplethysmografiaa (PPG), ihon sähkönjohtavuutta (EDA) ja kiihtyvyysanturi (ACC) signaaleita, lisäksi heille esitettiin kysymyksiä koetuista tunnetiloista kolmesti päivässä puhelinsovelluksen avulla. Signaalinkäsittelymenetelmiä sovellettiin signaaleissa esiintyvien liikeartefaktien ja muiden ongelmien korjaamiseksi. Sykettä (HR) ja sykevälinvaihtelua (HRV) kuvaavia piirteitä irroitettiin PPG signaalista, fysiologista aktivaatiota kuvaavia piirteitä EDA signaalista, sekä liikettä kuvaavia piirteitä ACC signaalista. Seuraavaksi koneoppimismalleja koulutettiin ennustamaan raportoituja tunnetiloja irroitetujen piirteiden avulla. Mallien suoriutumista vertailtiin suhteessa odotusarvoihin havaittavissa olevien tunnetilojen määrittämiseksi. Lisäksi permutaatiotärkeyttä sekä Shapley additive explanations (SHAP) arvoja hyödynnettiin malleille tärkeiden yhteyksien selvittämiseksi. Tulokset ja johtopäätökset. Mallit tunnetiloille virkeä, keskittynyt ja innostunut paransivat suoriutumistaan yli odotusarvon, joista mallit tunnetilalle virkeä paransivat suoriutumista tilastollisesti merkitsevästi. Permutaatiotärkeys korosti liike- ja HRV-piirteiden merkitystä, kun SHAP arvojen tarkastelu nosti esiin matalan liikkeen, matalan EDA:n, sekä korkean HRV:n merkityksen mallien ennusteille. Nämä tulokset ovat lupaavia korkean aktivaation positiivisten tunnetilojen havaitsemiselle arkielämässä, sekä nostavat esiin mahdollisia yhteyksiä jatkotutkimusta varten.Objectives. This study aims to evaluate feasibility of affect detection in daily life using wearable devices and machine learning models. Affective states play an important role in decision making, perception and behaviour, making objective detection of affective states a desirable goal both for potential applications and as a way to gain insight into affective phenomena. Affective states have been found to have measurable physiological and behavioral changes, which allows training of machine learning models for detecting the underlying affects. Majority of affect detection studies have been conducted in laboratory conditions using affect elicitation stimuli or tasks, raising the question whether results from these studies will generalize to daily life. Although development of wearable devices and mobile surveys have facilitated evaluation in the context of daily life, research here remains sparse. In this study, self-reported affective states are predicted using machine learning models to identify which affective states can be detected in daily life. Additionally, model interpretation methods will be used to identify which relationships the models found important for their predictions. Methods. Data for this thesis came from a study conducted as a part of Sisu at Work project between University of Helsinki and VTT, where 82 knowledge workers from four Finnish organizations were studied for a period of three weeks. During this period, the participants were queried by mobile surveys about their affective states thrice a day, while they also used wearable devices to record photoplethysmography (PPG), electrodermal activity (EDA) and accelerometry (ACC) signals. A signal processing pipeline was implemented to deal with movement artefacts and other issues with the data. Features describing heart rate (HR) and heart rate variation (HRV) were extraced from PPG, physiological activation from EDA and movement from ACC signals. Models were then fitted to predict the reported affective states using the extracted features. Model performance was compared against a baseline to identify which affects could be reliably detected, while permutation importance and Shapley additive explanations (SHAP) values were used to identify important relationships established by the models. Results and conclusions. Models for affective state vigor showed improvements over baseline with statistical significance, while improvements were also noted for affects focused and enthusiastic. Permutation importance highlighted the significance of movement and HRV features, while examination of SHAP values indicated that low movement, low EDA and high HRV impacted model predictions the most. These results indicate potential for detecting high activation affective states in daily life and propose potential relationships for future research