Clinical Decision Support Systems with Game-based Environments, Monitoring Symptoms of Parkinson’s Disease with Exergames

Parkinson’s Disease (PD) is a malady caused by progressive neuronal degeneration, deriving in several physical and cognitive symptoms that worsen with time. Like many other chronic diseases, it requires constant monitoring to perform medication and therapeutic adjustments. This is due to the significant variability in PD symptomatology and progress between patients. At the moment, this monitoring requires substantial participation from caregivers and numerous clinic visits. Personal diaries and questionnaires are used as data sources for medication and therapeutic adjustments. The subjectivity in these data sources leads to suboptimal clinical decisions. Therefore, more objective data sources are required to better monitor the progress of individual PD patients. A potential contribution towards more objective monitoring of PD is clinical decision support systems. These systems employ sensors and classification techniques to provide caregivers with objective information for their decision-making. This leads to more objective assessments of patient improvement or deterioration, resulting in better adjusted medication and therapeutic plans. Hereby, the need to encourage patients to actively and regularly provide data for remote monitoring remains a significant challenge. To address this challenge, the goal of this thesis is to combine clinical decision support systems with game-based environments. More specifically, serious games in the form of exergames, active video games that involve physical exercise, shall be used to deliver objective data for PD monitoring and therapy. Exergames increase engagement while combining physical and cognitive tasks. This combination, known as dual-tasking, has been proven to improve rehabilitation outcomes in PD: recent randomized clinical trials on exergame-based rehabilitation in PD show improvements in clinical outcomes that are equal or superior to those of traditional rehabilitation. In this thesis, we present an exergame-based clinical decision support system model to monitor symptoms of PD. This model provides both objective information on PD symptoms and an engaging environment for the patients. The model is elaborated, prototypically implemented and validated in the context of two of the most prominent symptoms of PD: (1) balance and gait, as well as (2) hand tremor and slowness of movement (bradykinesia). While balance and gait affections increase the risk of falling, hand tremors and bradykinesia affect hand dexterity. We employ Wii Balance Boards and Leap Motion sensors, and digitalize aspects of current clinical standards used to assess PD symptoms. In addition, we present two dual-tasking exergames: PDDanceCity for balance and gait, and PDPuzzleTable for tremor and bradykinesia. We evaluate the capability of our system for assessing the risk of falling and the severity of tremor in comparison with clinical standards. We also explore the statistical significance and effect size of the data we collect from PD patients and healthy controls. We demonstrate that the presented approach can predict an increased risk of falling and estimate tremor severity. Also, the target population shows a good acceptance of PDDanceCity and PDPuzzleTable. In summary, our results indicate a clear feasibility to implement this system for PD. Nevertheless, long-term randomized clinical trials are required to evaluate the potential of PDDanceCity and PDPuzzleTable for physical and cognitive rehabilitation effects

EEG Feature Variations under Stress Situations

The goal of this study is to identify EEG parameters and electrode positions with the highest significant values to differentiate between tasks and relax periods. Different signals were recorded as 12 subjects are doing arithmetic and memory tasks under stress condition. The test consisted of an initial and final 5-minute relax periods and three 4-minute performance phases with increased stress level. q and a bands concentrated mainly features whose variation were significant, and F3 and P4 were the best positions to distinguish between performed tasks and arousal level

Smart workplaces: a system proposal for stress management

Over the past last decades of contemporary society, workplaces have become the primary source of many health issues, leading to mental problems such as stress, depression, and anxiety. Among the others, environmental aspects have shown to be the causes of stress, illness, and lack of productivity. With the arrival of new technologies, especially in the smart workplaces field, most studies have focused on investigating the building energy efficiency models and human thermal comfort. However, little has been applied to occupants’ stress recognition and well-being overall. Due to this fact, this present study aims to propose a stress management solution for an interactive design system that allows the adapting of comfortable environmental conditions according to the user preferences by measuring in real-time the environmental and biological characteristics, thereby helping to prevent stress, as well as to enable users to cope stress when being stressed. The secondary objective will focus on evaluating one part of the system: the mobile application. The proposed system uses several usability methods to identify users’ needs, behavior, and expectations from the user-centered design approach. Applied methods, such as User Research, Card Sorting, and Expert Review, allowed us to evaluate the design system according to Heuristics Analysis, resulting in improved usability of interfaces and experience. The study presents the research results, the design interface, and usability tests. According to the User Research results, temperature and noise are the most common environmental stressors among the users causing stress and uncomfortable conditions to work in, and the preference for physical activities over the digital solutions for coping with stress. Additionally, the System Usability Scale (SUS) results identified that the system’s usability was measured as “excellent” and “acceptable” with a final score of 88 points out of the 100. It is expected that these conclusions can contribute to future investigations in the smart workplaces study field and their interaction with the people placed there.Nas últimas décadas da sociedade contemporânea, o local de trabalho tem se tornado principal fonte de muitos problemas de saúde mental, como o stress, depressão e ansiedade. Os aspetos ambientais têm se revelado como as causas de stress, doenças, falta de produtividade, entre outros. Atualmente, com a chegada de novas tecnologias, principalmente na área de locais de trabalho inteligentes, a maioria dos estudos tem se concentrado na investigação de modelos de eficiência energética de edifícios e conforto térmico humano. No entanto, pouco foi aplicado ao reconhecimento do stress dos ocupantes e ao bem-estar geral das pessoas. Diante disso, o objetivo principal é propor um sistema de design de gestão do stress para um sistema de design interativo que permita adaptar as condições ambientais de acordo com as preferências de utilizador, medindo em tempo real as características ambientais e biológicas, auxiliando assim na prevenção de stress, bem como ajuda os utilizadores a lidar com o stress quando estão sob o mesmo. O segundo objetivo é desenhar e avaliar uma parte do projeto — o protótipo da aplicação móvel através da realização de testes de usabilidade. O sistema proposto resulta da abordagem de design centrado no utilizador, utilizando diversos métodos de usabilidade para identificar as necessidades, comportamentos e as expectativas dos utilizadores. Métodos aplicados, como Pesquisa de Usuário, Card Sorting e Revisão de Especialistas, permitiram avaliar o sistema de design de acordo com a análise heurística, resultando numa melhoria na usabilidade das interfaces e experiência. O estudo apresenta os resultados da pesquisa, a interface do design e os testes de usabilidade. De acordo com os resultados de User Research, a temperatura e o ruído são os stressores ambientais mais comuns entre os utilizadores, causando stresse e condições menos favoráveis para trabalhar, igualmente existe uma preferência por atividades físicas sobre as soluções digitais na gestão do stresse. Adicionalmente, os resultados de System Usability Scale (SUS) identificaram a usabilidade do sistema de design como “excelente” e “aceitável” com pontuação final de 88 pontos em 100. É esperado que essas conclusões possam contribuir para futuras investigações no campo de estudo dos smart workplaces e sua interação com os utilizadores

Automatic Pain Assessment by Learning from Multiple Biopotentials

Kivun täsmällinen arviointi on tärkeää kivunhallinnassa, erityisesti sairaan- hoitoa vaativille ipupotilaille. Kipu on subjektiivista, sillä se ei ole pelkästään aistituntemus, vaan siihen saattaa liittyä myös tunnekokemuksia. Tällöin itsearviointiin perustuvat kipuasteikot ovat tärkein työkalu, niin auan kun potilas pystyy kokemuksensa arvioimaan. Arviointi on kuitenkin haasteellista potilailla, jotka eivät itse pysty kertomaan kivustaan. Kliinisessä hoito- työssä kipua pyritään objektiivisesti arvioimaan esimerkiksi havainnoimalla fysiologisia muuttujia kuten sykettä ja käyttäytymistä esimerkiksi potilaan kasvonilmeiden perusteella. Tutkimuksen päätavoitteena on automatisoida arviointiprosessi hyödyntämällä koneoppimismenetelmiä yhdessä biosignaalien prosessointnin kanssa. Tavoitteen saavuttamiseksi mitattiin autonomista keskushermoston toimintaa kuvastavia biopotentiaaleja: sydänsähkökäyrää, galvaanista ihoreaktiota ja kasvolihasliikkeitä mittaavaa lihassähkökäyrää. Mittaukset tehtiin terveillä vapaaehtoisilla, joille aiheutettiin kokeellista kipuärsykettä. Järestelmän kehittämiseen tarvittavaa tietokantaa varten rakennettiin biopotentiaaleja keräävä Internet of Things -pohjainen tallennusjärjestelmä. Koostetun tietokannan avulla kehitettiin biosignaaleille prosessointimenetelmä jatku- vaan kivun arviointiin. Signaaleista eroteltiin piirteitä sekuntitasoon mukautetuilla aikaikkunoilla. Piirteet visualisoitiin ja tarkasteltiin eri luokittelijoilla kivun ja kiputason tunnistamiseksi. Parhailla luokittelumenetelmillä saavutettiin kivuntunnistukseen 90% herkkyyskyky (sensitivity) ja 84% erottelukyky (specificity) ja kivun voimakkuuden arviointiin 62,5% tarkkuus (accuracy). Tulokset vahvistavat kyseisen käsittelytavan käyttökelpoisuuden erityis- esti tunnistettaessa kipua yksittäisessä arviointi-ikkunassa. Tutkimus vahvistaa biopotentiaalien avulla kehitettävän automatisoidun kivun arvioinnin toteutettavuuden kokeellisella kivulla, rohkaisten etenemään todellisen kivun tutkimiseen samoilla menetelmillä. Menetelmää kehitettäessä suoritettiin lisäksi vertailua ja yhteenvetoa automaattiseen kivuntunnistukseen kehitettyjen eri tutkimusten välisistä samankaltaisuuksista ja eroista. Tarkastelussa löytyi signaalien eroavaisuuksien lisäksi tutkimusmuotojen aiheuttamaa eroa arviointitavoitteisiin, mikä hankaloitti tutkimusten vertailua. Lisäksi pohdit- tiin mitkä perinteisten prosessointitapojen osiot rajoittavat tai edistävät ennustekykyä ja miten, sekä tuoko optimointi läpimurtoa järjestelmän näkökulmasta.Accurate pain assessment plays an important role in proper pain management, especially among hospitalized people experience acute pain. Pain is subjective in nature which is not only a sensory feeling but could also combine affective factors. Therefore self-report pain scales are the main assessment tools as long as patients are able to self-report. However, it remains a challenge to assess the pain from the patients who cannot self-report. In clinical practice, physiological parameters like heart rate and pain behaviors including facial expressions are observed as empirical references to infer pain objectively. The main aim of this study is to automate such process by leveraging machine learning methods and biosignal processing. To achieve this goal, biopotentials reflecting autonomic nervous system activities including electrocardiogram and galvanic skin response, and facial expressions measured with facial electromyograms were recorded from healthy volunteers undergoing experimental pain stimulus. IoT-enabled biopotential acquisition systems were developed to build the database aiming at providing compact and wearable solutions. Using the database, a biosignal processing flow was developed for continuous pain estimation. Signal features were extracted with customized time window lengths and updated every second. The extracted features were visualized and fed into multiple classifiers trained to estimate the presence of pain and pain intensity separately. Among the tested classifiers, the best pain presence estimating sensitivity achieved was 90% (specificity 84%) and the best pain intensity estimation accuracy achieved was 62.5%. The results show the validity of the proposed processing flow, especially in pain presence estimation at window level. This study adds one more piece of evidence on the feasibility of developing an automatic pain assessment tool from biopotentials, thus providing the confidence to move forward to real pain cases. In addition to the method development, the similarities and differences between automatic pain assessment studies were compared and summarized. It was found that in addition to the diversity of signals, the estimation goals also differed as a result of different study designs which made cross dataset comparison challenging. We also tried to discuss which parts in the classical processing flow would limit or boost the prediction performance and whether optimization can bring a breakthrough from the system’s perspective

Study of the light’s dazzling effect on the EEG signal of subjects performing tasks that require concentration

Tese de mestrado integrado, Engenharia Biomédica e Biofísica (Sinais e Imagens Médicas) Universidade de Lisboa, Faculdade de Ciências, 2019The objective of this work is to study the effect of luminous glare on the electroencephalographic (EEG) signals of subjects that perform concentration-based tasks. The increasing access to high-power and directional light sources (such as laser pointers, but also some flashlights) has led to a growing concern with the potential effects of its use. More than the direct damaging of the retina, the focus has been directed at the effects related to the change in states of concentration on individuals performing tasks whose concentration is critical (such as helicopter pilots or heavy vehicles drivers). This effect is known as ”dazzling” and is typically a temporary deleterious effect on the ability to see or concentrate. However, while damage to the retina can be quantified, glare effects, being indirect (based on the effect on the execution of a given task), are typically qualitative (or at least of more subjective quantification). In this context, the use of brain-computer interfaces capable of analyzing the brain response to external stimuli, opens a door towards the creation of a new tool to evaluate the effects of dazzle. Its potential was evaluated by defining a set of strategies involving the illumination process, EEG signal recording and analysis. A continuous performance task commonly used as an assessment in cognitive neuroscience (N-back) was used to test the attention under the effect of dazzling, in parallel with EEG signals acquisition. Statistical data analysis was performed with the R programming language. ANOVA statistical significant results (p<0.001) for answer scores and latency were obtained for differences between the levels of difficulty, both with or without dazzling. Tukey’s test further revealed that these statistical differences were on the 0-back/2-back and 1-back/2-back pairs (p<0.005). The differences in the pair 0-back/1-back were not significant. Peak band frequency statistical tests were not significant with or without dazzling. Statistical differences were found between dazzling conditions for the frequency band power. For the 0-back and 1-back levels, with the AF7-Fp1 electrode pair, T-student tests resulted in an alpha band frequency power increase (p<0.003, in both cases). The electrode pair AF8-Fp2 resulted in an alfa and beta frequency band increase for the 1-back level (p<0.014 and p<0.029, respectively). These results suggest that concentration is affected by dazzling and can be quantified by means of measuring the change in alpha and beta frequency band power. This technique holds potential and, if further researched and developed, may constitute an effective way of measuring the degree of loss of concentration under the effect of dazzling

Low-Cost Sensors and Biological Signals

Many sensors are currently available at prices lower than USD 100 and cover a wide range of biological signals: motion, muscle activity, heart rate, etc. Such low-cost sensors have metrological features allowing them to be used in everyday life and clinical applications, where gold-standard material is both too expensive and time-consuming to be used. The selected papers present current applications of low-cost sensors in domains such as physiotherapy, rehabilitation, and affective technologies. The results cover various aspects of low-cost sensor technology from hardware design to software optimization

Data-driven multivariate and multiscale methods for brain computer interface

This thesis focuses on the development of data-driven multivariate and multiscale methods for brain computer interface (BCI) systems. The electroencephalogram (EEG), the most convenient means to measure neurophysiological activity due to its noninvasive nature, is mainly considered. The nonlinearity and nonstationarity inherent in EEG and its multichannel recording nature require a new set of data-driven multivariate techniques to estimate more accurately features for enhanced BCI operation. Also, a long term goal is to enable an alternative EEG recording strategy for achieving long-term and portable monitoring. Empirical mode decomposition (EMD) and local mean decomposition (LMD), fully data-driven adaptive tools, are considered to decompose the nonlinear and nonstationary EEG signal into a set of components which are highly localised in time and frequency. It is shown that the complex and multivariate extensions of EMD, which can exploit common oscillatory modes within multivariate (multichannel) data, can be used to accurately estimate and compare the amplitude and phase information among multiple sources, a key for the feature extraction of BCI system. A complex extension of local mean decomposition is also introduced and its operation is illustrated on two channel neuronal spike streams. Common spatial pattern (CSP), a standard feature extraction technique for BCI application, is also extended to complex domain using the augmented complex statistics. Depending on the circularity/noncircularity of a complex signal, one of the complex CSP algorithms can be chosen to produce the best classification performance between two different EEG classes. Using these complex and multivariate algorithms, two cognitive brain studies are investigated for more natural and intuitive design of advanced BCI systems. Firstly, a Yarbus-style auditory selective attention experiment is introduced to measure the user attention to a sound source among a mixture of sound stimuli, which is aimed at improving the usefulness of hearing instruments such as hearing aid. Secondly, emotion experiments elicited by taste and taste recall are examined to determine the pleasure and displeasure of a food for the implementation of affective computing. The separation between two emotional responses is examined using real and complex-valued common spatial pattern methods. Finally, we introduce a novel approach to brain monitoring based on EEG recordings from within the ear canal, embedded on a custom made hearing aid earplug. The new platform promises the possibility of both short- and long-term continuous use for standard brain monitoring and interfacing applications