Ensemble Machine Learning Model Trained on a New Synthesized Dataset Generalizes Well for Stress Prediction Using Wearable Devices

Introduction. We investigate the generalization ability of models built on datasets containing a small number of subjects, recorded in single study protocols. Next, we propose and evaluate methods combining these datasets into a single, large dataset. Finally, we propose and evaluate the use of ensemble techniques by combining gradient boosting with an artificial neural network to measure predictive power on new, unseen data. Methods. Sensor biomarker data from six public datasets were utilized in this study. To test model generalization, we developed a gradient boosting model trained on one dataset (SWELL), and tested its predictive power on two datasets previously used in other studies (WESAD, NEURO). Next, we merged four small datasets, i.e. (SWELL, NEURO, WESAD, UBFC-Phys), to provide a combined total of 99 subjects,. In addition, we utilized random sampling combined with another dataset (EXAM) to build a larger training dataset consisting of 200 synthesized subjects,. Finally, we developed an ensemble model that combines our gradient boosting model with an artificial neural network, and tested it on two additional, unseen publicly available stress datasets (WESAD and Toadstool). Results. Our method delivers a robust stress measurement system capable of achieving 85% predictive accuracy on new, unseen validation data, achieving a 25% performance improvement over single models trained on small datasets. Conclusion. Models trained on small, single study protocol datasets do not generalize well for use on new, unseen data and lack statistical power. Ma-chine learning models trained on a dataset containing a larger number of varied study subjects capture physiological variance better, resulting in more robust stress detection.Comment: 37 pages, 11 figure

Machine Learning for Stress Monitoring from Wearable Devices: A Systematic Literature Review

Introduction. The stress response has both subjective, psychological and objectively measurable, biological components. Both of them can be expressed differently from person to person, complicating the development of a generic stress measurement model. This is further compounded by the lack of large, labeled datasets that can be utilized to build machine learning models for accurately detecting periods and levels of stress. The aim of this review is to provide an overview of the current state of stress detection and monitoring using wearable devices, and where applicable, machine learning techniques utilized. Methods. This study reviewed published works contributing and/or using datasets designed for detecting stress and their associated machine learning methods, with a systematic review and meta-analysis of those that utilized wearable sensor data as stress biomarkers. The electronic databases of Google Scholar, Crossref, DOAJ and PubMed were searched for relevant articles and a total of 24 articles were identified and included in the final analysis. The reviewed works were synthesized into three categories of publicly available stress datasets, machine learning, and future research directions. Results. A wide variety of study-specific test and measurement protocols were noted in the literature. A number of public datasets were identified that are labeled for stress detection. In addition, we discuss that previous works show shortcomings in areas such as their labeling protocols, lack of statistical power, validity of stress biomarkers, and generalization ability. Conclusion. Generalization of existing machine learning models still require further study, and research in this area will continue to provide improvements as newer and more substantial datasets become available for study.Comment: 50 pages, 8 figure

Una primera aproximación hacia la computación afectiva en entornos de realidad virtual multi-modales e interactivos

[ES] La computación afectiva es un campo de la informática con muchas aplicaciones por desarrollar y explotar. En este trabajo aplicaremos la computación afectiva a entornos de Realidad Virtual (RV) interactivos para estudiar la respuesta emotiva de los usuarios a distintos estímulos. En primer lugar, se ha creado un dataset propio, recopilando los datos fisiológicos de distintos usuarios tras exponerlos a distintos estímulos con tal de provocarles emociones, junto con sus respuestas a cuestionarios del tipo “Self-Assessment Manikin” para inferir las emociones sentidas. Estos datos fueron recopilados finalmente con un prototipo creado con una placa Arduino y varios sensores conectados y programados. Dicho dataset se ha utilizado posteriormente para para crear un modelo de regresión de las emociones sentidas por cada usuario usando una estructura de redes neuronales LTSM (Long Short-Term Memory), y se ha aplicado para la observación de la respuesta emotiva a distintos estímulos en escenarios de RV interactivos que se han preparado. Entre los estímulos comparados en los escenarios de RV están el uso de audio máquina o el uso de audio humano, el uso de distintos tipos de subtítulos, y el uso de texto o de audio para describir puntos de interés. En cuanto a los resultados, los estímulos seleccionados no han provocado una gran respuesta emotiva por parte del usuario. Por otra parte, el modelo de regresión ha tenido resultados aceptables a la hora de estimar la respuesta emotiva de los usuarios en base a sus métricas fisiológicas. Se espera que este estudio preliminar abra la puerta a una nueva línea de investigación en esta área, materializándose en una Tesis Doctoral. Los resultados obtenidos no han sido conclusivos por falta de medios, como el: reducido número de voluntarios para el estudio, y baja calidad de los sensores utilizados para recopilación de métricas (a falta de acceso a otros mejores), así como por las limitaciones de tiempo.[CA] La computació afectiva és un camp de la informàtica amb moltes aplicacions per desenvolupar i explotar. En aquest treball aplicarem la computació afectiva a entorns de Realitat Virtual (RV) interactius per estudiar la resposta emotiva dels usuaris a distints estímuls. En primer lloc, s'ha creat un dataset propi, recopilant les dades fisiològiques de distints usuaris, després d'exposar-los a distints estímuls per provocar-los emocions, junt amb les seues respostes a qüestionaris del tipus “Self-Assessment Manikin” per inferir les emocions sentides. Aquestes dades van ser recopilades finalment amb un prototip creat amb una placa Arduino i diversos sensors connectats i programats Aquest dataset s'ha utilitzat posteriorment per a crear un model de regressió de les emocions sentides usant una estructura de xarxes neuronals LTSM (Long Short-Term Memory), i s'ha aplicat per a l'observació de la resposta emotiva a cada estímul en els escenaris de RV que s'han preparat. Entre els estímuls comparats en els escenaris de RV estan l'ús d'àudio màquina o l'ús d'àudio humà, l'ús de diferents tipus de subtítols, i l'ús de text o d'àudio per a descriure punts d'interès. Quant als resultats, els estímuls seleccionats no han provocat una gran resposta emotiva per part de l'usuari. D'altra banda el model de regressió ha tingut resultats acceptables a l'hora d'estimar la resposta emotiva dels usuaris sobre la base de les seves mètriques fisiològiques. S'espera que aquest estudi preliminar siga el punt de partida a una nova línia d'investigació en aquesta àrea, materialitzant-se en una Tesi Doctoral. Els resultats obtinguts no han sigut conclusius per falta de mitjans, com un reduït nombre de voluntaris per a l'estudi, i baixa qualitat dels sensors utilitzats per a recopilació de mètrica (a falta d'accés a altres millors), així com per les limitacions temporals.[EN] Affective computing is a field of computing with many applications to develop and exploit. In this work we will apply affective computing to interactive Virtual Reality (VR) environments to study the emotional response of users to different stimuli. First, a dataset has been created, by collecting the physiological data from different users after exposing them to different stimuli to provoke emotions, together with their responses to “Self- Assessment Manikin” questionnaires to infer the emotions felt. This data was finally collected with a prototype created with an Arduino board and several sensors connected and programmed. This dataset has subsequently been used to create a regression model of the emotions felt by each user using an LTSM (Long Short-Term Memory) neural network structure, and it has been applied to observe the emotional response to each stimulus in the VR scenarios that have been prepared and presented to the users. Among the stimuli compared in VR scenarios are the use of computer-generated audio or the use of human audio, the use of different types of subtitles, and the use of text or audio to describe points of interest. Regarding the results, the selected stimuli have not elicited a great emotional response from the user. On the other hand, the regression model has had acceptable results when estimating the emotional response of users based on their physiological metrics. This preliminary study is expected to open the door to a new research line in this field, being further developed in a PhD Thesis. The obtained results have not been conclusive due to lack of means, like reduced number of volunteers for the study and low quality of the sensors used to collect the metrics (in the absence of access to better ones), as well as time constraints.Rus Arance, JAD. (2021). Una primera aproximación hacia la computación afectiva en entornos de realidad virtual multi-modales e interactivos. Universitat Politècnica de València. http://hdl.handle.net/10251/178155TFG