TOBE: Tangible Out-of-Body Experience

We propose a toolkit for creating Tangible Out-of-Body Experiences: exposing the inner states of users using physiological signals such as heart rate or brain activity. Tobe can take the form of a tangible avatar displaying live physiological readings to reflect on ourselves and others. Such a toolkit could be used by researchers and designers to create a multitude of potential tangible applications, including (but not limited to) educational tools about Science Technologies Engineering and Mathematics (STEM) and cognitive science, medical applications or entertainment and social experiences with one or several users or Tobes involved. Through a co-design approach, we investigated how everyday people picture their physiology and we validated the acceptability of Tobe in a scientific museum. We also give a practical example where two users relax together, with insights on how Tobe helped them to synchronize their signals and share a moment

Stress level assessment with non-intrusive sensors

Mención Internacional en el título de doctorStress is an involuntary reaction where the human body changes from a calm state to an excited state in order to preserve the integrity of the organism. Small amount of stress should be good to became entrepreneur and learn new ways of thinking, but continuous stress can carry an array of daily risks, such as, cardiovascular diseases, hair loss, diabetes or immune dysregulation. Recognize how, when and where it occurs has become a step in stress assessment. Stress recognition starts from 1973 until now. This disease has become a problem in recent years because has increased the number of cases, especially in workers where his/her performance decreases. Stress reactions are provoked for the Autonomous Nervous System (ANS) and one way to estimate it could be found in physiological signals. A list of a variety wearable sensor is presented to capture these reactions, trying to minimize the risk of distraction due to external factors. The aim of this work thesis is to detect stress for level assessment. A combination of different physiological signals is selected to extract stress feature an classify in a rating scale from relax to breakdown situations. This thesis proposes a new feature extraction model to understand physiological Galvanic Skin Response (GSR) reactions. Last methods conclude in incongruent results that are not interpretable. This model propose a robust algorithm that can be used in real-time (low time computability) and results are sparse in time to obtain an easily statistical and graphical interpretation. Signal processing methods of heart rhythm and hormone cortisol are included to develop a robust feature extraction method of stress reactions. A combination of electrodermal, heart and hormone analysis is presented to know in real-time the state of the individual. These features have been selected because the acquisition is non-intrusive avoiding other factor such as distractions. This thesis is application-focused and highly multidisciplinary. A complete feature extraction model is presented including the new electrodermal model named and usual heart rhythm techniques. Three experiments were evaluated: a) a feature selection model using neurocognitive games, b) a stress classifier in time during public talks, and c) a real-time stress assessment classifier in a five-star rating scale. This thesis improve stress detection overcoming a system to capture physiological responses, analyze and conclude a stress assessment decision. We discussed past state of the art and propose a new method of feature extraction using signal processing improvements. Three different scenarios were evaluated to confirm the achievement of aims proposed.Programa Oficial de Doctorado en Multimedia y ComunicacionesPresidente: Joaquín Míguez Arenas.- Secretario: Luis Ignacio Santamaría Caballero.- Vocal: Mª Isabel Valera Martíne

On-body Sensing and Signal Analysis for User Experience Recognition in Human-Machine Interaction

—In this paper, a new algorithm is proposed for recognition of user experience through emotion detection using physiological signals, for application in human-machine interaction. The algorithm recognizes user’s emotion quality and intensity in a two dimensional emotion space continuously. The continuous recognition of the user’s emotion during human-machine interaction will enable the machine to adapt its activity based on the user’s emotion in a real-time manner, thus improving user experience. The emotion model underlying the proposed algorithm is one of the most recent emotion models, which models emotion’s intensity and quality in a continuous two-dimensional space of valance and arousal axes. Using only two physiological signals, which are correlated to the valance and arousal axes of the emotion space, is among the contributions of this paper. Prediction of emotion through physiological signals has the advantage of elimination of social masking and making the prediction more reliable. The key advantage of the proposed algorithm over other algorithms presented to date is the use of the least number of modalities (only two physiological signals) to predict the quality and intensity of emotion continuously in time, and using the most recent widely accepted emotion model

Mobile psychiatry: Personalised Ambient Monitoring for the mentally ill

Mental health has long been a neglected problem in global healthcare. The social and economic impacts of conditions affecting the mind are still underestimated. However, in recent years it is becoming more apparent that mental disorders are a growing global concern that is not to be trivialised. Considering the rising burden of psychiatric illnesses, there is a necessity of developing novel services and researching effective means of providing interventions to sufferers. Such novel services could include technology-based solutions already used in other healthcare applications but are yet to make their way into standard psychiatric practice. This thesis presents a study on how pervasive technology can be utilised to devise an “early warning” system for patients with bipolar disorder. The system, containing wearable and environmental sensors, would collect behavioural data and use it to inform the user about subtle changes that might indicate an upcoming episode. To test the feasibility of the concept a prototype system was devised, which was followed by trials including four healthy volunteers as well as a bipolar patient. The system included a number of sensory inputs including: accelerometer, light sensors, microphones, GPS tracking and motion detectors. The experiences from the trials led to a conclusion that a large number of sensors may result in incompliance from the users. Therefore, a separate investigation was launched into developing a methodology for detecting behavioural patterns in inputs possible to collect from a mobile phone alone. The premise being that a phone is an everyday use appliance and is likely to be carried and accepted by the patient. The trial revealed that monitoring GPS tracks and Bluetooth encounters has the potential of gaining an insight into a person’s social and behavioural patterns, which usually are strongly influenced by the course of bipolar disorder. Lessons learned during these proceedings amounted to a clearer concept of how a future personalised ambient monitoring system could improve the outcome of treatment of bipolar disorder as well as other psychiatric conditions

Mobile psychiatry: Personalised Ambient Monitoring for the mentally ill

Mental health has long been a neglected problem in global healthcare. The social and economic impacts of conditions affecting the mind are still underestimated. However, in recent years it is becoming more apparent that mental disorders are a growing global concern that is not to be trivialised. Considering the rising burden of psychiatric illnesses, there is a necessity of developing novel services and researching effective means of providing interventions to sufferers. Such novel services could include technology-based solutions already used in other healthcare applications but are yet to make their way into standard psychiatric practice. This thesis presents a study on how pervasive technology can be utilised to devise an “early warning” system for patients with bipolar disorder. The system, containing wearable and environmental sensors, would collect behavioural data and use it to inform the user about subtle changes that might indicate an upcoming episode. To test the feasibility of the concept a prototype system was devised, which was followed by trials including four healthy volunteers as well as a bipolar patient. The system included a number of sensory inputs including: accelerometer, light sensors, microphones, GPS tracking and motion detectors. The experiences from the trials led to a conclusion that a large number of sensors may result in incompliance from the users. Therefore, a separate investigation was launched into developing a methodology for detecting behavioural patterns in inputs possible to collect from a mobile phone alone. The premise being that a phone is an everyday use appliance and is likely to be carried and accepted by the patient. The trial revealed that monitoring GPS tracks and Bluetooth encounters has the potential of gaining an insight into a person’s social and behavioural patterns, which usually are strongly influenced by the course of bipolar disorder. Lessons learned during these proceedings amounted to a clearer concept of how a future personalised ambient monitoring system could improve the outcome of treatment of bipolar disorder as well as other psychiatric conditions

Next Generation Cognition-Aware Hearing Aid Devices With Microwave Sensors: Opportunities and Challenges

The strong association between hearing loss and cognitive decline has developed into a major health challenge that calls for early detection, diagnosis and prevention. Hearing loss usually results in severe health implications that include loss of mobility, communication problems and cognitive decline. This study provides an overview of the effects of hearing loss on cognition and progressive neurological disorders with a discussion on the future scope of microwave portable technologies in care homes arrangement. Moreover, the efficacy of hearing aids in reversing cognitive decline and dementia has been investigated. The interconnection between hearing loss, cognitive load and neurodegeneration is also explored. Furthermore, this study looks into the prospects of using portable microwave sensors for the detection and monitoring of cognitive load. For early detection of dementia, this study proposes the integration of microwave sensors with hearing aid devices. Implications and design challenges of portable antenna systems for neurodegeneration detection have also been considered. Future improvement areas regarding robust analysis and diagnosis, system accuracy and security, user-centricity and device privacy for a broader clinical implementation are also discussed

Digitizing arquetypal human expereience through physiological signals

The problem of capturing human experience is relevant in many application domains. In fact, the process of describing and sharing individual experience lies at the heart of human culture. This advancement came at a price of losing some of the multidimensional aspects of primary, bodily experience during its projection into the symbolic formThroughout the courses of our lives we learn a great deal of information about the world from other people's experience. Besides the ability to share utilitarian experience such as whether a particular plant is poisonous, humans have developed a sophisticated competency of social signaling that enables us to express and decode emotional experience. The natural way of sharing emotional experiences requires those who share to be co-present during this event. However, people have overcome the limitation of physical presence by creating a symbolic system of representations.Recent research in the field of affective computing has addressed the question of digitization and transmission of emotional experience through monitoring and interpretation of physiological signals. Although the outcomes of this research represent a great step forward in developing a technology that supports sharing of emotional experiences, they do not seem to help in preserving the original phenomenological experience during the aforementioned projection. This circumstance is explained by the fact that in affective computing the focus of investigation has been aimed at emotional experiences which can be consciously evaluated and described by individuals themselves. Therefore, generally speaking, applying an affective computing technique for capturing emotions of an individual is not a deeper or more precise way to project her experience into the symbolic form than asking this person to write down a description of her emotions on a piece of paper. One can say that so far the research in affective computing has aimed at delivering technology that could automate the projection but it has not considered the problem of improving the projection in order to preserve more of the multidimensional aspects of human experience.This dissertation examines whether human experience, which individuals are not able to consciously transpose into the symbolic representation, can still be captured using the techniques of affective computing.First, a theoretical framework for description of human experience which is not accessible for conscious awareness was formulated. This framework was based on the work of Carl Jung who introduced a model of a psyche that includes three levels: consciousness, the personal unconscious and the collective unconscious. Consciousness is the external layer of the psyche that consists of those thoughts and emotions which are available for one¿s conscious recollection. The personal unconscious represents a repository for all of an individual¿s feelings, memories, knowledge and thoughts that are not conscious at a given moment of time.The collective unconscious is a repository of universal modes and behaviors that are similar in all individuals. According to Jung, the collective unconscious is populated with archetypes. Archetypes are prototypical categories of objects, people and situations that existed across evolutionary time and in different cultures.Esta tesis doctoral examina si la experiencia humana, que los individuos no pueden transponer conscientemente a la representación simbólica, aún puede capturarse utilizando las técnicas de computación afectiva. Primero, se formula un marco teórico para la descripción de la experiencia humana que no es accesible para la conciencia consciente. Este marco se basó en el trabajo de Carl Jung, quien introdujo un modelo de psique que incluye tres niveles: la conciencia, el inconsciente personal y el inconsciente colectivo. Habiendo definido nuestro marco teórico, realizamos un experimento en el que se mostraron a los sujetos estímulos visuales y auditivos de bases de datos estandarizadas para la obtención de emociones conscientes. Aparte de los estímulos para las emociones conscientes, los sujetos fueron expuestos a estímulos que representaban el arquetipo del yo. Durante la presentación de los estímulos cardiovasculares se registraron las señales de los sujetos. Los resultados experimentales indicaron que las respuestas de la frecuencia cardíaca de los participantes fueron únicas para cada categoría de estímulos, incluido el arquetípico. Estos hallazgos dieron impulso a realizar otro estudio en el que se examinó un espectro más amplio de experiencias arquetípicas. En nuestro segundo estudio, hicimos un cambio de estímulos visuales y auditivos a estímulos audiovisuales porque se esperaba que los videos fueran más eficientes en la obtención de emociones conscientes y experiencias arquetípicas que las imágenes fijas o los sonidos. La cantidad de arquetipos aumentó y los sujetos en general fueron estimulados a sentir ocho experiencias arquetípicas diferentes. También preparamos estímulos para emociones conscientes. En este experimento, las señales fisiológicas incluyeron actividades cardiovasculares, electrodérmicas, respiratorias y temperatura de la piel. El análisis estadístico sugirió que las experiencias arquetípicas podrían diferenciarse en función de las activaciones fisiológicas. Además, se construyeron varios modelos de predicción basados en los datos fisiológicos recopilados. Estos modelos demostraron la capacidad de clasificar los arquetipos con una precisión que era considerablemente más alta que el nivel de probabilidad. Como los resultados del segundo estudio sugirieron una relación positiva entre las experiencias arquetípicas y las activaciones de señales fisiológicas, parecía razonable realizar otro estudio para confirmar la generalización de nuestros hallazgos. Sin embargo, antes de comenzar un nuevo experimento, se decidió construir una herramienta que pudiera facilitar la recopilación de datos fisiológicos y el reconocimiento de experiencias arquetípicas, así como de emociones conscientes. Tal herramienta nos ayudaría a nosotros y a otros investigadores a realizar experimentos sobre la experiencia humana. Nuestra herramienta funciona en "tablets" y admite la recopilación y el análisis de datos de sensores fisiológicos. El último estudio se realizó utilizando una metodología similar al segundo experimento con varias modificaciones que tenían como objetivo obtener resultados más sólidos. El esfuerzo de realizar este estudio se redujo considerablemente al usar la herramienta desarrollada. Durante el experimento, sólo medimos las actividades cardiovasculares y electrodérmicas de los sujetos porque nuestros experimentos anteriores mostraron que estas dos señales contribuyeron significativamente a la clasificación de las emociones conscientes y las experiencias arquetípicas. El análisis estadístico indicó una relación significativa entre los arquetipos retratados en los videos y las respuestas fisiológicas de los sujetos. Además, utilizando métodos de minería de datos, creamos modelos de predicción que fueron capaces de reconocer las experiencias arquetípicas con una precisión menor que en el segundo estudio, pero todavía considerablemente..