Acquisition, analysis and visualization of data from physiological sensors for biofeedback applications

With the latest advances in technology and the rise of physiological sensors for everyday life, biofeedback is celebrating its revival and is a topic of great interest. The aim of this thesis is a mash-up of biofeedback techniques, modern physiological sensors and 3D technology. It investigates how to create a flexible and reusable biofeedback framework that can be used as extendable platform for future physiological sensors and research projects. It results in a fully operational biofeedback system that can be used to improve body awareness and control. The thesis explains what biofeedback is, investigates physiological sensor modalities and recording techniques, and provides a comprehensive analysis of related work in this domain. Simultaneous acquisition of data from multiple physiological sensors introduces new data management challenges on how to access stored data in an efficient way while still having enough processing power available for data visualization. Rather than just mapping a single value from a sensor like in traditional biofeedback systems, the thesis explains how to create an interactive classification graph, where customizable classifiers combine results from signal processing and map them to one or multiple feedback scores. The thesis extends the traditional biofeedback loop by a control and adjust mechanism and encapsulates analysis and classification from visualization. The two tier architecture allows the creation of state-of-the-art visualizations with any rendering engine. Several sample visualizations are created, including a virtual reality scene using the Oculus Rift in order to investigate the impact of virtual reality in biofeedback. An evaluation with 8 participants, each doing 7 tests, showed that key for successful biofeedback are (1) interaction with a human feedback controller who monitors the session, (2) interaction with a fast responding and simple visualization, and (3) customization of classification. The thesis provides guidelines on how to design useful biofeedback visualizations along with an investigation of the operational capability of physiological sensors and the effect of virtual reality. As a result of this research, a biofeedback framework with a visual and interactive graph-based classification system was created that enables feedback controllers to easily change the classification process and customize it for their users.Aufgrund neuester technologischer Fortschritte und der steigenden Verfügbarkeit von physiologischen Sensoren für das alltägliche Leben, wird das Thema Biofeedback wieder aktuell und mit großem Interesse verfolgt. Ziel dieser Diplomarbeit ist die nahtlose Verbindung aus Biofeedback-Techniken, modernen physiologischen Sensoren und 3D Technologien. Ein flexibles und wiederverwendbares System wird erstellt, das als erweiterbare Plattform für zukünftige Sensoren und Forschungsprojekte verwendet werden kann. Das Resultat ist eine funktionsfähige Biofeedback-Software, welche die eigene Körperwahrnehmung und Körperkontrolle verbessern kann. Ferner erklärt diese Arbeit was Biofeedback ist, untersucht Modalitäten und Aufnahmetechniken von physiologischen Sensoren und stellt eine umfassende Recherche und Analyse von verwandten Arbeiten und Projekten bereit. Die zeitgleiche Datenerfassung mehrerer physiologischer Sensoren erfordert eine effiziente Speichernutzung um der Daten-Visualisierung genügend Rechenleistung zur Verfügung stellen zu können. Im Gegensatz zu traditionellen Biofeedback-Systemen, welche leiglich einen Wert von einem Sensor abbilden, erklärt diese Arbeit, wie ein interaktiver Klassifizierungs-Graph verwendet werden kann, um anpassbare Klassifikatoren zu erstellen und die Ergebnisse von der Signalverarbeitung auf einen oder mehrere Feedback-Werte abzubilden. Die Arbeit erweitert die traditionelle Biofeedback-Schleife um einen Kontroll- und Veränderungsmechanismus und trennt die Analyse und Klassifizierung von der Visualisierung. Die Zwei-Schichten Architektur ermöglicht es state-of-the-art Visualisierungen mit beliebigen Render-Engines zu erstellen. Mehrere Beispiel-Visualisierungen werden entwickelt, inklusive einer Virtual Reality Szene, welche ein Oculus Rift verwendet, um die Auswirkungen von Virtual Reality auf Biofeedback zu untersuchen. Die Evaluation, bei der 8 Probanden jeweils 7 Testszenarien durchliefen, zeigt, dass mehrere Faktoren für erfolgreiches Biofeedback entscheidend sind. Dazu gehören (1) die Interaktion mit einem menschlichen Feedback-Controller, (2) die Interaktion mit einer schnell reagierenden und simplen Visualisierung sowie (3) die Anpassung der Klassifikatoren. Die Diplomarbeit liefert einen Leitfaden für die Gestaltung von Biofeedback-Visualisierungen, über den Effekt von Virtual Reality und eine Untersuchung der Funktionsfähigkeit von physiologischen Sensoren. Das Ergebnis dieser Arbeit ist ein Biofeedback-System mit einer visuellen und interaktiven Graph-basierten Klassifikation, welches es Feedback-Controller erlaubt den Klassifizierungsprozess an den jeweiligen Benutzer anzupassen

Designing interactive virtual environments with feedback in health applications.

One of the most important factors to influence user experience in human-computer interaction is the user emotional reaction. Interactive environments including serious games that are responsive to user emotions improve their effectiveness and user satisfactions. Testing and training for user emotional competence is meaningful in healthcare field, which has motivated us to analyze immersive affective games using emotional feedbacks. In this dissertation, a systematic model of designing interactive environment is presented, which consists of three essential modules: affect modeling, affect recognition, and affect control. In order to collect data for analysis and construct these modules, a series of experiments were conducted using virtual reality (VR) to evoke user emotional reactions and monitoring the reactions by physiological data. The analysis results lead to the novel approach of a framework to design affective gaming in virtual reality, including the descriptions on the aspects of interaction mechanism, graph-based structure, and user modeling. Oculus Rift was used in the experiments to provide immersive virtual reality with affective scenarios, and a sample application was implemented as cross-platform VR physical training serious game for elderly people to demonstrate the essential parts of the framework. The measurements of playability and effectiveness are discussed. The introduced framework should be used as a guiding principle for designing affective VR serious games. Possible healthcare applications include emotion competence training, educational softwares, as well as therapy methods

Heart Rate Variability Biofeedback Training as an Intervention for Chronic Pain

Millions of Americans suffer from chronic pain. Treatment costs are in the billions and some patients still do not find relief. Current effectiveness research shows positive results for biofeedback training as an intervention for headache and other types of chronic pain. The present retrospective, archival study used patient information (N=72) collected during a heart rate variability biofeedback training program to assess treatment effectiveness among patients who experience chronic pain. More specifically, the study was designed to examine six research questions focused on patient-reported levels of pain and distress, as well as catastrophizing, depression, anxiety, and somatization. It was hypothesized that after three sessions of biofeedback, the patient scores on these six variables would decrease. A significant reduction in self-reported pain and distress was found immediately after the biofeedback session, however, pain and distress scores generally returned to the pre-session baseline by the beginning of the next biofeedback session and the reductions in pain and distress were not maintained between sessions revealing a “sawtooth” pattern. On average, patients reported a decrease of more than one point on a numeric (0 to 10) rating scale when rating their pain after the biofeedback training intervention (1.21 for Session 1; 1.63 Session 2; and 1.50 for Session 3). There was a slightly greater reduction in distress ratings than pain ratings after each session of biofeedback (i.e., distress ratings decreased an average of 1.75 after Session 1, 1.67 after Session 2, and 1.74 after Session 3). Of the four symptom measures (catastrophizing, depression, anxiety, and somatization), a statistically significant reduction was found only in the case of catastrophizing scores. When comparing Session 1 and Session 3 catastrophizing, the scores decreased 3.14 points on average (SD = 7.63), t(69) = 3.45, p = .001. Catastrophizing influences one’s beliefs about his or her ability to cope with pain and how much it interferes with his or her life. This finding strengthens the existing research literature that highlights the importance of targeting physical and psychological symptoms when developing a comprehensive pain management plan

Every sign of life

Thesis (Ph. D.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2003.MIT Institute Archives copy: pages 151-[182] bound in reverse order.Includes bibliographical references (p. 142-150).Every Sign of Life introduces an approach to and motivational schema for personal health monitoring. It is an exploration of how to make information collected by personal health-monitoring devices fun and engaging, and consequently more useful to the non-specialist. In contrast to the common methodology of adding game elements to established biofeedback systems, the Every Sign of Life approach is to design and build games that use biosensor information to effect the game environment. This work tests the hypothesis that fun (the joy of learning, achieving, competing, etc.) is a way to achieve the goal of self-efficacy; to induce people to take care of their own health by altering their habits and lifestyles. One result is a basic architecture for personal health-monitoring systems that has led to an approach to the design of sensor peripherals and wearable computer components called "Extremity Computing." This approach is used to redefine biosensor monitoring from periodic to continuous (ultimately saving data over a lifetime). Another result is an approach to adding implicit biofeedback to computer games. This has led to a new genre of games called "Bio-Analytical Games" that straddles the boundary between sports and computer games. A series of studies of how to present health information to children and adults have demonstrated the ability of consumers to use bioinformatics without involving professionals.by Vadim Gerasimov.Ph.D

Investigating resilience patterns based on within-subject changes in sleep and resting heart rate variability

Occupational stress can cause all kinds of health problems. Resilience interventions that help employees deal with and adapt to adverse events can prevent these negative consequences. Due to advances in sensor technology and smartphone applications, relatively unobtrusive self-monitoring of resilience-related outcomes is possible. With models that can recognize intra-individual changes in these outcomes and relate them to causal factors within the employee’s own context, an automated resilience intervention that gives personalized, just-in-time feedback can be developed. The Wearables and app-based resilience Modelling in employees (WearMe) project aims to develop such models. A cyclical conceptual framework based on existing theories of stress and resilience is presented, as the basis for the WearMe project. The included concepts are operationalized and measured using sleep tracking (Fitbit Charge 2), heart rate variability measurements (Elite HRV + Polar H7) and Ecological Momentary Assessment (mobile app), administered in the morning (7 questions) and evening (12 questions). The first (ongoing) study within the WearMe project investigates the feasibility of the developed measurement cycle and explores the development of such models in social studies students that are on their first major internship. Analyses will target the development of both within-subject (n=1) models, as well as between-subjects models. The first results will be shared at the Health By Tech 2019 conference in Groningen. If successful, future work will focus on further developing these models and eventually exploring the effectiveness of the envisioned personalized resilience system

The Acceptability and Efficacy of Wearable Devices in Digital Mental Health

Rapid advancements in technology through the early twenty-first century have led to the emergence of a new paradigm in mental health, in which digital platforms could become a fundamental part of mental healthcare delivery. Wearable devices, which are computational devices worn on the body, might form an important element of these new approaches by capturing and interpreting physiological data associated with psychological states. This thesis presents a series of studies investigating the range of wearable devices for the treatment of mental health problems, the perceived acceptability of these devices, and the evidence for one specific device modality, aided meditation. In Study 1, a literature review was conducted to identify wearable devices that could be used in the treatment of anxiety-related symptoms, determine what supporting evidence existed for each device modality, and explore potential clinical implications of using those devices. The review identified early-stage evidence for the use of heart rate variability biofeedback devices, but limited research on other modalities, indicating a need for further high-quality research. Study 2 surveyed a community sample of 427 adults to investigate perceived acceptability of wearable devices for treating mental health problems. Interest in using wearable devices as adjuncts to conventional therapy was strong, with acceptability closely linked to perceived device effectiveness (β = 0.28-0.30). Wearable devices also appeared to have greater acceptability in the presence of negative attitudes toward conventional therapies, suggesting they might help reduce barriers to treatment. Studies 3 and 4 focused on evaluating one particular device: the Muse electroencephalogram (EEG) meditation headband. In these studies, 68 adult participants used the device during a series of lab-based meditation tasks, with a subset (n = 29) also completing 14 days of home practice. Study 3 investigated the potential of the headband measures to assess state mindfulness, a process variable linked to psychological benefits resulting from meditation practice. The primary headband measure showed sensitivity to both within-participants (d = 0.56) and between-participants (r = -0.50) differences on a task measure of state mindfulness. Aggregate measures over 14 days’ practice explained around 30% of variance in self-reported trait mindfulness and related constructs. EEG biomarkers thus appear to have potential as a novel objective method of mindfulness measurement. Study 4 used a crossover trial design (auditory feedback of the primary headband measure vs no feedback) to examine the effect of receiving feedback. The feedback condition resulted in a higher level of state mindfulness (RR = 1.15), a lower level of the primary headband measure (d = -0.22), and differences in subjective experience of meditation. These results suggest that with appropriate guidance, feedback may be an effective adjunct to meditation. Together, these studies support the notion that wearable devices could be effective and engaging adjunctive digital mental health interventions. The results support the use of synchronous feedback of practice quality data to enhance the therapeutic benefits of meditation practice, and were consistent with the mechanisms through which neurofeedback is theorised to function. Continuing engagement with wearable devices by both researchers and clinicians is recommended.Thesis (Combined PhD & MPsych (Clin)) -- University of Adelaide, School of Psychology, 202

A Design Exploration of Affective Gaming

Physiological sensing has been a prominent fixture in games user research (GUR) since the late 1990s, when researchers began to explore its potential to enhance and understand experience within digital game play. Since these early days, it has been widely argued that “affective gaming”—in which gameplay is influenced by a player’s emotional state—can enhance player experience by integrating physiological sensors into play. In this thesis, I conduct a design exploration of the field of affective gaming by first, systematically exploring the field and creating a framework (the affective game loop) to classify existing literature; and second by presenting two design probes, to probe and explore the design space of affective games contextualized within the affective game loop: In the Same Boat and Commons Sense. The systematic review explored this unique design space of affective gaming, opening up future avenues for exploration. The affective game loop was created as a way to classify the physiological signals and sensors most commonly used in prior literature within the context of how they are mapped into the gameplay itself. Findings suggest that the physiological input mappings can be more action-based (e.g., affecting mechanics in the game such as the movement of the character) or more context-based (e.g., affecting things like environmental or difficulty variables in the game). Findings also suggested that while the field has been around for decades, there is still yet to be any commercial successes, so does physiological interaction really heighten player experience? This question instigated the design of the two probes, exploring ways to implement these mappings and effectively heighten player experience. In the Same Boat (Design Probe One) is an embodied mirroring game designed to promote an intimate interaction, using players’ breathing rate and facial expressions to control movement of a canoe down a river. Findings suggest that playing In the Same Boat fostered the development of affiliation between the players, and that while embodied controls were less intuitive, people enjoyed them more, indicating the potential of embodied controls to foster social closeness in synchronized play over a distance. Commons Sense (Design Probe Two) is a communication modality intended to heighten audience engagement and effectively capture and communicate the audience experience, using a webcam-based heart rate detection software that takes an average of each spectator’s heart rate as input to affect in-game variables such as lighting and sound design, and game difficulty. Findings suggest that Commons Sense successfully facilitated the communication of audience response in an online entertainment context—where these social cues and signals are inherently diminished. In addition, Commons Sense is a communication modality that can both enhance a play experience while offering a novel way to communicate. Overall, findings from this design exploration shows that affective games offer a novel way to deliver a rich gameplay experience for the player