Detecting System Errors in Virtual Reality Using EEG Through Error-Related Potentials

When persons interact with the environment and experience or wit-ness an error (e.g. an unexpected event), a specific brain pattern,known as error-related potential (ErrP) can be observed in the elec-troencephalographic signals (EEG). Virtual Reality (VR) technologyenables users to interact with computer-generated simulated envi-ronments and to provide multi-modal sensory feedback. Using VRsystems can, however, be error-prone. In this paper, we investigatethe presence of ErrPs when Virtual Reality users face 3 types ofvisualization errors: (Te) tracking errors when manipulating virtualobjects, (Fe) feedback errors, and (Be) background anomalies. Weconducted an experiment in which 15 participants were exposed tothe 3 types of errors while performing a center-out pick and placetask in virtual reality. The results showed that tracking errors gener-ate error-related potentials, the other types of errors did not generatesuch discernible patterns. In addition, we show that it is possible todetect the ErrPs generated by tracking losses in single trial, with anaccuracy of 85%. This constitutes a first step towards the automaticdetection of error-related potentials in VR applications, paving theway to the design of adaptive and self-corrective VR/AR applicationsby exploiting information directly from the user’s brain

Interactive Haptics for Remote and On-Site Assessment of Arm Function Following a Stroke

There is a great need to improve the rehabilitation and assessment of arm and hand function of stroke survivors in the home due to cost, time and availability of healthcare professionals. Robotics and haptic technologies can be used to improve and facilitate rehabilitation and assessment in the home. The primary goal of this thesis was to explore the feasibility of using lightweight, low-cost haptic devices for remote home-based rehabilitation. The strategy that this thesis followed was to develop tools, perform unit testing, and finally assess feasibility with target users in a series of case studies. The thesis started by developing an assessment tool, specifically the Nine Hole Peg Test (NHPT), and investigated how haptic devices can be used to enhance the data collection for this task to garner more information regarding the level of manual dexterity a stroke survivor has in their impaired limb. The next study investigated collaboration in haptic environments and how the findings from a collaborative haptic experiment could be used to influence task design for future experiments with haptic environments. The final study assessed the feasibility of a home-based assessment and rehabilitation system with elements of telerehabilitation and remote collaboration and interaction providing four complete case studies from stroke survivors. In summary, our findings showed that by combining physical apparatus with a virtual world, less variable results are observed than in purely virtual haptic tasks. We also showed that interaction techniques in collaborative haptic environments change depending on the shape of the objects in the virtual task – this information can be used to influence task design to target specific motor deficits when using the device for exercise. Finally, the home-based study showed the feasibility of using the experimental rig at home and provided improvement measures that matched the perceived benefits to arm function that the participants described on completing the trial

Grasping the Void: Immersion Tactics Using Gesture Controlled Physics Interaction Systems in Virtual Reality

This thesis uses the HTC Vive in Unity to compare two different types of object interaction systems in order to determine the effectiveness of physics based interaction systems in a virtual environment. The research problem that motivates this project is the fact that there is no standardized method for defining successful object interaction techniques in VR. There are numerous interaction techniques in VR that fall short of simulating realistic object interaction. This project explores a physics based interaction system and examines how effective it is by comparing it to a non-physics based system. A model house with various interactable objects is created to compare the two interaction systems. The first system, the naive interaction system, parents an object to the controller model, allowing the user to pick up and throw things in a very simple fashion. This system is compared to a physics based Newtonian system that takes into account mass and velocity during object interactions. The Newtonian system promotes a much deeper sense of immersion for a user due to how accurately the system simulates real life physical interactions. It is clear that creating a high level of mental and physical presence is crucial for a VR experience. Object interaction systems are an integral component of a VR experience that directly contribute to the realism and levels of virtual presence that a user achieves within a virtual environment. The results of this project conclude that physics based interaction systems provide levels of realism and immersion that the naive systems currently cannot achieve The results of this project are beneficial because they demonstrate the positive impact physics based interaction systems have on a VR experience and the need for improved physics systems for the future of VR development

Microscope Embedded Neurosurgical Training and Intraoperative System

In the recent years, neurosurgery has been strongly influenced by new technologies. Computer Aided Surgery (CAS) offers several benefits for patients\u27 safety but fine techniques targeted to obtain minimally invasive and traumatic treatments are required, since intra-operative false movements can be devastating, resulting in patients deaths. The precision of the surgical gesture is related both to accuracy of the available technological instruments and surgeon\u27s experience. In this frame, medical training is particularly important. From a technological point of view, the use of Virtual Reality (VR) for surgeon training and Augmented Reality (AR) for intra-operative treatments offer the best results. In addition, traditional techniques for training in surgery include the use of animals, phantoms and cadavers. The main limitation of these approaches is that live tissue has different properties from dead tissue and that animal anatomy is significantly different from the human. From the medical point of view, Low-Grade Gliomas (LGGs) are intrinsic brain tumours that typically occur in younger adults. The objective of related treatment is to remove as much of the tumour as possible while minimizing damage to the healthy brain. Pathological tissue may closely resemble normal brain parenchyma when looked at through the neurosurgical microscope. The tactile appreciation of the different consistency of the tumour compared to normal brain requires considerable experience on the part of the neurosurgeon and it is a vital point. The first part of this PhD thesis presents a system for realistic simulation (visual and haptic) of the spatula palpation of the LGG. This is the first prototype of a training system using VR, haptics and a real microscope for neurosurgery. This architecture can be also adapted for intra-operative purposes. In this instance, a surgeon needs the basic setup for the Image Guided Therapy (IGT) interventions: microscope, monitors and navigated surgical instruments. The same virtual environment can be AR rendered onto the microscope optics. The objective is to enhance the surgeon\u27s ability for a better intra-operative orientation by giving him a three-dimensional view and other information necessary for a safe navigation inside the patient. The last considerations have served as motivation for the second part of this work which has been devoted to improving a prototype of an AR stereoscopic microscope for neurosurgical interventions, developed in our institute in a previous work. A completely new software has been developed in order to reuse the microscope hardware, enhancing both rendering performances and usability. Since both AR and VR share the same platform, the system can be referred to as Mixed Reality System for neurosurgery. All the components are open source or at least based on a GPL license

Augmented Reality

Augmented Reality (AR) is a natural development from virtual reality (VR), which was developed several decades earlier. AR complements VR in many ways. Due to the advantages of the user being able to see both the real and virtual objects simultaneously, AR is far more intuitive, but it's not completely detached from human factors and other restrictions. AR doesn't consume as much time and effort in the applications because it's not required to construct the entire virtual scene and the environment. In this book, several new and emerging application areas of AR are presented and divided into three sections. The first section contains applications in outdoor and mobile AR, such as construction, restoration, security and surveillance. The second section deals with AR in medical, biological, and human bodies. The third and final section contains a number of new and useful applications in daily living and learning

Attention Restraint, Working Memory Capacity, and Mind Wandering: Do Emotional Valence or Intentionality Matter?

Attention restraint appears to mediate the relationship between working memory capacity (WMC) and mind wandering (Kane et al., 2016). Prior work has identifed two dimensions of mind wandering—emotional valence and intentionality. However, less is known about how WMC and attention restraint correlate with these dimensions. Te current study examined the relationship between WMC, attention restraint, and mind wandering by emotional valence and intentionality. A confrmatory factor analysis demonstrated that WMC and attention restraint were strongly correlated, but only attention restraint was related to overall mind wandering, consistent with prior fndings. However, when examining the emotional valence of mind wandering, attention restraint and WMC were related to negatively and positively valenced, but not neutral, mind wandering. Attention restraint was also related to intentional but not unintentional mind wandering. Tese results suggest that WMC and attention restraint predict some, but not all, types of mind wandering

Mathematical models for glioma growh and migration inside the brain

284 p.Los gliomas forman el subtipo más prevalente, agresivo e invasivo de tumores cerebrales primarios,caracterizados por una rápida proliferación celular y una elevada capacidad de infiltración. A pesar de los avances de la investigación clínica, estos tumores suelen ser resistentes al tratamiento; la supervivencia media oscila entre 9 y 12 meses, siendo la recurrencia la principal causa de mortalidad.La migración y la invasión de los gliomas en el cerebro son fenómenos complejos y aún se desconocen varios de los mecanismos subyacentes que guían la progresión de estos tumores.En esta tesis, proponemos varios modelos matemáticos para estudiar diversos aspectos de la progresión del glioma en relación con las escalas microscópicas y macroscópicas que caracterizan este proceso. Considerar el carácter intrínsico multiescala de la evolución del glioma permite definir modelos basados en sistemas dinámicos, ecuaciones cinéticas y EDP macroscópicas con diferentes roles dependiendo de los fenómenos a estudiar. Uno de los objetivos principales de esta tesis es integrar datos biológicos y clínicos con los modelos matemáticos. Los datos experimentales utilizados se han obtenido de imágenes por resonancia magnética, de imágenes con tensor de difusión del cerebro humano y de análisis de inmunofluorescencia in vivo de distribuciones de varias proteínas en Drosophila, un modelo fiable para el estudio de la dinámica del glioblastoma.Analizamos las características de anisotropía del tejido nervioso, utilizando los datos del tensor de difusión, y la influencia de la estructura de las fibras en la dinámica de las células tumorales.Mostramos cómo la red de fibras guía la migración celular a lo largo de rutas preferenciales,reproduciendo los patrones ramificados y heterogéneos típicos de la evolución del glioma; asimismo,demostramos cómo los tratamientos multimodales pueden reducir este comportamiento.Estudiamos la interdependencia entre la acidez del microambiente y la vascularización en el proceso de angiogénesis tumoral. Para ello, construimos un modelo capaz de reproducir la influencia de estos mecanismos en el desarrollo de la heterogeneidad intratumoral y de características típicas de la progresión del glioma relacionadas con la hipoxia (e.g. la necrosis). Este estudio permite formular una clasificación de los tumores basada en el nivel de necrosis, así como la investigación de terapias multimodales que incluyan efectos antiangiogénicos.Investigamos la influencia de las protrusiones celulares desde una perspectiva no local.Analizamos su rol en el fenómeno de la guía por contacto y en la manifestación de efectos colaborativos o competitivos entre dos estímulos que determinan cambios de dirección de la velocidad celular.Utilizando el análisis experimental de las distribuciones de varias proteínas, evaluamos la relación de las protrusiones celulares con las integrinas y las proteasas como principales mecanismos de progresión del glioblastoma. Mostramos cómo las interacciones bioquímicas y biomecánicas de estos agentes dan como resultado el desarrollo de frentes de propagación tumoral, que pueden presentar una evolución dinámica y heterogénea en relación a los cambios ambientales.bcam:basque center for applied mathematics; La Caixa Foundatio

Mathematical models for glioma growh and migration inside the brain

284 p.Los gliomas forman el subtipo más prevalente, agresivo e invasivo de tumores cerebrales primarios,caracterizados por una rápida proliferación celular y una elevada capacidad de infiltración. A pesar de los avances de la investigación clínica, estos tumores suelen ser resistentes al tratamiento; la supervivencia media oscila entre 9 y 12 meses, siendo la recurrencia la principal causa de mortalidad.La migración y la invasión de los gliomas en el cerebro son fenómenos complejos y aún se desconocen varios de los mecanismos subyacentes que guían la progresión de estos tumores.En esta tesis, proponemos varios modelos matemáticos para estudiar diversos aspectos de la progresión del glioma en relación con las escalas microscópicas y macroscópicas que caracterizan este proceso. Considerar el carácter intrínsico multiescala de la evolución del glioma permite definir modelos basados en sistemas dinámicos, ecuaciones cinéticas y EDP macroscópicas con diferentes roles dependiendo de los fenómenos a estudiar. Uno de los objetivos principales de esta tesis es integrar datos biológicos y clínicos con los modelos matemáticos. Los datos experimentales utilizados se han obtenido de imágenes por resonancia magnética, de imágenes con tensor de difusión del cerebro humano y de análisis de inmunofluorescencia in vivo de distribuciones de varias proteínas en Drosophila, un modelo fiable para el estudio de la dinámica del glioblastoma.Analizamos las características de anisotropía del tejido nervioso, utilizando los datos del tensor de difusión, y la influencia de la estructura de las fibras en la dinámica de las células tumorales.Mostramos cómo la red de fibras guía la migración celular a lo largo de rutas preferenciales,reproduciendo los patrones ramificados y heterogéneos típicos de la evolución del glioma; asimismo,demostramos cómo los tratamientos multimodales pueden reducir este comportamiento.Estudiamos la interdependencia entre la acidez del microambiente y la vascularización en el proceso de angiogénesis tumoral. Para ello, construimos un modelo capaz de reproducir la influencia de estos mecanismos en el desarrollo de la heterogeneidad intratumoral y de características típicas de la progresión del glioma relacionadas con la hipoxia (e.g. la necrosis). Este estudio permite formular una clasificación de los tumores basada en el nivel de necrosis, así como la investigación de terapias multimodales que incluyan efectos antiangiogénicos.Investigamos la influencia de las protrusiones celulares desde una perspectiva no local.Analizamos su rol en el fenómeno de la guía por contacto y en la manifestación de efectos colaborativos o competitivos entre dos estímulos que determinan cambios de dirección de la velocidad celular.Utilizando el análisis experimental de las distribuciones de varias proteínas, evaluamos la relación de las protrusiones celulares con las integrinas y las proteasas como principales mecanismos de progresión del glioblastoma. Mostramos cómo las interacciones bioquímicas y biomecánicas de estos agentes dan como resultado el desarrollo de frentes de propagación tumoral, que pueden presentar una evolución dinámica y heterogénea en relación a los cambios ambientales.bcam:basque center for applied mathematics; La Caixa Foundatio

Mathematical models for glioma growth and migration inside the brain

Gliomas are the most prevalent, aggressive, and invasive subtype of primary brain tumors, characterized by rapid cell proliferation and great infiltration capacity. De- spite the advances of clinical research, these tumors are often resistant to treatment, the median survival ranges between 9 and 12 months, and recurrence is the main cause of mortality. Glioma migration and invasion into the brain tissue is a complex phenomenon and little is still known about the underlying mechanisms that lead to tumor progression. In this thesis, we propose several mathematical models studying various aspects of glioma progression in relation to the microscopic and macroscopic scales charac- terizing this process. Exploiting the inherently multiscale nature of glioma evolution allows to define models based on dynamical systems, kinetic equations, and macro- scopic PDEs with different roles depending on the considered phenomena. The in- tegration of biological and clinical data with the mathematical models is one of the key objectives of this thesis. The experimental data at hand are obtained from mag- netic resonance and diffusion tensor images of the human brain and from in-vivo im- munofluorescence analysis of protein distributions in Drosophila, a reliable model for the study of glioblastoma dynamics. We analyze the anisotropic characteristics of the brain tissue, using the diffusion tensor data, and the influence of the fiber structures on tumor cell dynamics. We show how the fiber network directs cell migration along preferential paths, reproducing the branched and heterogeneous patterns typical of glioma evolution, and how multi- modal treatments can reduce this behavior. We study the interdependency of microenvironmental acidity and vasculature in tumor angiogenesis, defining a model capable of reproducing their influence on the emergence of phenotypic heterogeneity and hypoxia-related features (like necrosis) typical of glioma progression. This study enables the testing of a necrosis-based tumor grading and the investigation of multi-modal therapies with anti-angiogenic effects. We investigated the role of cell protrusions from a non-local perspective. We ex- plore their influence on the contact guidance phenomenon and on the emergence of collaborative or competitive effects between two cues driving cell velocity changes. Using the experimental analysis of protein distributions, we evaluate cell protru- sion relationship with integrins and proteases as leading mechanisms of glioblastoma progression. We show how the biochemical and biomechanical interactions of these agents result in the emergence of tumor propagation fronts, which can feature a dy- namical and heterogenous evolution in relation to environmental changes.European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 713673. ”la Caixa” Foundation (ID 100010434), with fellowship code LCF/BQ/IN17/11620056