Decoding Neural Correlates of Cognitive States to Enhance Driving Experience

Modern cars can support their drivers by assessing and autonomously performing different driving maneuvers based on information gathered by in-car sensors. We propose that brain–machine interfaces (BMIs) can provide complementary information that can ease the interaction with intelligent cars in order to enhance the driving experience. In our approach, the human remains in control, while a BMI is used to monitor the driver's cognitive state and use that information to modulate the assistance provided by the intelligent car. In this paper, we gather our proof-of-concept studies demonstrating the feasibility of decoding electroencephalography correlates of upcoming actions and those reflecting whether the decisions of driving assistant systems are in-line with the drivers' intentions. Experimental results while driving both simulated and real cars consistently showed neural signatures of anticipation, movement preparation, and error processing. Remarkably, despite the increased noise inherent to real scenarios, these signals can be decoded on a single-trial basis, reflecting some of the cognitive processes that take place while driving. However, moderate decoding performance compared to the controlled experimental BMI paradigms indicate there exists room for improvement of the machine learning methods typically used in the state-of-the-art BMIs. We foresee that neural fusion correlates with information extracted from other physiological measures, e.g., eye movements or electromyography as well as contextual information gathered by in-car sensors will allow intelligent cars to provide timely and tailored assistance only if it is required; thus, keeping the user in the loop and allowing him to fully enjoy the driving experience

Virtual Reality for Enhanced Ecological Validity and Experimental Control in the Clinical, Affective and Social Neurosciences

This article highlights the potential of virtual reality environments for enhanced ecological validity in the clinical, affective, and social neurosciences

Recipes for calibration and validation of agent-based models in cancer biomedicine

Computational models and simulations are not just appealing because of their intrinsic characteristics across spatiotemporal scales, scalability, and predictive power, but also because the set of problems in cancer biomedicine that can be addressed computationally exceeds the set of those amenable to analytical solutions. Agent-based models and simulations are especially interesting candidates among computational modelling strategies in cancer research due to their capabilities to replicate realistic local and global interaction dynamics at a convenient and relevant scale. Yet, the absence of methods to validate the consistency of the results across scales can hinder adoption by turning fine-tuned models into black boxes. This review compiles relevant literature to explore strategies to leverage high-fidelity simulations of multi-scale, or multi-level, cancer models with a focus on validation approached as simulation calibration. We argue that simulation calibration goes beyond parameter optimization by embedding informative priors to generate plausible parameter configurations across multiple dimensions

How the Brain Makes Up the Mind: a heuristic approach to the hard problem of consciousness

A solution to the “hard problem” requires taking the point of view of the organism and its sub- agents. The organism constructs phenomenality through acts of fiat, much as we create meaning in language, through the use of symbols that are assigned meaning in the context of an embodied evolutionary history. Phenomenality is a virtual representation, made to itself by an executive agent (the conscious self), which is tasked with monitoring the state of the organism and its environment, planning future action, and coordinating various sub-agencies. Consciousness is not epiphenomenal and serves a function for higher organisms that is distinct from unconscious processing. While a strictly scientific solution to the hard problem is not possible for a science that excludes the subjectivity it seeks to explain, there is hope to at least informally bridge the explanatory gulf between mind and matter

An Exploration of the Feasibility of Functional Near-Infrared Spectroscopy as a Neurofeedback Cueing System for the Mitigation of the Vigilance Decrement

Vigilance is the capacity for observers to maintain attention over extended periods of time, and has most often been operationalized as the ability to detect rare and critical signals (Davies & Parasuraman, 1982; Parasuraman, 1979; Warm, 1984). Humans, however, have natural physical and cognitive limitations that preclude successful long-term vigilance performance and consequently, without some means of assistance, failures in operator vigilance are likely to occur. Such a decline in monitoring performance over time has been a robust finding in vigilance experiments for decades and has been called the vigilance decrement function (Davies & Parasuraman, 1982; Mackworth, 1948). One of the most effective countermeasures employed to maintain effective performance has been cueing: providing the operator with a reliable prompt concerning signal onset probability. Most protocols have based such cues on task-related or environmental factors. The present dissertation examines the efficacy of cueing when nominally based on operator state (i.e., blood oxygenation of cortical tissue) in a novel vigilance task incorporating dynamic displays over three studies. Results pertaining to performance outcomes, physiological measures (cortical blood oxygenation and heart rate variability), and perceived workload and stress are interpreted via Signal Detection Theory and the Resource Theory of vigilance

Machine learning methods for the study of cybersickness: a systematic review

This systematic review offers a world-first critical analysis of machine learning methods and systems, along with future directions for the study of cybersickness induced by virtual reality (VR). VR is becoming increasingly popular and is an important part of current advances in human training, therapies, entertainment, and access to the metaverse. Usage of this technology is limited by cybersickness, a common debilitating condition experienced upon VR immersion. Cybersickness is accompanied by a mix of symptoms including nausea, dizziness, fatigue and oculomotor disturbances. Machine learning can be used to identify cybersickness and is a step towards overcoming these physiological limitations. Practical implementation of this is possible with optimised data collection from wearable devices and appropriate algorithms that incorporate advanced machine learning approaches. The present systematic review focuses on 26 selected studies. These concern machine learning of biometric and neuro-physiological signals obtained from wearable devices for the automatic identification of cybersickness. The methods, data processing and machine learning architecture, as well as suggestions for future exploration on detection and prediction of cybersickness are explored. A wide range of immersion environments, participant activity, features and machine learning architectures were identified. Although models for cybersickness detection have been developed, literature still lacks a model for the prediction of first-instance events. Future research is pointed towards goal-oriented data selection and labelling, as well as the use of brain-inspired spiking neural network models to achieve better accuracy and understanding of complex spatio-temporal brain processes related to cybersickness

The distracted robot: what happens when artificial agents behave like us

In everyday life, we are frequently exposed to different smart technologies. From our smartphones to avatars in computer games, and soon perhaps humanoid robots, we are surrounded by artificial agents created to interact with us. Already during the design phase of an artificial agent, engineers often endow it with functions aimed to promote the interaction and engagement with it, ranging from its \u201ccommunicative\u201d abilities to the movements it produces. Still, whether an artificial agent that can behave like a human could boost the spontaneity and naturalness of interaction is still an open question. Even during the interaction with conspecifics, humans rely partially on motion cues when they need to infer the mental states underpinning behavior. Similar processes may be activated during the interaction with embodied artificial agents, such as humanoid robots. At the same time, a humanoid robot that can faithfully reproduce human-like behavior may undermine the interaction, causing a shift in attribution: from being endearing to being uncanny. Furthermore, it is still not clear whether individual biases and prior knowledge related to artificial agents can override perceptual evidence of human-like traits. A relatively new area of research emerged in the context of investigating individuals\u2019 reactions towards robots, widely referred to as Human-Robot Interaction (HRI). HRI is a multidisciplinary community that comprises psychologists, neuroscientists, philosophers as well as roboticists, and engineers. However, HRI research has been often based on explicit measures (i.e. self-report questionnaires, a-posteriori interviews), while more implicit social cognitive processes that are elicited during the interaction with artificial agents took second place behind more qualitative and anecdotal results. The present work aims to demonstrate the usefulness of combining the systematic approach of cognitive neuroscience with HRI paradigms to further investigate social cognition processes evoked by artificial agents. Thus, this thesis aimed at exploring human sensitivity to anthropomorphic characteristics of a humanoid robot's (i.e. iCub robot) behavior, based on motion cues, under different conditions of prior knowledge. To meet this aim, we manipulated the human-likeness of the behaviors displayed by the robot and the explicitness of instructions provided to the participants, in both screen-based and real-time interaction scenarios. Furthermore, we explored some of the individual differences that affect general attitudes towards robots, and the attribution of human-likeness consequently

Global neural rhythm control by local neuromodulation

Neural oscillations are a ubiquitous form of neural activity seen across scales and modalities. These neural rhythms correlate with diverse cognitive functions and brain states. One mechanism for changing the oscillatory dynamics of large neuronal populations is through neuromodulator activity. An intriguing phenomenon explored here is when local neuromodulation of a distinct neuron type within a single brain nucleus exerts a powerful influence on global cortical rhythms. One approach to investigate the impact of local circuits on global rhythms is through optogenetic techniques. My first project involves the statistical analysis of electrophysiological recordings of an optogenetically-mediated Parkinsonian phenotype. Empirical studies demonstrate that Parkinsonian motor deficits correlate with the emergence of exaggerated beta frequency (15-30 Hz) oscillations throughout the cortico-basal ganglia-thalamic network. However, the mechanism of these aberrant oscillatory dynamics is not well understood. A previous modeling study predicted that cholinergic neuromodulation of medium spiny neurons in the striatum of the basal ganglia may mediate the pathologic beta rhythm. Here, this hypothesis was tested using selective optogenetic stimulation of striatal cholinergic interneurons in normal mice; stimulation robustly and reversibly amplified beta oscillations and Parkinsonian motor symptoms. The modulation of global rhythms by local networks was further studied using computational modeling in the context of intrathalamic neuromodulation. While intrathalamic vasoactive intestinal peptide (VIP) is known to cause long-lasting excitation in vitro, its in vivo dynamical effects have not been reported. Here, biophysical computational models were used to elucidate the impact of VIP on thalamocortical dynamics during sleep and propofol general anesthesia. The modeling results suggest that VIP can form robust sleep spindle oscillations and control aspects of sleep architecture through a novel homeostatic mechanism. This homeostatic mechanism would be inhibited by general anesthesia, representing a new mechanism contributing to anesthetic-induced loss of consciousness. While the previous two projects differed in their use of empirical versus theoretical methods, a challenge common to both domains is the difficulty in visualizing and analyzing large multi-dimensional datasets. A tool to mitigate these issues is introduced here: GIMBL-Vis is a Graphical Interactive Multi-dimensional extensiBLe Visualization toolbox for Matlab. This toolbox simplifies the process of exploring multi-dimensional data in Matlab by providing a graphical interface for visualization and analysis. Furthermore, it provides an extensible open platform for distributed development by the community

Connectivity Analysis of Brain States and Applications in Brain-Computer Interfaces

Human brain is organized by a large number of functionally correlated but spatially distributed cortical neurons. Cognitive processes are usually associated with dynamic interactions among multiple brain regions. Therefore, the understanding of brain functions requires the inves- tigation of the brain interaction patterns. This thesis contains two main aspects. The first aspect focuses on the neural basis for cognitive processes through the use of brain connectivity analysis. The second part targets on assessing brain connectivity patterns in realistic scenarios, e.g., in-car BCI and stroke patients. In the first part, we explored the neural correlates of error-related brain activity. We recorded scalp electroencephalogram (EEG) from 15 healthy subjects while monitoring the movement of a cursor on a computer screen, yielding particular brain connectivity patterns after monitoring external errors. This supports the presence of common role of medial frontal cortex in coordinating cross-regional activity during brain error processes, independent of their causes, either self-generated or external events. This part also included the investigation of the connectivity during left/right hand motor imagery, including 9 healthy subjects, which demonstrated particular intrahemispheric and interhemispheric information flows in two motor imagery tasks, i.e., the ÎŒ rhythm is highly modulated in intrahemispheric, whereas β and γ are modulated in interhemispheric interactions. This part also explored the neural correlates of reaction time during driving. An experiment with 15 healthy subjects in car simulator was designed, in which they needed to perform lane change to avoid collision with obstacles. Significant neural modulations were found in ERP (event-related potential), PSD (power spectral density), and frontoparietal network, which seems to reflect the underlying information transfer from sensory representation in the parietal cortex to behavioral adjusting in the frontal cortex. In the second part, we first explored the feasibility of using BCI as driving assistant system, in which visual stimuli were presented to evoke error/correct related potentials, and were classified to infer driverâs preferred turning direction. The system was validated in a car simulator with 22 subjects, and 7 joined online tests. The system was also tested in real car, yielding similar brain patterns and comparable classification accuracy. The second part also carried out the brain connectivity analysis in stroke patients.We performed exploratory study to correlate the recovery effects of BCI therapy, through the quantification of connectivity between healthy and lesioned hemispheres. The results indicate the benefits of BCI therapy for stroke patients, i.e., brain connectivity are more similar as healthy patterns, increased (decreased) flow from the damaged (undamaged) to the undamaged (damaged) cortex. Briefly, this thesis presents exploratory studies of brain connectivity analysis, investigating the neural basis of cognitive processes, and its contributions in the decoding phase. In particular, such analysis is not limited to laboratory researches, but also extended to clinical trials and driving scenarios, further supporting the findings observed in the ideal condition