A review of rapid serial visual presentation-based brain-computer interfaces

International audienceRapid serial visual presentation (RSVP) combined with the detection of event related brain responses facilitates the selection of relevant information contained in a stream of images presented rapidly to a human. Event related potentials (ERPs) measured non-invasively with electroencephalography (EEG) can be associated with infrequent targets amongst a stream of images. Human-machine symbiosis may be augmented by enabling human interaction with a computer, without overt movement, and/or enable optimization of image/information sorting processes involving humans. Features of the human visual system impact on the success of the RSVP paradigm, but pre-attentive processing supports the identification of target information post presentation of the information by assessing the co-occurrence or time-locked EEG potentials. This paper presents a comprehensive review and evaluation of the limited but significant literature on research in RSVP-based brain-computer interfaces (BCIs). Applications that use RSVP-based BCIs are categorized based on display mode and protocol design, whilst a range of factors influencing ERP evocation and detection are analyzed. Guidelines for using the RSVP-based BCI paradigms are recommended, with a view to further standardizing methods and enhancing the inter-relatability of experimental design to support future research and the use of RSVP-based BCIs in practice

EEG Microstates in Neurofeedback Attention Training

Attention has come under acute focus within the neuropsychological world in past decades, and the rise of brain-computer interfaces (BCI) during EEG offers a means to personalize attention training therapies. Semi-stable EEG topographies, called “microstates,” have been found to be functionally relevant to attention-oriented tasks and shown to influence awareness in the time period directly before a stimulus. In a BCI designed to train attention, we may expect to see a group difference in microstates. Specifically, it could be that microstate D—functionally relevant to attention and task-switching—increases while microstate C—functionally relevant to task-negative and saliency networks—decreases within the group that successfully learns via neurofeedback. The reversed pattern may be true in groups that either fails to learn through neurofeedback or received sham neurofeedback. We may also expect microstates D and C to relate to a behavioral outcome measure that indexes training performance. Accordingly, we used EEGLAB to process BCI attention-training data, derive microstate topographies for individual participants, cluster grand mean topographies for the entire study group, and extract temporal statistics to measure microstate temporal presence during pre-stimulus training. Overall, microstate D had greater temporal presence in those who successfully self-regulated neural cognition during the BCI task compared to those who could not achieve this; microstate C had greater temporal presence in those who could not self-regulate neural cognition during the BCI task compared to those who did so successfully. This analysis highlights differences in BCI performance but failed to find meaningful changes over training

Exploring the Design of mHealth Systems for Health Behavior Change using Mobile Biosensors

A person’s health behavior plays a vital role in mitigating their risk of disease and promoting positive health outcomes. In recent years, mHealth systems have emerged to offer novel approaches for encouraging and supporting users in changing their health behavior. Mobile biosensors represent a promising technology in this regard; that is, sensors that collect physiological data (e.g., heart rate, respiration, skin conductance) that individuals wear, carry, or access during their normal daily activities. mHealth system designers have started to use the health information from physiological data to deliver behavior-change interventions. However, little research provides guidance about how one can design mHealth systems to use mobile biosensors for health behavior change. In order to address this research gap, we conducted an exploratory study. Following a hybrid approach that combines deductive and inductive reasoning, we integrated a body of fragmented literature and conducted 30 semi-structured interviews with mHealth stakeholders. From this study, we developed a theoretical framework and six general design guidelines that shed light on the theoretical pathways for how the mHealth interface can facilitate behavior change and provide practical design considerations

Plug-in to fear: game biosensors and negative physiological responses to music

The games industry is beginning to embark on an ambitious journey into the world of biometric gaming in search of more exciting and immersive gaming experiences. Whether or not biometric game technologies hold the key to unlock the “ultimate gaming experience” hinges not only on technological advancements alone but also on the game industry’s understanding of physiological responses to stimuli of different kinds, and its ability to interpret physiological data in terms of indicative meaning. With reference to horror genre games and music in particular, this article reviews some of the scientific literature relating to specific physiological responses induced by “fearful” or “unpleasant” musical stimuli, and considers some of the challenges facing the games industry in its quest for the ultimate “plugged-in” experience

Inclusive Intelligent Learning Management System Framework

Machado, D. S-M., & Santos, V. (2023). Inclusive Intelligent Learning Management System Framework. International Journal of Automation and Smart Technology, 13(1), [2423]. https://doi.org/10.5875/ausmt.v13i1.2423The article finds context and the current state of the art in a systematic literature review on intelligent systems employing PRISMA Methodology which is complemented with narrative literature review on disabilities, digital accessibility and legal and standards context. The main conclusion from this review was the existing gap between the available knowledge, standards, and law and what is put into practice in higher education institutions in Portugal. Design Science Research Methodology was applied to output an Inclusive Intelligent Learning Management System Framework aiming to help higher education professors to share accessible pedagogic content and deliver on-line and presential classes with a high level of accessibility for students with different types of disabilities, assessing the uploaded content with Web content Accessibility Guidelines 3.0, clustering students according to their profile, conscient feedback and emotional assessment during content consumption, applying predictive models and signaling students at risk of failing classes according to study habits and finally applying a recommender system. The framework was validated by a focus group to which experts in digital accessibility, information systems and a disabled PhD graduate.publishersversionpublishe

Sensorimotor Integration Along the Neuraxis

Sensory feedback is a vital component of behavior. Oftentimes we use sensory feedback to learn new motor skills, such as riding a bike. Other times we passively consume sensory information for our own enjoyment or to support cognitive processes such as learning. We occasionally even move to acquire certain sensory experiences or achieve sensory states. When sensory feedback is impaired, movement is also impaired. However, deficiencies of sensory feedback do not necessarily prohibit movement---motor tasks can be accomplished with limited or noisy sensory feedback, even if slowly and with reduced accuracy. This suggests that there are interactions between sensory modalities as well as interactions between sensation and movement at every level of the nervous system and raises the question of how sensory feedback interacts with motor commands along the neuraxis, from the periphery to the cerebral cortex. Here we studied the neural mechanisms of interactions and integration between sensation and movement in the periphery, in spinal circuits, and finally in volitional control of a brain-computer interface using signals extracted from motor cortex. We found that contact force influences the neural encoding of tactile stimuli in primary afferents, that spinal circuits encode sterotyped motor output to sensory stimulation, and that sensory feedback interacts with movement intent in motor cortex. Together, these studies suggest that sensation and movement are intertwined at every level of the nervous system, and that sensory signals must be demultiplexed and contextualized in terms of the movements used to acquire them, and motor commands must take into account the evolving sensory state

A review and framework for designing interactive technologies for emotion regulation training

Emotion regulation is foundational to mental health and well-being. In the last ten years there has been an increasing focus on the use of interactive technologies to support emotion regulation training in a variety of contexts. However, research has been done in diverse fields, and no cohesive framework exists that explicates what features of such system are important to consider, guidance on how to design these features, and what remains unknown, which should be explored in future design research. To address this gap, this thesis presents the results of a descriptive review of 54 peer-reviewed papers. Through qualitative and frequency analysis I analyzed previous technologies, reviewed their theoretical foundations, the opportunities where they appear to provide unique benefits, and their conceptual and usability challenges. Based on the findings I synthesized a design framework that presents the main concepts and design considerations that researchers and designers may find useful in designing future technologies in the context of emotion regulation training

NON INVASIVE INVESTIGATION OF SENSORIMOTOR CONTROL FOR FUTURE DEVELOPMENT OF BRAIN-MACHINE-INTERFACE (BMI)

My thesis focuses on describing novel functional connectivity properties of the sensorimotor system that are of potential interest in the field of brain-machine interface. In particular, I have investigated how the connectivity changes as a consequence of either pathologic conditions or spontaneous fluctuations of the brain's internal state. An ad-hoc electronic device has been developed to implement the appropriate experimental settings. First, the functional communication among sensorimotor primary nodes was investigated in multiple sclerosis patients afflicted by persistent fatigue. I selected this condition, for which there is no effective pharmacological treatment, since existing literature links this type of fatigue to the motor control system. In this study, electroencephalographic (EEG) and electromyographic (EMG) traces were acquired together with the pressure exerted on a bulb during an isometric hand grip. The results showed a higher frequency connection between central and peripheral nervous systems (CMC) and an overcorrection of the exerted movement in fatigued multiple sclerosis patients. In fact, even though any fatigue-dependent brain and muscular oscillatory activity alterations were absent, their connectivity worked at higher frequencies as fatigue increased, explaining 67% of the fatigue scale (MFIS) variance (p=.002). In other terms, the functional communication within the central-peripheral nervous systems, namely involving primary sensorimotor areas, was sensitive to tiny alterations in neural connectivity leading to fatigue, well before the appearance of impairments in single nodes of the network. The second study was about connectivity intended as propagation of information and studied in dependence on spontaneous fluctuations of the sensorimotor system triggered by an external stimulus. Knowledge of the propagation mechanisms and of their changes is essential to extract significant information from single trials. The EEG traces were acquired during transcranial magnetic stimulation (TMS) to yield to a deeper knowledge about the response to an external stimulation while the cortico-spinal system passes through different states. The results showed that spontaneous increases of the excitation of the node originating the transmission within the hand control network gave rise to dynamic recruitment patterns with opposite behaviors, weaker in homotopic and parietal circuits, stronger in frontal ones. As probed by TMS, this behavior indicates that the effective connectivity within bilateral circuits orchestrating hand control are dynamically modulated in time even in resting state. The third investigation assessed the plastic changes in the sensorimotor system after stroke induced by 3 months of robotic rehabilitation in chronic phase. A functional source extraction procedure was applied on the acquired EEG data, enabling the investigation of the functional connectivity between homologous areas in the resting state. The most significant result was that the clinical ameliorations were associated to a ‘normalization’ of the functional connectivity between homologous areas. In fact, the brain connectivity did not necessarily increase or decrease, but it settled within a ‘physiological’ range of connectivity. These studies strengthen our knowledge about the behavioral role of the functional connectivity among neuronal networks’ nodes, which will be essential in future developments of enhanced rehabilitative interventions, including brain-machine interfaces. The presented research also moves the definition of new indices of clinical state evaluation relevant for compensating interventions, a step forward

Inclusive Intelligent Learning Management System Framework - Application of Data Science in Inclusive Education

Dissertation presented as the partial requirement for obtaining a Master's degree in Data Science and Advanced Analytics, specialization in Data ScienceBeing a disabled student the author faced higher education with a handicap which as experience studying during COVID 19 confinement periods matched the findings in recent research about the importance of digital accessibility through more e-learning intensive academic experiences. Narrative and systematic literature reviews enabled providing context in World Health Organization’s International Classification of Functioning, Disability and Health, legal and standards framework and information technology and communication state-of-the art. Assessing Portuguese higher education institutions’ web sites alerted to the fact that only outlying institutions implemented near perfect, accessibility-wise, websites. Therefore a gap was identified in how accessible the Portuguese higher education websites are, the needs of all students, including those with disabilities, and even the accessibility minimum legal requirements for digital products and the services provided by public or publicly funded organizations. Having identified a problem in society and exploring the scientific base of knowledge for context and state of the art was a first stage in the Design Science Research methodology, to which followed development and validation cycles of an Inclusive Intelligent Learning Management System Framework. The framework blends various Data Science study fields contributions with accessibility guidelines compliant interface design and content upload accessibility compliance assessment. Validation was provided by a focus group whose inputs were considered for the version presented in this dissertation. Not being the purpose of the research to deliver a complete implementation of the framework and lacking consistent data to put all the modules interacting with each other, the most relevant modules were tested with open data as proof of concept. The rigor cycle of DSR started with the inclusion of the previous thesis on Atlântica University Institute Scientific Repository and is to be completed with the publication of this thesis and the already started PhD’s findings in relevant journals and conferences