A System to Monitor Cognitive Workload in Naturalistic High-Motion Environments

Across many careers, individuals face alternating periods of high and low attention and cognitive workload can impair cognitive function and undermine job performance. We have designed and are developing an unobtrusive system to Monitor, Extract, and Decode Indicators of Cognitive Workload (MEDIC) in naturalistic, high-motion environments. MEDIC is designed to warn individuals, teammates, or supervisors when steps should be taken to augment cognitive readiness. We first designed and manufactured a forehead sensor device that includes a custom fNIRS sensor and a three-axis accelerometer designed to be mounted on the inside of a baseball cap or headband, or standard issue gear such as a helmet or surgeon’s cap. Because the conditions under which MEDIC is designed to operate are more strenuous than typical research efforts assessing cognitive workload, motion artifacts in our data were a persistent issue. Results show wavelet-based filtering improved data quality to salvage data from even the highest-motion conditions. MARA spline motion correction did not further improve data quality. Our testing shows that each of the methods is extremely effective in reducing the effects of motion transients present in the data. In combination, they are able to almost completely remove the transients in the signal while preserving cardiac and low frequency information in the signal which was previously unrecoverable. This has substantially improved the stability of the physiological measures produced by the sensors in high noise conditions

Discovering group dynamics in synchronous time series via hierarchical recurrent switching-state models

We seek to model a collection of time series arising from multiple entities interacting over the same time period. Recent work focused on modeling individual time series is inadequate for our intended applications, where collective system-level behavior influences the trajectories of individual entities. To address such problems, we present a new hierarchical switching-state model that can be trained in an unsupervised fashion to simultaneously explain both system-level and individual-level dynamics. We employ a latent system-level discrete state Markov chain that drives latent entity-level chains which in turn govern the dynamics of each observed time series. Feedback from the observations to the chains at both the entity and system levels improves flexibility via context-dependent state transitions. Our hierarchical switching recurrent dynamical models can be learned via closed-form variational coordinate ascent updates to all latent chains that scale linearly in the number of individual time series. This is asymptotically no more costly than fitting separate models for each entity. Experiments on synthetic and real datasets show that our model can produce better forecasts of future entity behavior than existing methods. Moreover, the availability of latent state chains at both the entity and system level enables interpretation of group dynamics

An Unobtrusive System to Measure, Assess, and Predict Cognitive Workload in Real-World Environments

Across many careers, individuals face alternating periods of high and low attention and cognitive workload, which can result in impaired cognitive functioning and can be detrimental to job performance. For example, some professions (e.g., fire fighters, emergency medical personnel, doctors and nurses working in an emergency room, pilots) require long periods of low workload (boredom), followed by sudden, high-tempo operations during which they may be required to respond to an emergency and perform at peak cognitive levels. Conversely, other professions (e.g., air traffic controllers, market investors in financial industries, analysts) require long periods of high workload and multitasking during which the addition of just one more task results in cognitive overload resulting in mistakes. An unobtrusive system to measure, assess, and predict cognitive workload could warn individuals, their teammates, or their supervisors when steps should be taken to augment cognitive readiness. In this talk I will describe an approach to this problem that we have found to be successful across work domains including: (1) a suite of unobtrusive, field-ready neurophysiological, physiological, and behavioral sensors that are chosen to best suit the target environment; (2) custom algorithms and statistical techniques to process and time-align raw data originating from the sensor suite; (3) probabilistic and statistical models designed to interpret the data into the human state of interest (e.g., cognitive workload, attention, fatigue); (4) and machine-learning techniques to predict upcoming performance based on the current pattern of events, and (5) display of each piece of information depending on the needs of the target user who may or may not want to drill down into the functioning of the system to determine how conclusions about human state and performance are determined. I will then focus in on our experimental results from our custom functional near-infrared spectroscopy sensor, designed to operate in real-world environments to be worn comfortably (e.g., positioned into a baseball cap or a surgeons cap) to measure changes in brain blood oxygenation without adding burden to the individual being assessed

Anatomical and functional impact of critical brain areas to network activity and basic visual function

Thesis (Ph.D.)--Boston University PLEASE NOTE: Boston University Libraries did not receive an Authorization To Manage form for this thesis or dissertation. It is therefore not openly accessible, though it may be available by request. If you are the author or principal advisor of this work and would like to request open access for it, please contact us at [email protected]. Thank you.A set of widely distributed brain areas, collectively known as the fronto-parietal network, serve to modulate aspects of visual perception. However, the unique influence exerted by these regions on low-level visual processing remains unclear. The goals of this thesis were (1) to examine how right frontal, parietal and occipital brain areas interact to process and modulate visual function and (2) to investigate the ability to improve foveal visual performance by means of noninvasive neurostimulation. In a first set of experiments, visual percepts known as 'phosphenes' were measured following low-frequency neurostimulation of the right occipital pole, Intraparietal Sulcus (IPS) or Frontal Eye Fields (FEF). Stimulation of the occipital pole and IPS were capable of evoking phosphenes with similar appearances. Furthermore, occipital or IPS stimulation decreased the excitability of the locally stimulated region but had no effect on the non-stimulated brain area. These results indicate a lack of sufficient inter-regional interactions capable of supporting long-range changes in brain activity. In a second set of experiments, contrast sensitivity and reaction times were assessed as the capacity to detect centrally located, high or low spatial frequency stimuli. Low-frequency rTMS to the FEF, but not the occipital pole or IPS improved contrast sensitivity for high spatial frequency stimuli. Stimulation of the occipital pole decreased reaction times for low spatial frequency stimuli and was shown to depend on transcollicular projections. Finally, stimulation of the IPS decreased reaction times for both types of stimuli. These effects however did not appear to depend on transcollicular pathways, indicating that performance was enhanced through cortico-cortical connections. In a final set of experiments, we investigated whether patterns of individual white matter connectivity linking stimulated brain regions could predict the effects of neurostimulation on visual processing and performance. None of the probability measures however correlated with changes in visual performance. Overall, these data suggest that occipital, parietal, frontal and tectal areas uniquely contribute to the modulation of visual perception. Moreover, results show that targeted stimulation to these brain regions serves to generate lasting improvements in visual performance, which could be used to enhance aspects of vision in healthy and clinical populations

Anatomical and functional impact of critical brain areas to network activity and basic visual function

Thesis (Ph.D.)--Boston University PLEASE NOTE: Boston University Libraries did not receive an Authorization To Manage form for this thesis or dissertation. It is therefore not openly accessible, though it may be available by request. If you are the author or principal advisor of this work and would like to request open access for it, please contact us at [email protected]. Thank you.A set of widely distributed brain areas, collectively known as the fronto-parietal network, serve to modulate aspects of visual perception. However, the unique influence exerted by these regions on low-level visual processing remains unclear. The goals of this thesis were (1) to examine how right frontal, parietal and occipital brain areas interact to process and modulate visual function and (2) to investigate the ability to improve foveal visual performance by means of noninvasive neurostimulation. In a first set of experiments, visual percepts known as 'phosphenes' were measured following low-frequency neurostimulation of the right occipital pole, Intraparietal Sulcus (IPS) or Frontal Eye Fields (FEF). Stimulation of the occipital pole and IPS were capable of evoking phosphenes with similar appearances. Furthermore, occipital or IPS stimulation decreased the excitability of the locally stimulated region but had no effect on the non-stimulated brain area. These results indicate a lack of sufficient inter-regional interactions capable of supporting long-range changes in brain activity. In a second set of experiments, contrast sensitivity and reaction times were assessed as the capacity to detect centrally located, high or low spatial frequency stimuli. Low-frequency rTMS to the FEF, but not the occipital pole or IPS improved contrast sensitivity for high spatial frequency stimuli. Stimulation of the occipital pole decreased reaction times for low spatial frequency stimuli and was shown to depend on transcollicular projections. Finally, stimulation of the IPS decreased reaction times for both types of stimuli. These effects however did not appear to depend on transcollicular pathways, indicating that performance was enhanced through cortico-cortical connections. In a final set of experiments, we investigated whether patterns of individual white matter connectivity linking stimulated brain regions could predict the effects of neurostimulation on visual processing and performance. None of the probability measures however correlated with changes in visual performance. Overall, these data suggest that occipital, parietal, frontal and tectal areas uniquely contribute to the modulation of visual perception. Moreover, results show that targeted stimulation to these brain regions serves to generate lasting improvements in visual performance, which could be used to enhance aspects of vision in healthy and clinical populations

Low frequency transcranial magnetic stimulation of right posterior parietal cortex reduces reaction time to perithreshold low spatial frequency visual stimuli

International audienceResearch in humans and animal models suggests that visual responses in early visual cortical areas may be modulated by top-down influences from distant cortical areas, particularly in the frontal and parietal regions. The right posterior parietal cortex is part of a broad cortical network involved in aspects of visual search and attention, but its role in modulating activity in early visual cortical areas is less well understood. This study evaluated the influence of right posterior parietal cortex (PPC) on a direct measure of visual processing in humans. Contrast sensitivity (CS) and detection response times were recorded using a visual detection paradigm to two types of centrally-presented stimuli. Participants were tested on the detection task before, after, and 1 hour after low-frequency repetitive transcranial magnetic stimulation (rTMS) to the right PPC or to the scalp vertex. Low-frequency rTMS to the right PPC did not significantly change measures of contrast sensitivity, but increased the speed at which participants responded to visual stimuli of low spatial frequency. Response times returned to baseline 1-hour after rTMS. These data indicate that low frequency rTMS to the right PPC speeds up aspects of early visual processing, likely due to a disinhibition of the homotopic left posterior parietal cortex

Characterizing Motion Prediction in Small Autonomous Swarms

The use of robotic swarms has become increasingly common in research, industrial, and military domains for tasks such as collective exploration, coordinated movement, and collective localization. Despite the expanded use of robotic swarms, little is known about how swarms are perceived by human operators. To characterize human-swarm interactions, we evaluate how operators perceive swarm characteristics, including movement patterns, control schemes, and occlusion. In a series of experiments manipulating movement patterns and control schemes, participants tracked swarms on a computer screen until they were occluded from view, at which point participants were instructed to estimate the spatiotemporal dynamics of the occluded swarm by mouse click. In addition to capturing mouse click responses, eye tracking was used to capture participants eye movements while visually tracking swarms. We observed that manipulating control schemes had minimal impact on the perception of swarms, and that swarms are easier to track when they are visible compared to when they were occluded. Regarding swarm movements, a complex pattern of data emerged. For example, eye tracking indicates that participants more closely track a swarm in an arc pattern compared to sinusoid and linear movement patterns. When evaluating behavioral click-responses, data show that time is underestimated, and that spatial accuracy is reduced in complex patterns. Results suggest that measures of performance may capture different patterns of behavior, underscoring the need for multiple measures to accurately characterize performance. In addition, the lack of generalizable data across different movement patterns highlights the complexity involved in the perception of swarms of objects

Characterization of Visual Percepts Evoked by Noninvasive Stimulation of the Human Posterior Parietal Cortex

Phosphenes are commonly evoked by transcranial magnetic stimulation (TMS) to study the functional organization, connectivity, and excitability of the human visual brain. For years, phosphenes have been documented only from stimulating early visual areas (V1–V3) and a handful of specialized visual regions (V4, V5/MT+) in occipital cortex. Recently, phosphenes were reported after applying TMS to a region of posterior parietal cortex involved in the top-down modulation of visuo-spatial processing. In the present study, we systematically characterized parietal phosphenes to determine if they are generated directly by local mechanisms or emerge through indirect activation of other visual areas. Using technology developed in-house to record the subjective features of phosphenes, we found no systematic differences in the size, shape, location, or frame-of-reference of parietal phosphenes when compared to their occipital counterparts. In a second experiment, discrete deactivation by 1 Hz repetitive TMS yielded a double dissociation: phosphene thresholds increased at the deactivated site without producing a corresponding change at the non-deactivated location. Overall, the commonalities of parietal and occipital phosphenes, and our ability to independently modulate their excitability thresholds, lead us t

Visual contrast sensitivity improvement by right frontal high-beta activity is mediated by contrast gain mechanisms and influenced by fronto-parietal white matter microstructure

Behavioral and electrophysiological studies in humans and non-human primates have correlated frontal high-beta activity with the orienting of endogenous attention and shown the ability of the latter function to modulate visual performance. We here combined rhythmic transcranial magnetic stimulation (TMS) and diffusion imaging to study the relation between frontal oscillatory activity and visual performance, and we associated these phenomena to a specific set of white matter pathways that in humans subtend attentional processes. High-beta rhythmic activity on the right frontal eye field (FEF) was induced with TMS and its causal effects on a contrast sensitivity function were recorded to explore its ability to improve visual detection performance across different stimulus contrast levels. Our results show that frequency-specific activity patterns engaged in the right FEF have the ability to induce a leftward shift of the psychometric function. This increase in visual performance across different levels of stimulus contrast is likely mediated by a contrast gain mechanism. Interestingly, microstructural measures of white matter connectivity suggest a strong implication of right fronto-parietal connectivity linking the FEF and the intraparietal sulcus in propagating high-beta rhythmic signals across brain networks and subtending top-down frontal influences on visual performance

A Novel Approach for Documenting Phosphenes Induced by Transcranial Magnetic Stimulation

Stimulation of the human visual cortex produces a transient perception of light, known as a phosphene. Phosphenes are induced by invasive electrical stimulation of the occipital cortex, but also by non-invasive Transcranial Magnetic Stimulation (TMS)1 of the same cortical regions. The intensity at which a phosphene is induced (phosphene threshold) is a well established measure of visual cortical excitability and is used to study cortico-cortical interactions, functional organization 2, susceptibility to pathology 3,4 and visual processing 5-7. Phosphenes are typically defined by three characteristics: they are observed in the visual hemifield contralateral to stimulation; they are induced when the subject s eyes are open or closed, and their spatial location changes with the direction of gaze 2. Various methods have been used to document phosphenes, but a standardized methodology is lacking. We demonstrate a reliable procedure to obtain phosphene threshold values and introduce a novel system for the documentation and analysis of phosphenes. We developed the Laser Tracking and Painting system (LTaP), a low cost, easily built and operated system that records the location and size of perceived phosphenes in real-time. The LTaP system provides a stable and customizable environment for quantification and analysis of phosphenes