Neural Encoding and Decoding with Deep Learning for Natural Vision

The overarching objective of this work is to bridge neuroscience and artificial intelligence to ultimately build machines that learn, act, and think like humans. In the context of vision, the brain enables humans to readily make sense of the visual world, e.g. recognizing visual objects. Developing human-like machines requires understanding the working principles underlying the human vision. In this dissertation, I ask how the brain encodes and represents dynamic visual information from the outside world, whether brain activity can be directly decoded to reconstruct and categorize what a person is seeing, and whether neuroscience theory can be applied to artificial models to advance computer vision. To address these questions, I used deep neural networks (DNN) to establish encoding and decoding models for describing the relationships between the brain and the visual stimuli. Using the DNN, the encoding models were able to predict the functional magnetic resonance imaging (fMRI) responses throughout the visual cortex given video stimuli; the decoding models were able to reconstruct and categorize the visual stimuli based on fMRI activity. To further advance the DNN model, I have implemented a new bidirectional and recurrent neural network based on the predictive coding theory. As a theory in neuroscience, predictive coding explains the interaction among feedforward, feedback, and recurrent connections. The results showed that this brain-inspired model significantly outperforms feedforward-only DNNs in object recognition. These studies have positive impact on understanding the neural computations under human vision and improving computer vision with the knowledge from neuroscience

Contour Integration via Cortical Interactions in Visual Cortex

The visual system possesses a remarkable ability to group fragmented line segments into coherent contours and to segregate them from background. This process, known as contour integration, is critical to identifying object boundaries in complex visual scenes, and thus particularly important for performing shape discrimination, image segmentation and ultimately object recognition. Current evidence supports the idea that long-range horizontal connections in early visual cortex contribute to the process of contour integration, but the underling cortical circuitry, particularly the top-down feedback influence from higher visual areas, is not fully understood. Throughout the thesis, we took computational approaches to systematically examine how contour information is represented across the network of cortical areas and the circuitry by which this information is encoded. Three closely related projects, each having new methods development and hypothesis testing, were performed to analyze and interpret a very large set of neural data. The data set consists of recently acquired multi-electrode multi-unit spikes and local field potentials (LFPs) simultaneously recorded in visual areas V1 and V4 of monkeys performing a visual contour detection task. In the first project, well-established Granger causality measure was extended to the analysis of spiking trains data, which enabled us to quantify the causal interactions within and between areas V1 and V4. Our findings provided clear evidence that there is a top-down V4 feedback influence upon early visual area V1 during contour integration. In the second project, we investigated whether the contour signals in V1 are derived from feedback inputs alone, or whether they are mediated by an intimate interaction between feedback and horizontal connections within V1. Conditional causality measure was developed to dissect the respective contributions of V1 horizontal connections and V4 feedback to contour grouping. Our results suggest that feedback and lateral connections closely interact to mediate the contour integration process. In the third project, a novel Granger causality measure was proposed for the analysis of mixed neural data of spikes and LFP. Spikes and LFP are generated by separate sources with distinct signal characteristics. A joint analysis of spikes and LFP was performed to address the fundamental question about how contour regulates cortical communication between individual neurons and local network activity. The results conform to the general input-output relationship between LFP and spikes within an area. Importantly, we found that contour-related causality is only observed from spikes to LFP, but not in the opposite direction. These findings suggest that Granger causality from spikes to LFP, rather than that from LFP to spikes, carries contour-related information. Taken together, these results indicate that cortical interactions underlie contour integration, thus contribute to a better understanding of the cortical circuitry for parsing visual images and for sensory processing in general. Given the increasing use of multi-electrode recordings in multiple cortical areas, the methodology developed in this thesis should also have a broad impact.Ph.D., Biomedical Engineering -- Drexel University, 201

New Perspectives in Rehabilitation after Traumatic Brain Injury

There has been increased focus on evaluating the scientific knowledge base within the field of traumatic brain injury (TBI) rehabilitation. TBI rehabilitation comprises several phases, from acute medical care to post-acute care in rehabilitation facilities and chronic care in the community. Rehabilitation is a multidisciplinary effort that covers the full spectrum of medical neuroscience, cognitive neuroscience, pharmacology, brain imaging, and assistive and smart technology. A future challenge is to integrate these areas to guide TBI rehabilitation into extensive research and clinical practice. The use of smart technologies and improved brain imaging techniques has an important future in the rehabilitation of patients with cognitive difficulties and disabilities. There is also the need for broad international collaboration to establish large multinational clinical trials in order to define effective service provision and to reach a consensus on the best evidence-based practice of TBI rehabilitation. With this Special Issue, we hope to encourage submissions that discuss ongoing knowledge gaps and controversies, and focus on new perspectives regarding the rehabilitation and management of TBI

Neuromagnetic Decomposition of Social Interaction

Not Available