TEMPORAL CODING OF SPEECH IN HUMAN AUDITORY CORTEX

Human listeners can reliably recognize speech in complex listening environments. The underlying neural mechanisms, however, remain unclear and cannot yet be emulated by any artificial system. In this dissertation, we study how speech is represented in the human auditory cortex and how the neural representation contributes to reliable speech recognition. Cortical activity from normal hearing human subjects is noninvasively recorded using magnetoencephalography, during natural speech listening. It is first demonstrated that neural activity from auditory cortex is precisely synchronized to the slow temporal modulations of speech, when the speech signal is presented in a quiet listening environment. How this neural representation is affected by acoustic interference is then investigated. Acoustic interference degrades speech perception via two mechanisms, informational masking and energetic masking, which are addressed respectively by using a competing speech stream and a stationary noise as the interfering sound. When two speech streams are presented simultaneously, cortical activity is predominantly synchronized to the speech stream the listener attends to, even if the unattended, competing speech stream is 8 dB more intense. When speech is presented together with spectrally matched stationary noise, cortical activity remains precisely synchronized to the temporal modulations of speech until the noise is 9 dB more intense. Critically, the accuracy of neural synchronization to speech predicts how well individual listeners can understand speech in noise. Further analysis reveals that two neural sources contribute to speech synchronized cortical activity, one with a shorter response latency of about 50 ms and the other with a longer response latency of about 100 ms. The longer-latency component, but not the shorter-latency component, shows selectivity to the attended speech and invariance to background noise, indicating a transition from encoding the acoustic scene to encoding the behaviorally important auditory object, in auditory cortex. Taken together, we have demonstrated that during natural speech comprehension, neural activity in the human auditory cortex is precisely synchronized to the slow temporal modulations of speech. This neural synchronization is robust to acoustic interference, whether speech or noise, and therefore provides a strong candidate for the neural basis of acoustic background invariant speech recognition

Dynamic Decomposition of Spatiotemporal Neural Signals

Neural signals are characterized by rich temporal and spatiotemporal dynamics that reflect the organization of cortical networks. Theoretical research has shown how neural networks can operate at different dynamic ranges that correspond to specific types of information processing. Here we present a data analysis framework that uses a linearized model of these dynamic states in order to decompose the measured neural signal into a series of components that capture both rhythmic and non-rhythmic neural activity. The method is based on stochastic differential equations and Gaussian process regression. Through computer simulations and analysis of magnetoencephalographic data, we demonstrate the efficacy of the method in identifying meaningful modulations of oscillatory signals corrupted by structured temporal and spatiotemporal noise. These results suggest that the method is particularly suitable for the analysis and interpretation of complex temporal and spatiotemporal neural signals

Allocation of Computational Resources in the Nervous System.

The nervous system integrates past information together with predictions about the future in order to produce rewarding actions for the organism. This dissertation focuses on the resources underlying these computations, and the task-dependent allocation of these resources. We present evidence that principles from optimal coding and optimal estimation account for overt and covert orienting phenomena, as observed from both behavioral experiments and neuronal recordings. First, we review behavioral measurements related to selective attention and discuss models that account for these data. We show that reallocation of resources emerges as a natural property of systems that encode their inputs efficiently under non-uniform constraints. We continue by discussing the attentional modulation of neuronal activity, and showthat: (1) Modulation of coding strategies does not require special mechanisms: it is possible to obtain dramatic modulation even when signals informing the system about fidelity requirements enter the system in a fashion indistinguishable from sensory signals. (2) Optimal coding under non-uniform fidelity requirements is sufficient to account for the firing rate modulation observed during selective attention experiments. (3) The response of a single neuron cannot bewell characterized by measurements of attentional modulation of only a single sensory stimulus. (4) The magnitude of the activity modulation depends on the capacity of the neural circuit. A later chapter discusses the neural mechanisms for resource allocation, and the relation between attentional mechanisms and receptive field formation. The remainder of the dissertation focuses on overt orienting phenomena and active perception. We present a theoretical analysis of the allocation of resources during state estimation of multiple targets with different uncertainties, together with eye-tracking experiments that confirm our predictions. We finish by discussing the implications of these results to our current understanding of orienting phenomena and the neural code

Time-varying functional connectivity and dynamic neurofeedback with MEG: methods and applications to visual perception

Cognitive function involves the interplay of functionally-separate regions of the human brain. Of critical importance to neuroscience research is to accurately measure the activity and communication between these regions. The MEG imaging modality is well-suited to capturing functional cortical communication due to its high temporal resolution, on the millisecond scale. However, localizing the sources of cortical activity from the sensor measurements is an ill-posed problem, where different solutions trade-off between spatial accuracy, correcting for linear mixing of cortical signals, and computation time. Linear mixing, in particular, affects the reliability of many connectivity measures. We present a MATLAB-based pipeline that we developed to correct for linear mixing and compute time-varying connectivity (phase synchrony, Granger Causality) between cortically-defined regions interfacing with established toolboxes for MEG data processing (Minimum Norm Estimation Toolbox, Brainstorm, Fieldtrip). In Chapter 1, we present a new method for localizing cortical activation while controlling cross-talk on the cortex. In Chapter 2, we apply a nonparametric statistical test for measuring phase locking in the presence of cross-talk. Chapters 3 and 4 describe the application of the pipeline to MEG data collected from subjects performing a visual object motion detection task. Chapter 5 focuses on real-time MEG (rt-MEG) neurofeedback which is the real-time measurement of brain activity and its self-regulation through feedback. Typically neurofeedback modulates directly brain activation for the purpose of training sensory, motor, emotional or cognitive functions. Direct measures, however, are not suited to training dynamic measures of brain activity, such as the speed of switching between tasks, for example. We developed a novel rt-MEG neurofeedback method called state-based neurofeedback, where brain activity states related to subject behavior are decoded in real-time from the MEG sensor measurements. The timing related to maintaining or transitioning between decoded states is then presented as feedback to the subject. In a group of healthy subjects we applied the state-based neurofeedback method for training the time required for switching spatial attention from one side of the visual field to the other (e.g. left side to right side) following a brief presentation of a visual cue. In Chapter 6, we used our pipeline to investigate training-related changes in cortical activation and network connectivity in each subject. Our results suggested that the rt-MEG neurofeedback training resulted in strengthened beta-band connectivity prior to the switch of spatial attention, and strengthened gamma-band connectivity during the switch. There were two goals of this dissertation: First was the development of the MATLAB-based pipeline for computing time-evolving functional connectivity analysis in MEG and its application to visual motion perception. The second goal was the development of a real-time MEG neurofeedback method to train the dynamics of brain states and its application to a group of healthy subjects.2019-11-02T00:00:00

A Self-Organizing Neural System for Learning to Recognize Textured Scenes

A self-organizing ARTEX model is developed to categorize and classify textured image regions. ARTEX specializes the FACADE model of how the visual cortex sees, and the ART model of how temporal and prefrontal cortices interact with the hippocampal system to learn visual recognition categories and their names. FACADE processing generates a vector of boundary and surface properties, notably texture and brightness properties, by utilizing multi-scale filtering, competition, and diffusive filling-in. Its context-sensitive local measures of textured scenes can be used to recognize scenic properties that gradually change across space, as well a.s abrupt texture boundaries. ART incrementally learns recognition categories that classify FACADE output vectors, class names of these categories, and their probabilities. Top-down expectations within ART encode learned prototypes that pay attention to expected visual features. When novel visual information creates a poor match with the best existing category prototype, a memory search selects a new category with which classify the novel data. ARTEX is compared with psychophysical data, and is benchmarked on classification of natural textures and synthetic aperture radar images. It outperforms state-of-the-art systems that use rule-based, backpropagation, and K-nearest neighbor classifiers.Defense Advanced Research Projects Agency; Office of Naval Research (N00014-95-1-0409, N00014-95-1-0657

Relative advantage of touch over vision in the exploration of texture

Texture segmentation is an effortless process in scene analysis, yet its mechanisms have not been sufficiently understood. Several theories and algorithms exist for texture discrimination based on vision. These models diverge from one another in algorithmic approaches to address texture imagery using spatial elements and their statistics. Even though there are differences among these approaches, they all begin from the assumption that texture segmentation is a visual task. However, considering that texture is basically a surface property, this assumption can at times be misleading. An interesting possibility is that since surface properties are most immediately accessible to touch, texture perception may be more intimately associated with texture than with vision (it is known that tactile input can affect vision). Coincidentally, the basic organization of the touch (somatosensory) system bears some analogy to that of the visual system. In particular, recent neurophysiological findings showed that receptive fields for touch resemble that of vision, albeit with some subtle differences. The main novelty and contribution of this thesis is in the use of tactile receptive field responses for texture segmentation. Furthermore, we showed that touch-based representation is superior to its vision-based counterpart when used in texture boundary detection. Tactile representations were also found to be more discriminable (LDA and ANOVA). We expect our results to help better understand the nature of texture perception and build more powerful texture processing algorithms. The results suggest that touch has an advantage over vision in texture processing. Findings in this study are expected to shed new light on the role of tactile perception of texture and its interaction with vision, and help develop more powerful, biologically inspired texture segmentation algorithms

Scalable Machine Learning Methods for Massive Biomedical Data Analysis.

Modern data acquisition techniques have enabled biomedical researchers to collect and analyze datasets of substantial size and complexity. The massive size of these datasets allows us to comprehensively study the biological system of interest at an unprecedented level of detail, which may lead to the discovery of clinically relevant biomarkers. Nonetheless, the dimensionality of these datasets presents critical computational and statistical challenges, as traditional statistical methods break down when the number of predictors dominates the number of observations, a setting frequently encountered in biomedical data analysis. This difficulty is compounded by the fact that biological data tend to be noisy and often possess complex correlation patterns among the predictors. The central goal of this dissertation is to develop a computationally tractable machine learning framework that allows us to extract scientifically meaningful information from these massive and highly complex biomedical datasets. We motivate the scope of our study by considering two important problems with clinical relevance: (1) uncertainty analysis for biomedical image registration, and (2) psychiatric disease prediction based on functional connectomes, which are high dimensional correlation maps generated from resting state functional MRI.PhDElectrical Engineering: SystemsUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/111354/1/takanori_1.pd

Online Multi-Stage Deep Architectures for Feature Extraction and Object Recognition

Multi-stage visual architectures have recently found success in achieving high classification accuracies over image datasets with large variations in pose, lighting, and scale. Inspired by techniques currently at the forefront of deep learning, such architectures are typically composed of one or more layers of preprocessing, feature encoding, and pooling to extract features from raw images. Training these components traditionally relies on large sets of patches that are extracted from a potentially large image dataset. In this context, high-dimensional feature space representations are often helpful for obtaining the best classification performances and providing a higher degree of invariance to object transformations. Large datasets with high-dimensional features complicate the implementation of visual architectures in memory constrained environments. This dissertation constructs online learning replacements for the components within a multi-stage architecture and demonstrates that the proposed replacements (namely fuzzy competitive clustering, an incremental covariance estimator, and multi-layer neural network) can offer performance competitive with their offline batch counterparts while providing a reduced memory footprint. The online nature of this solution allows for the development of a method for adjusting parameters within the architecture via stochastic gradient descent. Testing over multiple datasets shows the potential benefits of this methodology when appropriate priors on the initial parameters are unknown. Alternatives to batch based decompositions for a whitening preprocessing stage which take advantage of natural image statistics and allow simple dictionary learners to work well in the problem domain are also explored. Expansions of the architecture using additional pooling statistics and multiple layers are presented and indicate that larger codebook sizes are not the only step forward to higher classification accuracies. Experimental results from these expansions further indicate the important role of sparsity and appropriate encodings within multi-stage visual feature extraction architectures

Towards building a more complex view of the lateral geniculate nucleus: Recent advances in understanding its role

The lateral geniculate nucleus (LGN) has often been treated in the past as a linear filter that adds little to retinal processing of visual inputs. Here we review anatomical, neurophysiological, brain imaging, and modeling studies that have in recent years built up a much more complex view of LGN . These include effects related to nonlinear dendritic processing, cortical feedback, synchrony and oscillations across LGN populations, as well as involvement of LGN in higher level cognitive processing. Although recent studies have provided valuable insights into early visual processing including the role of LGN, a unified model of LGN responses to real-world objects has not yet been developed. In the light of recent data, we suggest that the role of LGN deserves more careful consideration in developing models of high-level visual processing