Encoding and Decoding Brain Signals in the Primate Visual Cortex Using Deep Learning

Tohoku University岡谷貴之課

Learning to Dream, Dreaming to Learn

The importance of sleep for healthy brain function is widely acknowledged. However, it remains mysterious how the sleeping brain, disconnected from the outside world and plunged into the fantastic experiences of dreams, is actively learning. A main feature of dreams is the generation of new realistic sensory experiences in absence of external input, from the combination of diverse memory elements. How do cortical networks host the generation of these sensory experiences during sleep? What function could these generated experiences serve? In this thesis, we attempt to answer these questions using an original, computational approach inspired by modern artificial intelligence. In light of existing cognitive theories and experimental data, we suggest that cortical networks implement a generative model of the sensorium that is systematically optimized during wakefulness and sleep states. By performing network simulations on datasets of natural images, our results not only propose potential mechanisms for dream generation during sleep states, but suggest that dreaming is an essential feature for learning semantic representations throughout mammalian development

Machine Learning As Tool And Theory For Computational Neuroscience

Computational neuroscience is in the midst of constructing a new framework for understanding the brain based on the ideas and methods of machine learning. This is effort has been encouraged, in part, by recent advances in neural network models. It is also driven by a recognition of the complexity of neural computation and the challenges that this poses for neuroscience’s methods. In this dissertation, I first work to describe these problems of complexity that have prompted a shift in focus. In particular, I develop machine learning tools for neurophysiology that help test whether tuning curves and other statistical models in fact capture the meaning of neural activity. Then, taking up a machine learning framework for understanding, I consider theories about how neural computation emerges from experience. Specifically, I develop hypotheses about the potential learning objectives of sensory plasticity, the potential learning algorithms in the brain, and finally the consequences for sensory representations of learning with such algorithms. These hypotheses pull from advances in several areas of machine learning, including optimization, representation learning, and deep learning theory. Each of these subfields has insights for neuroscience, offering up links for a chain of knowledge about how we learn and think. Together, this dissertation helps to further an understanding of the brain in the lens of machine learning

Getting aligned on representational alignment

Biological and artificial information processing systems form representations that they can use to categorize, reason, plan, navigate, and make decisions. How can we measure the extent to which the representations formed by these diverse systems agree? Do similarities in representations then translate into similar behavior? How can a system's representations be modified to better match those of another system? These questions pertaining to the study of representational alignment are at the heart of some of the most active research areas in cognitive science, neuroscience, and machine learning. For example, cognitive scientists measure the representational alignment of multiple individuals to identify shared cognitive priors, neuroscientists align fMRI responses from multiple individuals into a shared representational space for group-level analyses, and ML researchers distill knowledge from teacher models into student models by increasing their alignment. Unfortunately, there is limited knowledge transfer between research communities interested in representational alignment, so progress in one field often ends up being rediscovered independently in another. Thus, greater cross-field communication would be advantageous. To improve communication between these fields, we propose a unifying framework that can serve as a common language between researchers studying representational alignment. We survey the literature from all three fields and demonstrate how prior work fits into this framework. Finally, we lay out open problems in representational alignment where progress can benefit all three of these fields. We hope that our work can catalyze cross-disciplinary collaboration and accelerate progress for all communities studying and developing information processing systems. We note that this is a working paper and encourage readers to reach out with their suggestions for future revisions.Comment: Working paper, changes to be made in upcoming revision

Computational Mechanisms of Face Perception

The intertwined history of artificial intelligence and neuroscience has significantly impacted their development, with AI arising from and evolving alongside neuroscience. The remarkable performance of deep learning has inspired neuroscientists to investigate and utilize artificial neural networks as computational models to address biological issues. Studying the brain and its operational mechanisms can greatly enhance our understanding of neural networks, which has crucial implications for developing efficient AI algorithms. Many of the advanced perceptual and cognitive skills of biological systems are now possible to achieve through artificial intelligence systems, which is transforming our knowledge of brain function. Thus, the need for collaboration between the two disciplines demands emphasis. It\u27s both intriguing and challenging to study the brain using computer science approaches, and this dissertation centers on exploring computational mechanisms related to face perception. Face recognition, being the most active artificial intelligence research area, offers a wealth of data resources as well as a mature algorithm framework. From the perspective of neuroscience, face recognition is an important indicator of social cognitive formation and neural development. The ability to recognize faces is one of the most important cognitive functions. We first discuss the problem of how the brain encodes different face identities. By using DNNs to extract features from complex natural face images and project them into the feature space constructed by dimension reduction, we reveal a new face code in the human medial temporal lobe (MTL), where neurons encode visually similar identities. On this basis, we discover a subset of DNN units that are selective for facial identity. These identity-selective units exhibit a general ability to discriminate novel faces. By establishing coding similarities with real primate neurons, our study provides an important approach to understanding primate facial coding. Lastly, we discuss the impact of face learning during the critical period. We identify a critical period during DNN training and systematically discuss the use of facial information by the neural network both inside and outside the critical period. We further provide a computational explanation for the critical period influencing face learning through learning rate changes. In addition, we show an alternative method to partially recover the model outside the critical period by knowledge refinement and attention shifting. Our current research not only highlights the importance of training orientation and visual experience in shaping neural responses to face features and reveals potential mechanisms for face recognition but also provides a practical set of ideas to test hypotheses and reconcile previous findings in neuroscience using computer methods