The Neural Representation Benchmark and its Evaluation on Brain and Machine

A key requirement for the development of effective learning representations is their evaluation and comparison to representations we know to be effective. In natural sensory domains, the community has viewed the brain as a source of inspiration and as an implicit benchmark for success. However, it has not been possible to directly test representational learning algorithms directly against the representations contained in neural systems. Here, we propose a new benchmark for visual representations on which we have directly tested the neural representation in multiple visual cortical areas in macaque (utilizing data from [Majaj et al., 2012]), and on which any computer vision algorithm that produces a feature space can be tested. The benchmark measures the effectiveness of the neural or machine representation by computing the classification loss on the ordered eigendecomposition of a kernel matrix [Montavon et al., 2011]. In our analysis we find that the neural representation in visual area IT is superior to visual area V4. In our analysis of representational learning algorithms, we find that three-layer models approach the representational performance of V4 and the algorithm in [Le et al., 2012] surpasses the performance of V4. Impressively, we find that a recent supervised algorithm [Krizhevsky et al., 2012] achieves performance comparable to that of IT for an intermediate level of image variation difficulty, and surpasses IT at a higher difficulty level. We believe this result represents a major milestone: it is the first learning algorithm we have found that exceeds our current estimate of IT representation performance. We hope that this benchmark will assist the community in matching the representational performance of visual cortex and will serve as an initial rallying point for further correspondence between representations derived in brains and machines.Comment: The v1 version contained incorrectly computed kernel analysis curves and KA-AUC values for V4, IT, and the HT-L3 models. They have been corrected in this versio

Visualizing classification of natural video sequences using sparse, hierarchical models of cortex.

Recent work on hierarchical models of visual cortex has reported state-of-the-art accuracy on whole-scene labeling using natural still imagery. This raises the question of whether the reported accuracy may be due to the sophisticated, non-biological back-end supervised classifiers typically used (support vector machines) and/or the limited number of images used in these experiments. In particular, is the model classifying features from the object or the background? Previous work (Landecker, Brumby, et al., COSYNE 2010) proposed tracing the spatial support of a classifier’s decision back through a hierarchical cortical model to determine which parts of the image contributed to the classification, compared to the positions of objects in the scene. In this way, we can go beyond standard measures of accuracy to provide tools for visualizing and analyzing high-level object classification. We now describe new work exploring the extension of these ideas to detection of objects in video sequences of natural scenes

Totally Implantable Bidirectional Neural Prostheses: A Flexible Platform for Innovation in Neuromodulation.

Implantable neural prostheses are in widespread use for treating a variety of brain disorders. Until recently, most implantable brain devices have been unidirectional, either delivering neurostimulation without brain sensing, or sensing brain activity to drive external effectors without a stimulation component. Further, many neural interfaces that incorporate a sensing function have relied on hardwired connections, such that subjects are tethered to external computers and cannot move freely. A new generation of neural prostheses has become available, that are both bidirectional (stimulate as well as record brain activity) and totally implantable (no externalized connections). These devices provide an opportunity for discovering the circuit basis for neuropsychiatric disorders, and to prototype personalized neuromodulation therapies that selectively interrupt neural activity underlying specific signs and symptoms

Reduced neural sensitivity to social stimuli in infants at risk for autism

In the hope of discovering early markers of autism, attention has recently turned to the study of infants at risk owing to being the younger siblings of children with autism. Because the condition is highly heritable, later-born siblings of diagnosed children are at substantially higher risk for developing autism or the broader autism phenotype than the general population. Currently, there are no strong predictors of autism in early infancy and diagnosis is not reliable until around 3 years of age. Because indicators of brain functioning may be sensitive predictors, and atypical social interactions are characteristic of the syndrome, we examined whether temporal lobe specialization for processing visual and auditory social stimuli during infancy differs in infants at risk. In a functional near-infrared spectroscopy study, infants aged 4–6 months at risk for autism showed less selective neural responses to social stimuli (auditory and visual) than low-risk controls. These group differences could not be attributed to overall levels of attention, developmental stage or chronological age. Our results provide the first demonstration of specific differences in localizable brain function within the first 6 months of life in a group of infants at risk for autism. Further, these differences closely resemble known patterns of neural atypicality in children and adults with autism. Future work will determine whether these differences in infant neural responses to social stimuli predict either later autism or the broader autism phenotype frequently seen in unaffected family members