The Neural Representation Benchmark and its Evaluation on Brain and Machine

A key requirement for the development of effective learning representations is their evaluation and comparison to representations we know to be effective. In natural sensory domains, the community has viewed the brain as a source of inspiration and as an implicit benchmark for success. However, it has not been possible to directly test representational learning algorithms directly against the representations contained in neural systems. Here, we propose a new benchmark for visual representations on which we have directly tested the neural representation in multiple visual cortical areas in macaque (utilizing data from [Majaj et al., 2012]), and on which any computer vision algorithm that produces a feature space can be tested. The benchmark measures the effectiveness of the neural or machine representation by computing the classification loss on the ordered eigendecomposition of a kernel matrix [Montavon et al., 2011]. In our analysis we find that the neural representation in visual area IT is superior to visual area V4. In our analysis of representational learning algorithms, we find that three-layer models approach the representational performance of V4 and the algorithm in [Le et al., 2012] surpasses the performance of V4. Impressively, we find that a recent supervised algorithm [Krizhevsky et al., 2012] achieves performance comparable to that of IT for an intermediate level of image variation difficulty, and surpasses IT at a higher difficulty level. We believe this result represents a major milestone: it is the first learning algorithm we have found that exceeds our current estimate of IT representation performance. We hope that this benchmark will assist the community in matching the representational performance of visual cortex and will serve as an initial rallying point for further correspondence between representations derived in brains and machines.Comment: The v1 version contained incorrectly computed kernel analysis curves and KA-AUC values for V4, IT, and the HT-L3 models. They have been corrected in this versio

Why is Real-World Visual Object Recognition Hard?

Progress in understanding the brain mechanisms underlying vision requires the construction of computational models that not only emulate the brain's anatomy and physiology, but ultimately match its performance on visual tasks. In recent years, “natural” images have become popular in the study of vision and have been used to show apparently impressive progress in building such models. Here, we challenge the use of uncontrolled “natural” images in guiding that progress. In particular, we show that a simple V1-like model—a neuroscientist's “null” model, which should perform poorly at real-world visual object recognition tasks—outperforms state-of-the-art object recognition systems (biologically inspired and otherwise) on a standard, ostensibly natural image recognition test. As a counterpoint, we designed a “simpler” recognition test to better span the real-world variation in object pose, position, and scale, and we show that this test correctly exposes the inadequacy of the V1-like model. Taken together, these results demonstrate that tests based on uncontrolled natural images can be seriously misleading, potentially guiding progress in the wrong direction. Instead, we reexamine what it means for images to be natural and argue for a renewed focus on the core problem of object recognition—real-world image variation

Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognition

The primate visual system achieves remarkable visual object recognition performance even in brief presentations and under changes to object exemplar, geometric transformations, and background variation (a.k.a. core visual object recognition). This remarkable performance is mediated by the representation formed in inferior temporal (IT) cortex. In parallel, recent advances in machine learning have led to ever higher performing models of object recognition using artificial deep neural networks (DNNs). It remains unclear, however, whether the representational performance of DNNs rivals that of the brain. To accurately produce such a comparison, a major difficulty has been a unifying metric that accounts for experimental limitations such as the amount of noise, the number of neural recording sites, and the number trials, and computational limitations such as the complexity of the decoding classifier and the number of classifier training examples. In this work we perform a direct comparison that corrects for these experimental limitations and computational considerations. As part of our methodology, we propose an extension of "kernel analysis" that measures the generalization accuracy as a function of representational complexity. Our evaluations show that, unlike previous bio-inspired models, the latest DNNs rival the representational performance of IT cortex on this visual object recognition task. Furthermore, we show that models that perform well on measures of representational performance also perform well on measures of representational similarity to IT and on measures of predicting individual IT multi-unit responses. Whether these DNNs rely on computational mechanisms similar to the primate visual system is yet to be determined, but, unlike all previous bio-inspired models, that possibility cannot be ruled out merely on representational performance grounds.Comment: 35 pages, 12 figures, extends and expands upon arXiv:1301.353

HPV16 Seropositivity and Subsequent HPV16 Infection Risk in a Naturally Infected Population: Comparison of Serological Assays

Background: Several serological assays have been developed to detect antibodies elicited against infections with oncogenic human papillomavirus (HPV) type 16. The association between antibody levels measured by various assays and subsequent HPV infection risk may differ. We compared HPV16-specific antibody levels previously measured by a virus-like particle (VLP)-based direct enzyme-linked immunoassay (ELISA) with levels measured by additional assays and evaluated the protection against HPV16 infection conferred at different levels of the assays. Methodology/Principal Findings Replicate enrollment serum aliquots from 388 unvaccinated women in the control arm of the Costa Rica HPV vaccine trial were measured for HPV16 seropositivity using three serological assays: a VLP-based direct ELISA; a VLP-based competitive Luminex immunoassay (cLIA); and a secreted alkaline phosphatase protein neutralization assay (SEAP-NA). We assessed the association of assay seropositivity and risk of subsequent HPV16 infection over four years of follow-up by calculating sampling-adjusted odds ratios (OR) and HPV16 seropositivity based on standard cutoff from the cLIA was significantly associated with protection from subsequent HPV16 infection (OR = 0.48, CI = 0.27–0.86, compared with seronegatives). Compared with seronegatives, the highest seropositive tertile antibody levels from the direct ELISA (OR = 0.53, CI = 0.28–0.90) as well as the SEAP-NA (OR = 0.20, CI = 0.06, 0.64) were also significantly associated with protection from HPV16 infection. Conclusions/Significance: Enrollment HPV16 seropositivity by any of the three serological assays evaluated was associated with protection from subsequent infection, although cutoffs for immune protection were different. We defined the assays and seropositivity levels after natural infection that better measure and translate to protective immunity

Single hadron response measurement and calorimeter jet energy scale uncertainty with the ATLAS detector at the LHC

The uncertainty on the calorimeter energy response to jets of particles is derived for the ATLAS experiment at the Large Hadron Collider (LHC). First, the calorimeter response to single isolated charged hadrons is measured and compared to the Monte Carlo simulation using proton-proton collisions at centre-of-mass energies of sqrt(s) = 900 GeV and 7 TeV collected during 2009 and 2010. Then, using the decay of K_s and Lambda particles, the calorimeter response to specific types of particles (positively and negatively charged pions, protons, and anti-protons) is measured and compared to the Monte Carlo predictions. Finally, the jet energy scale uncertainty is determined by propagating the response uncertainty for single charged and neutral particles to jets. The response uncertainty is 2-5% for central isolated hadrons and 1-3% for the final calorimeter jet energy scale.Comment: 24 pages plus author list (36 pages total), 23 figures, 1 table, submitted to European Physical Journal

Setting the standard: multidisciplinary hallmarks for structural, equitable and tracked antibiotic policy

There is increasing concern globally about the enormity of the threats posed by antimicrobial resistance (AMR) to human, animal, plant and environmental health. A proliferation of international, national and institutional reports on the problems posed by AMR and the need for antibiotic stewardship have galvanised attention on the global stage. However, the AMR community increasingly laments a lack of action, often identified as an ‘implementation gap’. At a policy level, the design of internationally salient solutions that are able to address AMR’s interconnected biological and social (historical, political, economic and cultural) dimensions is not straightforward. This multidisciplinary paper responds by asking two basic questions: (A) Is a universal approach to AMR policy and antibiotic stewardship possible? (B) If yes, what hallmarks characterise ‘good’ antibiotic policy? Our multistage analysis revealed four central challenges facing current international antibiotic policy: metrics, prioritisation, implementation and inequality. In response to this diagnosis, we propose three hallmarks that can support robust international antibiotic policy. Emerging hallmarks for good antibiotic policies are: Structural, Equitable and Tracked. We describe these hallmarks and propose their consideration should aid the design and evaluation of international antibiotic policies with maximal benefit at both local and international scale

Standalone vertex finding in the ATLAS muon spectrometer

A dedicated reconstruction algorithm to find decay vertices in the ATLAS muon spectrometer is presented. The algorithm searches the region just upstream of or inside the muon spectrometer volume for multi-particle vertices that originate from the decay of particles with long decay paths. The performance of the algorithm is evaluated using both a sample of simulated Higgs boson events, in which the Higgs boson decays to long-lived neutral particles that in turn decay to bbar b final states, and pp collision data at √s = 7 TeV collected with the ATLAS detector at the LHC during 2011