Semantic bottleneck for computer vision tasks

This paper introduces a novel method for the representation of images that is semantic by nature, addressing the question of computation intelligibility in computer vision tasks. More specifically, our proposition is to introduce what we call a semantic bottleneck in the processing pipeline, which is a crossing point in which the representation of the image is entirely expressed with natural language , while retaining the efficiency of numerical representations. We show that our approach is able to generate semantic representations that give state-of-the-art results on semantic content-based image retrieval and also perform very well on image classification tasks. Intelligibility is evaluated through user centered experiments for failure detection

'Part'ly first among equals: Semantic part-based benchmarking for state-of-the-art object recognition systems

An examination of object recognition challenge leaderboards (ILSVRC, PASCAL-VOC) reveals that the top-performing classifiers typically exhibit small differences amongst themselves in terms of error rate/mAP. To better differentiate the top performers, additional criteria are required. Moreover, the (test) images, on which the performance scores are based, predominantly contain fully visible objects. Therefore, `harder' test images, mimicking the challenging conditions (e.g. occlusion) in which humans routinely recognize objects, need to be utilized for benchmarking. To address the concerns mentioned above, we make two contributions. First, we systematically vary the level of local object-part content, global detail and spatial context in images from PASCAL VOC 2010 to create a new benchmarking dataset dubbed PPSS-12. Second, we propose an object-part based benchmarking procedure which quantifies classifiers' robustness to a range of visibility and contextual settings. The benchmarking procedure relies on a semantic similarity measure that naturally addresses potential semantic granularity differences between the category labels in training and test datasets, thus eliminating manual mapping. We use our procedure on the PPSS-12 dataset to benchmark top-performing classifiers trained on the ILSVRC-2012 dataset. Our results show that the proposed benchmarking procedure enables additional differentiation among state-of-the-art object classifiers in terms of their ability to handle missing content and insufficient object detail. Given this capability for additional differentiation, our approach can potentially supplement existing benchmarking procedures used in object recognition challenge leaderboards.Comment: Extended version of our ACCV-2016 paper. Author formatting modifie

Localization Recall Precision (LRP): A New Performance Metric for Object Detection

Average precision (AP), the area under the recall-precision (RP) curve, is the standard performance measure for object detection. Despite its wide acceptance, it has a number of shortcomings, the most important of which are (i) the inability to distinguish very different RP curves, and (ii) the lack of directly measuring bounding box localization accuracy. In this paper, we propose 'Localization Recall Precision (LRP) Error', a new metric which we specifically designed for object detection. LRP Error is composed of three components related to localization, false negative (FN) rate and false positive (FP) rate. Based on LRP, we introduce the 'Optimal LRP', the minimum achievable LRP error representing the best achievable configuration of the detector in terms of recall-precision and the tightness of the boxes. In contrast to AP, which considers precisions over the entire recall domain, Optimal LRP determines the 'best' confidence score threshold for a class, which balances the trade-off between localization and recall-precision. In our experiments, we show that, for state-of-the-art object (SOTA) detectors, Optimal LRP provides richer and more discriminative information than AP. We also demonstrate that the best confidence score thresholds vary significantly among classes and detectors. Moreover, we present LRP results of a simple online video object detector which uses a SOTA still image object detector and show that the class-specific optimized thresholds increase the accuracy against the common approach of using a general threshold for all classes. At https://github.com/cancam/LRP we provide the source code that can compute LRP for the PASCAL VOC and MSCOCO datasets. Our source code can easily be adapted to other datasets as well.Comment: to appear in ECCV 201

Direct evidence of low work function on SrVO3_3 cathode using thermionic electron emission microscopy and high-field ultraviolet photoemission spectroscopy

Perovskite SrVO$_3$ has recently been proposed as a novel electron emission cathode material. Density functional theory (DFT) calculations suggest multiple low work function surfaces and recent experimental efforts have consistently demonstrated effective work functions of ~2.7 eV for polycrystalline samples, both results suggesting, but not directly confirming, some fraction of even lower work function surface is present. In this work, thermionic electron emission microscopy (ThEEM) and high-field ultraviolet photoemission spectroscopy are used to study the local work function distribution and measure the work function of a partially-oriented-(110)-SrVO$_3$ perovskite oxide cathode surface. Our results show direct evidence of low work function patches of about 2.1 eV on the cathode surface, with corresponding onset of observable thermionic emission at 750 $^o$C. We hypothesize that, in our ThEEM experiments, the high applied electric field suppresses the patch field effect, enabling the direct measurement of local work functions. This measured work function of 2.1 eV is comparable to the previous DFT-calculated work function value of the SrVO-terminated (110) SrVO$_3$ surface (2.3 eV) and SrO terminated (100) surface (1.9 eV). The measured 2.1 eV value is also much lower than the work function for the (001) LaB$_6$ single crystal cathode (~2.7 eV) and comparable to the effective work function of B-type dispenser cathodes (~2.1 eV). If SrVO$_3$ thermionic emitters can be engineered to access domains of this low 2.1 eV work function, they have potential to significantly improve thermionic emitter-based technologies

Feature pyramid transformer

Feature interactions across space and scales underpin modern visual recognition systems because they introduce beneficial visual contexts. Conventionally, spatial contexts are passively hidden in the CNN's increasing receptive fields or actively encoded by non-local convolution. Yet, the non-local spatial interactions are not across scales, and thus they fail to capture the non-local contexts of objects (or parts) residing in different scales. To this end, we propose a fully active feature interaction across both space and scales, called Feature Pyramid Transformer (FPT). It transforms any feature pyramid into another feature pyramid of the same size but with richer contexts, by using three specially designed transformers in self-level, top-down, and bottom-up interaction fashion. FPT serves as a generic visual backbone with fair computational overhead. We conduct extensive experiments in both instance-level (i.e., object detection and instance segmentation) and pixel-level segmentation tasks, using various backbones and head networks, and observe consistent improvement over all the baselines and the state-of-the-art methods.Comment: Published at the European Conference on Computer Vision, 202

Normal-State Spin Dynamics and Temperature-Dependent Spin Resonance Energy in an Optimally Doped Iron Arsenide Superconductor

The proximity of superconductivity and antiferromagnetism in the phase diagram of iron arsenides, the apparently weak electron-phonon coupling and the "resonance peak" in the superconducting spin excitation spectrum have fostered the hypothesis of magnetically mediated Cooper pairing. However, since most theories of superconductivity are based on a pairing boson of sufficient spectral weight in the normal state, detailed knowledge of the spin excitation spectrum above the superconducting transition temperature Tc is required to assess the viability of this hypothesis. Using inelastic neutron scattering we have studied the spin excitations in optimally doped BaFe1.85Co0.15As2 (Tc = 25 K) over a wide range of temperatures and energies. We present the results in absolute units and find that the normal state spectrum carries a weight comparable to underdoped cuprates. In contrast to cuprates, however, the spectrum agrees well with predictions of the theory of nearly antiferromagnetic metals, without complications arising from a pseudogap or competing incommensurate spin-modulated phases. We also show that the temperature evolution of the resonance energy follows the superconducting energy gap, as expected from conventional Fermi-liquid approaches. Our observations point to a surprisingly simple theoretical description of the spin dynamics in the iron arsenides and provide a solid foundation for models of magnetically mediated superconductivity.Comment: 8 pages, 4 figures, and an animatio

Intra-articular angiolipoma of the knee: a case report

We report a case of intra-articular angiolipoma of the knee. This case report describes our experience in excising an intra-articular angiolipoma of the knee joint. Complete resection under arthroscopy was performed in a 30-year-old man. Two years after the surgery, no evidence of recurrence was seen. Intra-articular angiolipomas should be considered in the differential diagnosis of intra-articular masses in adolescents with recurrent hemarthrosis without trauma

Timed inhibition of CDC7 increases CRISPR-Cas9 mediated templated repair.

Repair of double strand DNA breaks (DSBs) can result in gene disruption or gene modification via homology directed repair (HDR) from donor DNA. Altering cellular responses to DSBs may rebalance editing outcomes towards HDR and away from other repair outcomes. Here, we utilize a pooled CRISPR screen to define host cell involvement in HDR between a Cas9 DSB and a plasmid double stranded donor DNA (dsDonor). We find that the Fanconi Anemia (FA) pathway is required for dsDonor HDR and that other genes act to repress HDR. Small molecule inhibition of one of these repressors, CDC7, by XL413 and other inhibitors increases the efficiency of HDR by up to 3.5 fold in many contexts, including primary T cells. XL413 stimulates HDR during a reversible slowing of S-phase that is unexplored for Cas9-induced HDR. We anticipate that XL413 and other such rationally developed inhibitors will be useful tools for gene modification