Not Using the Car to See the Sidewalk: Quantifying and Controlling the Effects of Context in Classification and Segmentation

Importance of visual context in scene understanding tasks is well recognized in the computer vision community. However, to what extent the computer vision models for image classification and semantic segmentation are dependent on the context to make their predictions is unclear. A model overly relying on context will fail when encountering objects in context distributions different from training data and hence it is important to identify these dependencies before we can deploy the models in the real-world. We propose a method to quantify the sensitivity of black-box vision models to visual context by editing images to remove selected objects and measuring the response of the target models. We apply this methodology on two tasks, image classification and semantic segmentation, and discover undesirable dependency between objects and context, for example that "sidewalk" segmentation relies heavily on "cars" being present in the image. We propose an object removal based data augmentation solution to mitigate this dependency and increase the robustness of classification and segmentation models to contextual variations. Our experiments show that the proposed data augmentation helps these models improve the performance in out-of-context scenarios, while preserving the performance on regular data.Comment: 14 pages (12 figures

Baryogenesis from Gravitational Decay of TeV-Particles in Theories with Low Scale Gravity

In models with the fundamental gravity scale in the TeV range, early cosmology is quite different from the standard picture, because the universe must have arisen at a much lower temperature and the electroweak symmetry was probably never restored. In this context, baryogenesis appears to be problematic: if the involved physics is essentially that of the Standard Model, ``conventional'' non-conserving baryon number processes are completely negligible at such low temperatures. In this paper we show that the observed matter-antimatter asymmetry of the universe may be generated by gravitational decay of TeV-mass particles: such objects can be out of equilibrium after inflation and, if their mass is of the same order of magnitude as the true quantum gravity scale, they can quickly decay through a black hole intermediate state, violating global symmetries, in particular, baryon number. In this context, we take advantage of the fact that the ``Sakharov conditions'' for baryogenesis can be more easily satisfied with a low fundamental scale of gravity.Comment: 18 pages, added reference

A region based approach to background modeling in a wavelet multi-resolution framework

In the field of detection and monitoring of dynamic objects in quasi-static scenes, background subtraction techniques where background is modeled at pixel-level, although showing very significant limitations, are extensively used. In this work we propose a novel approach to background modeling that operates at region-level in a wavelet based multi-resolution framework. Based on a segmentation of the background, characterization is made for each region independently as a mixture of K Gaussian modes, considering the model of the approximation and detail coefficients at the different wavelet decomposition levels. Background region characterization is updated along time, and the detection of elements of interest is carried out computing the distance between background region models and those of each incoming image in the sequence. The inclusion of the context in the modeling scheme through each region characterization makes the model robust, being able to support not only gradual illumination and long-term changes, but also sudden illumination changes and the presence of strong shadows in the scen

The Angular Momentum Content and Evolution of Class I and Flat-Spectrum Protostars

We report on the angular momentum content of heavily embedded protostars based on our analysis of the projected rotation velocities (v sin i s) of 38 Class I/flat spectrum young stellar objects presented by Doppmann et al (2005). After correcting for projection effects, we find that infrared-selected Class I/flat spectrum objects rotate significantly more quickly (median equatorial rotation velocity ~ 38 km/sec) than Classical T Tauri stars (CTTSs; median equatorial rotation velocity ~ 18 km/sec) in the Rho Ophiuchi and Taurus-Aurigae regions. The detected difference in rotation speeds between Class I/flat spectrum sources and CTTSs proves difficult to explain without some mechanism which transfers angular momentum out of the protostar between the two phases. Assuming Class I/flat spectrum sources possess physical characteristics (M_*,R_*,B_*) typical of pre-main sequence stars, fully disk locked Class I objects should have co-rotation radii within their protostellar disks that match well (within 30%) with the predicted magnetic coupling radii of Shu et al (1994). The factor of two difference in rotation rates between Class I/flat spectrum and CTTS sources, when interpreted in the context of disk locking models, also imply a factor of 5 or greater difference in mass accretion rates between the two phases.Comment: 13 pages, 6 figures. Accepted for publication in the Astronomical Journal (tentatively for June 2005 edition

Neurobiological Mechanisms for Semantic Feature Extraction and Conceptual Flexibility

Signs and symbols relate to concepts and can be used to speak about objects, actions, and their features. Theories of semantic grounding address the question how the latter two, concepts and real‐world entities, come into play and interlink in symbol learning. Here, a neurobiological model is used to spell out concrete mechanisms of symbol grounding, which implicate the “association” of information about sign and referents and, at the same time, the extraction of semantic features and the formation of abstract representations best described as conjoined and disjoined feature sets that may or may not have a real‐life equivalent. The mechanistic semantic circuits carrying these feature sets are not static conceptual entries, but exhibit rich activation dynamics related to memory, prediction, and contextual modulation. Four key issues in specifying these activation dynamics will be highlighted: (a) the inner structure of semantic circuits, (b) mechanisms of semantic priming, (c) task specificity in semantic activation, and (d) context‐dependent semantic circuit activation in the processing of referential, existential, and universal statements. These linguistic‐semantic examples show that specific mechanisms are required to account for context‐dependent semantic function or conceptual “flexibility.” Static context‐independent concepts as such are insufficient to account for these different semantic functions. Whereas abstract amodal models of concepts did so far not spell out concrete mechanisms for context‐dependent semantic function, neuronal assembly mechanisms offer a workable perspective

A Tree-Based Context Model for Object Recognition

There has been a growing interest in exploiting contextual information in addition to local features to detect and localize multiple object categories in an image. A context model can rule out some unlikely combinations or locations of objects and guide detectors to produce a semantically coherent interpretation of a scene. However, the performance benefit of context models has been limited because most of the previous methods were tested on datasets with only a few object categories, in which most images contain one or two object categories. In this paper, we introduce a new dataset with images that contain many instances of different object categories, and propose an efficient model that captures the contextual information among more than a hundred object categories using a tree structure. Our model incorporates global image features, dependencies between object categories, and outputs of local detectors into one probabilistic framework. We demonstrate that our context model improves object recognition performance and provides a coherent interpretation of a scene, which enables a reliable image querying system by multiple object categories. In addition, our model can be applied to scene understanding tasks that local detectors alone cannot solve, such as detecting objects out of context or querying for the most typical and the least typicalscenes in a dataset.This research was partially funded by Shell International Exploration and Production Inc., by Army Research Office under award W911NF-06-1-0076, by NSF Career Award (ISI 0747120), and by the Air Force Office of Scientific Research under Award No.FA9550-06-1-0324. Any opinions, findings, and conclusions or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the views of the Air Force

SHOP: A Deep Learning Based Pipeline for near Real-Time Detection of Small Handheld Objects Present in Blurry Video

While prior works have investigated and developed computational models capable of object detection, models still struggle to reliably interpret images with motion blur and small objects. Moreover, none of these models are specifically designed for handheld object detection. In this work, we present SHOP (Small Handheld Object Pipeline), a pipeline that reliably and efficiently interprets blurry images containing handheld objects. The specific models used in each stage of the pipeline are flexible and can be changed based on performance requirements. First, images are deblurred and then run through a pose detection system where areas-of-interest are proposed around the hands of any people present. Next, object detection is performed on the images by a single-stage object detector. Finally, the proposed areas-of-interest are used to filter out low confidence detections. Testing on a handheld subset of Microsoft Common Objects in Context (MS COCO) demonstrates that this 3 stage process results in a 70 percent decrease in false positives while only reducing true positives by 17 percent in its strongest configuration. We also present a subset of MS COCO consisting solely of handheld objects that can be used to continue the development of handheld object detection methods. https://github.com/spider-sense/SHOPComment: 8 pages, 5 figures. Accepted to IEEE SoutheastCon 202