Understanding deep features with computer-generated imagery

We introduce an approach for analyzing the variation of features generated by convolutional neural networks (CNNs) with respect to scene factors that occur in natural images. Such factors may include object style, 3D viewpoint, color, and scene lighting configuration. Our approach analyzes CNN feature responses corresponding to different scene factors by controlling for them via rendering using a large database of 3D CAD models. The rendered images are presented to a trained CNN and responses for different layers are studied with respect to the input scene factors. We perform a decomposition of the responses based on knowledge of the input scene factors and analyze the resulting components. In particular, we quantify their relative importance in the CNN responses and visualize them using principal component analysis. We show qualitative and quantitative results of our study on three CNNs trained on large image datasets: AlexNet, Places, and Oxford VGG. We observe important differences across the networks and CNN layers for different scene factors and object categories. Finally, we demonstrate that our analysis based on computer-generated imagery translates to the network representation of natural images

Deep Exemplar 2D-3D Detection by Adapting from Real to Rendered Views

This paper presents an end-to-end convolutional neural network (CNN) for 2D-3D exemplar detection. We demonstrate that the ability to adapt the features of natural images to better align with those of CAD rendered views is critical to the success of our technique. We show that the adaptation can be learned by compositing rendered views of textured object models on natural images. Our approach can be naturally incorporated into a CNN detection pipeline and extends the accuracy and speed benefits from recent advances in deep learning to 2D-3D exemplar detection. We applied our method to two tasks: instance detection, where we evaluated on the IKEA dataset, and object category detection, where we out-perform Aubry et al. for "chair" detection on a subset of the Pascal VOC dataset.Comment: To appear in CVPR 201

Painting-to-3D Model Alignment Via Discriminative Visual Elements

International audienceThis paper describes a technique that can reliably align arbitrary 2D depictions of an architectural site, including drawings, paintings and historical photographs, with a 3D model of the site. This is a tremendously difficult task as the appearance and scene structure in the 2D depictions can be very different from the appearance and geometry of the 3D model, e.g., due to the specific rendering style, drawing error, age, lighting or change of seasons. In addition, we face a hard search problem: the number of possible alignments of the painting to a large 3D model, such as a partial reconstruction of a city, is huge. To address these issues, we develop a new compact representation of complex 3D scenes. The 3D model of the scene is represented by a small set of discriminative visual elements that are automatically learnt from rendered views. Similar to object detection, the set of visual elements, as well as the weights of individual features for each element, are learnt in a discriminative fashion. We show that the learnt visual elements are reliably matched in 2D depictions of the scene despite large variations in rendering style (e.g. watercolor, sketch, historical photograph) and structural changes (e.g. missing scene parts, large occluders) of the scene. We demonstrate an application of the proposed approach to automatic re-photography to find an approximate viewpoint of historical paintings and photographs with respect to a 3D model of the site. The proposed alignment procedure is validated via a human user study on a new database of paintings and sketches spanning several sites. The results demonstrate that our algorithm produces significantly better alignments than several baseline methods

Where was this picture painted ? - Localizing paintings by alignment to 3D models

National audienceCet article présente une technique qui peut de manière fiable aligner une représentation non photo-réaliste d'un site architectural, tel un dessin ou une peinture, avec un model 3D du site. Pour ce faire, nous représentons le model 3D par un ensemble d'éléments discriminatifs qui sont automatiquement découverts dans des vues du modèle. Nous montrons que les éléments trouvés sont reliés de manière robuste aux changements de style (aquarelle, croquis, photographies anciennes) et aux différences structurelles. D'avantage de détails sur notre méthode et une évaluation plus détaillée est disponible [1]

Condensation of helium in aerogels and athermal dynamics of the Random Field Ising Model

High resolution measurements reveal that condensation isotherms of $^4$He in a silica aerogel become discontinuous below a critical temperature. We show that this behaviour does not correspond to an equilibrium phase transition modified by the disorder induced by the aerogel structure, but to the disorder-driven critical point predicted for the athermal out-of-equilibrium dynamics of the Random Field Ising Model. Our results evidence the key role of non-equilibrium effects in the phase transitions of disordered systems.Comment: 5 p + suppl. materia

Fast Local Laplacian Filters: Theory and Applications

International audienceMulti-scale manipulations are central to image editing but they are also prone to halos. Achieving artifact-free results requires sophisticated edge- aware techniques and careful parameter tuning. These shortcomings were recently addressed by the local Laplacian filters, which can achieve a broad range of effects using standard Laplacian pyramids. However, these filters are slow to evaluate and their relationship to other approaches is unclear. In this paper, we show that they are closely related to anisotropic diffusion and to bilateral filtering. Our study also leads to a variant of the bilateral filter that produces cleaner edges while retaining its speed. Building upon this result, we describe an acceleration scheme for local Laplacian filters on gray-scale images that yields speed-ups on the order of 50×. Finally, we demonstrate how to use local Laplacian filters to alter the distribution of gradients in an image. We illustrate this property with a robust algorithm for photographic style transfer

Fast and Robust Pyramid-based Image Processing

Multi-scale manipulations are central to image editing but they are also prone to halos. Achieving artifact-free results requires sophisticated edgeaware techniques and careful parameter tuning. These shortcomings were recently addressed by the local Laplacian filters, which can achieve a broad range of effects using standard Laplacian pyramids. However, these filters are slow to evaluate and their relationship to other approaches is unclear. In this paper, we show that they are closely related to anisotropic diffusion and to bilateral filtering. Our study also leads to a variant of the bilateral filter that produces cleaner edges while retaining its speed. Building upon this result, we describe an acceleration scheme for local Laplacian filters that yields speed-ups on the order of 50x. Finally, we demonstrate how to use local Laplacian filters to alter the distribution of gradients in an image. We illustrate this property with a robust algorithm for photographic style transfer

Critical behavior of the liquid gas transition of 4 He confined in a silica aerogel

6 pagesInternational audienceWe have studied 4 He confined in a 95% porosity silica aerogel in the vicinity of the bulk liquid gas critical point. Both thermodynamic measurements and light scattering experiments were performed to probe the effect of a quenched disorder on the liquid gas transition, in relation with the Random Field Ising Model (RFIM). We find that the hysteresis between condensation and evaporation present at lower temperatures disappears at a temperature T ch between 25 and 30 mK below the critical point. Slow relaxations are observed for temperatures slightly below T ch , indicating that some energy barriers, but not all, can be overcome. Above T ch , no density step is observed along the (reversible) isotherms, showing that the critical behavior of the equilibrium phase transition in presence of disorder, if it exists, is shifted to smaller temperatures, where it cannot be observed due to the impossibility to reach equilibrium. Above T ch , light scattering exhibits a weak maximum close to the pressure where the isotherm slope is maximal. This behavior can be accounted for by a simple model incorporating the compression of 4 He close to the silica strands