The Visual Centrifuge: Model-Free Layered Video Representations

True video understanding requires making sense of non-lambertian scenes where the color of light arriving at the camera sensor encodes information about not just the last object it collided with, but about multiple mediums -- colored windows, dirty mirrors, smoke or rain. Layered video representations have the potential of accurately modelling realistic scenes but have so far required stringent assumptions on motion, lighting and shape. Here we propose a learning-based approach for multi-layered video representation: we introduce novel uncertainty-capturing 3D convolutional architectures and train them to separate blended videos. We show that these models then generalize to single videos, where they exhibit interesting abilities: color constancy, factoring out shadows and separating reflections. We present quantitative and qualitative results on real world videos.Comment: Appears in: 2019 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2019). This arXiv contains the CVPR Camera Ready version of the paper (although we have included larger figures) as well as an appendix detailing the model architectur

The Casimir effect from the point of view of algebraic quantum field theory

We consider a region of Minkowski spacetime bounded either by one or by two parallel, infinitely extended plates orthogonal to a spatial direction and a real Klein-Gordon field satisfying Dirichlet boundary conditions. We quantize these two systems within the algebraic approach to quantum field theory using the so-called functional formalism. As a first step we construct a suitable unital *-algebra of observables whose generating functionals are characterized by a labelling space which is at the same time optimal and separating and fulfils the F-locality property. Subsequently we give a definition for these systems of Hadamard states and we investigate explicit examples. In the case of a single plate, it turns out that one can build algebraic states via a pull-back of those on the whole Minkowski spacetime, moreover inheriting from them the Hadamard property. When we consider instead two plates, algebraic states can be put in correspondence with those on flat spacetime via the so-called method of images, which we translate to the algebraic setting. For a massless scalar field we show that this procedure works perfectly for a large class of quasi-free states including the Poincar\'e vacuum and KMS states. Eventually Wick polynomials are introduced. Contrary to the Minkowski case, the extended algebras, built in globally hyperbolic subregions can be collected in a global counterpart only after a suitable deformation which is expressed locally in terms of a *-isomorphism. As a last step, we construct explicitly the two-point function and the regularized energy density, showing, moreover, that the outcome is consistent with the standard results of the Casimir effect.Comment: 45 pages, section 2 improved, typos correcte

Surface Vacuum Energy in Cutoff Models: Pressure Anomaly and Distributional Gravitational Limit

Vacuum-energy calculations with ideal reflecting boundaries are plagued by boundary divergences, which presumably correspond to real (but finite) physical effects occurring near the boundary. Our working hypothesis is that the stress tensor for idealized boundary conditions with some finite cutoff should be a reasonable ad hoc model for the true situation. The theory will have a sensible renormalized limit when the cutoff is taken away; this requires making sense of the Einstein equation with a distributional source. Calculations with the standard ultraviolet cutoff reveal an inconsistency between energy and pressure similar to the one that arises in noncovariant regularizations of cosmological vacuum energy. The problem disappears, however, if the cutoff is a spatial point separation in a "neutral" direction parallel to the boundary. Here we demonstrate these claims in detail, first for a single flat reflecting wall intersected by a test boundary, then more rigorously for a region of finite cross section surrounded by four reflecting walls. We also show how the moment-expansion theorem can be applied to the distributional limits of the source and the solution of the Einstein equation, resulting in a mathematically consistent differential equation where cutoff-dependent coefficients have been identified as renormalizations of properties of the boundary. A number of issues surrounding the interpretation of these results are aired.Comment: 22 pages, 2 figures, 1 table; PACS 03.70.+k, 04.20.Cv, 11.10.G

A generative model for separating illumination and reflectance from images

It is well known that even slight changes in nonuniform illumination lead to a large image variability and are crucial for many visual tasks. This paper presents a new ICA related probabilistic model where the number of sources exceeds the number of sensors to perform an image segmentation and illumination removal, simultaneously. We model illumination and reflectance in log space by a generalized autoregressive process and Hidden Gaussian Markov random field, respectively. The model ability to deal with segmentation of illuminated images is compared with a Canny edge detector and homomorphic filtering. We apply the model to two problems: synthetic image segmentation and sea surface pollution detection from intensity images

Difference image photometry with bright variable backgrounds

Over the last two decades the Andromeda Galaxy (M31) has been something of a test-bed for methods aimed at obtaining accurate time-domain relative photometry within highly crowded fields. Difference imaging methods, originally pioneered towards M31, have evolved into sophisticated methods, such as the Optimal Image Subtraction (OIS) method of Alard & Lupton (1998), that today are most widely used to survey variable stars, transients and microlensing events in our own Galaxy. We show that modern difference image (DIA) algorithms such as OIS, whilst spectacularly successful towards the Milky Way bulge, may perform badly towards high surface brightness targets such as the M31 bulge. Poor results can occur in the presence of common systematics which add spurious flux contributions to images, such as internal reflections, scattered light or fringing. Using data from the Angstrom Project microlensing survey of the M31 bulge, we show that very good results are usually obtainable by first performing careful photometric alignment prior to using OIS to perform point-spread function (PSF) matching. This separation of background matching and PSF matching, a common feature of earlier M31 photometry techniques, allows us to take full advantage of the powerful PSF matching flexibility offered by OIS towards high surface brightness targets. We find that difference images produced this way have noise distributions close to Gaussian, showing significant improvement upon results achieved using OIS alone. We show that with this correction light-curves of variable stars and transients can be recovered to within ~10 arcseconds of the M31 nucleus. Our method is simple to implement and is quick enough to be incorporated within real-time DIA pipelines. (Abridged)Comment: 12 pages. Accepted for publication in MNRAS. Includes an expanded discussion of DIA testing and results, including additional lightcurve example

A stochastic algorithm for probabilistic independent component analysis

The decomposition of a sample of images on a relevant subspace is a recurrent problem in many different fields from Computer Vision to medical image analysis. We propose in this paper a new learning principle and implementation of the generative decomposition model generally known as noisy ICA (for independent component analysis) based on the SAEM algorithm, which is a versatile stochastic approximation of the standard EM algorithm. We demonstrate the applicability of the method on a large range of decomposition models and illustrate the developments with experimental results on various data sets.Comment: Published in at http://dx.doi.org/10.1214/11-AOAS499 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org