Discrete and Continuous Optimization for Motion Estimation

The study of motion estimation reaches back decades and has become one of the central topics of research in computer vision. Even so, there are situations where current approaches fail, such as when there are extreme lighting variations, significant occlusions, or very large motions. In this thesis, we propose several approaches to address these issues. First, we propose a novel continuous optimization framework for estimating optical flow based on a decomposition of the image domain into triangular facets. We show how this allows for occlusions to be easily and naturally handled within our optimization framework without any post-processing. We also show that a triangular decomposition enables us to use a direct Cholesky decomposition to solve the resulting linear systems by reducing its memory requirements. Second, we introduce a simple method for incorporating additional temporal information into optical flow using inertial estimates of the flow, which leads to a significant reduction in error. We evaluate our methods on several datasets and achieve state-of-the-art results on MPI-Sintel. Finally, we introduce a discrete optimization framework for optical flow computation. Discrete approaches have generally been avoided in optical flow because of the relatively large label space that makes them computationally expensive. In our approach, we use recent advances in image segmentation to build a tree-structured graphical model that conforms to the image content. We show how the optimal solution to these discrete optical flow problems can be computed efficiently by making use of optimization methods from the object recognition literature, even for large images with hundreds of thousands of labels

On the Conductance Sum Rule for the Hierarchical Edge States of the Fractional Quantum Hall Effect

The conductance sum rule for the hierarchical edge channel currents of a Fractional Quantum Hall Effect state is derived analytically within the Haldane-Halperin hierarchy scheme. We provide also an intuitive interpretation for the hierarchical drift velocities of the edge excitations.Comment: 11 pages, no figure, Revtex 3.0, IC/93/329, ASITP-93-5

Dust Scattering In Turbulent Media: Correlation Between The Scattered Light and Dust Column Density

Radiative transfer models in a spherical, turbulent interstellar medium (ISM) in which the photon source is situated at the center are calculated to investigate the correlation between the scattered light and the dust column density. The medium is modeled using fractional Brownian motion structures that are appropriate for turbulent ISM. The correlation plot between the scattered light and optical depth shows substantial scatter and deviation from simple proportionality. It was also found that the overall density contrast is smoothed out in scattered light. In other words, there is an enhancement of the dust-scattered flux in low-density regions, while the scattered flux is suppressed in high-density regions. The correlation becomes less significant as the scattering becomes closer to be isotropic and the medium becomes more turbulent. Therefore, the scattered light observed in near-infrared wavelengths would show much weaker correlation than the observations in optical and ultraviolet wavelengths. We also find that the correlation plot between scattered lights at two different wavelengths shows a tighter correlation than that of the scattered light versus the optical depth.Comment: 6 pages, 5 figure, accepted for publication in the ApJ Letter

Joint coarse-and-fine reasoning for deep optical flow

© 20xx IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.We propose a novel representation for dense pixel-wise estimation tasks using CNNs that boosts accuracy and reduces training time, by explicitly exploiting joint coarse-and-fine reasoning. The coarse reasoning is performed over a discrete classification space to obtain a general rough solution, while the fine details of the solution are obtained over a continuous regression space. In our approach both components are jointly estimated, which proved to be beneficial for improving estimation accuracy. Additionally, we propose a new network architecture, which combines coarse and fine components by treating the fine estimation as a refinement built on top of the coarse solution, and therefore adding details to the general prediction. We apply our approach to the challenging problem of optical flow estimation and empirically validate it against state-of-the-art CNN-based solutions trained from scratch and tested on large optical flow datasets.Peer ReviewedPostprint (author's final draft

Sunyaev-Zel'dovich Effects from Quasars in Galaxies and Groups

The energy fed by active galactic nuclei to the surrounding diffuse baryons changes their amount, temperature, and distribution; so in groups and in member galaxies it affects the X-ray luminosity and also the Sunyaev-Zel'dovich effect. Here we compute how the latter is enhanced by the transient blastwave driven by an active quasar, and is depressed when the equilibrium is recovered with a depleted density. We constrain such depressions and enhancements with the masses of relic black holes in galaxies and the X-ray luminosities in groups. We discuss how all these linked observables can tell the quasar contribution to the thermal history of the baryons pervading galaxies and groups.Comment: 4 pages, 3 figures, uses REVTeX4 and emulateapj.cls. Accepted by ApJ

Beyond Short Snippets: Deep Networks for Video Classification

Convolutional neural networks (CNNs) have been extensively applied for image recognition problems giving state-of-the-art results on recognition, detection, segmentation and retrieval. In this work we propose and evaluate several deep neural network architectures to combine image information across a video over longer time periods than previously attempted. We propose two methods capable of handling full length videos. The first method explores various convolutional temporal feature pooling architectures, examining the various design choices which need to be made when adapting a CNN for this task. The second proposed method explicitly models the video as an ordered sequence of frames. For this purpose we employ a recurrent neural network that uses Long Short-Term Memory (LSTM) cells which are connected to the output of the underlying CNN. Our best networks exhibit significant performance improvements over previously published results on the Sports 1 million dataset (73.1% vs. 60.9%) and the UCF-101 datasets with (88.6% vs. 88.0%) and without additional optical flow information (82.6% vs. 72.8%)

Constraints on the bulk Lorentz factor in the internal shock scenario for gamma-ray bursts

We investigate, independently of specific emission models, the constraints on the value of the bulk Lorentz factor Gamma of a fireball. We assume that the burst emission comes from internal shocks in a region transparent to Thomson scattering and before deceleration due to the swept up external matter is effective. We consider the role of Compton drag in decelerating fast moving shells before they interact with slower ones, thus limiting the possible differences in bulk Lorentz factor of shells. Tighter constraints on the possible range of Gamma are derived by requiring that the internal shocks transform more than a few per cent of the bulk energy into radiation. Efficient bursts may require a hierarchical scenario, where a shell undergoes multiple interactions with other shells. We conclude that fireballs with average Lorentz factors larger than 1000 are unlikely to give rise to the observed bursts.Comment: 5 pages, 3 figures, accepted for publication in MNRAS, pink page

Optimization-based interactive segmentation interface for multiregion problems.

Interactive segmentation is becoming of increasing interest to the medical imaging community in that it combines the positive aspects of both manual and automated segmentation. However, general-purpose tools have been lacking in terms of segmenting multiple regions simultaneously with a high degree of coupling between groups of labels. Hierarchical max-flow segmentation has taken advantage of this coupling for individual applications, but until recently, these algorithms were constrained to a particular hierarchy and could not be considered general-purpose. In a generalized form, the hierarchy for any given segmentation problem is specified in run-time, allowing different hierarchies to be quickly explored. We present an interactive segmentation interface, which uses generalized hierarchical max-flow for optimization-based multiregion segmentation guided by user-defined seeds. Applications in cardiac and neonatal brain segmentation are given as example applications of its generality

Action Recognition in Videos: from Motion Capture Labs to the Web

This paper presents a survey of human action recognition approaches based on visual data recorded from a single video camera. We propose an organizing framework which puts in evidence the evolution of the area, with techniques moving from heavily constrained motion capture scenarios towards more challenging, realistic, "in the wild" videos. The proposed organization is based on the representation used as input for the recognition task, emphasizing the hypothesis assumed and thus, the constraints imposed on the type of video that each technique is able to address. Expliciting the hypothesis and constraints makes the framework particularly useful to select a method, given an application. Another advantage of the proposed organization is that it allows categorizing newest approaches seamlessly with traditional ones, while providing an insightful perspective of the evolution of the action recognition task up to now. That perspective is the basis for the discussion in the end of the paper, where we also present the main open issues in the area.Comment: Preprint submitted to CVIU, survey paper, 46 pages, 2 figures, 4 table