Learning to Extract Motion from Videos in Convolutional Neural Networks

This paper shows how to extract dense optical flow from videos with a convolutional neural network (CNN). The proposed model constitutes a potential building block for deeper architectures to allow using motion without resorting to an external algorithm, \eg for recognition in videos. We derive our network architecture from signal processing principles to provide desired invariances to image contrast, phase and texture. We constrain weights within the network to enforce strict rotation invariance and substantially reduce the number of parameters to learn. We demonstrate end-to-end training on only 8 sequences of the Middlebury dataset, orders of magnitude less than competing CNN-based motion estimation methods, and obtain comparable performance to classical methods on the Middlebury benchmark. Importantly, our method outputs a distributed representation of motion that allows representing multiple, transparent motions, and dynamic textures. Our contributions on network design and rotation invariance offer insights nonspecific to motion estimation

Optical Flow in Mostly Rigid Scenes

The optical flow of natural scenes is a combination of the motion of the observer and the independent motion of objects. Existing algorithms typically focus on either recovering motion and structure under the assumption of a purely static world or optical flow for general unconstrained scenes. We combine these approaches in an optical flow algorithm that estimates an explicit segmentation of moving objects from appearance and physical constraints. In static regions we take advantage of strong constraints to jointly estimate the camera motion and the 3D structure of the scene over multiple frames. This allows us to also regularize the structure instead of the motion. Our formulation uses a Plane+Parallax framework, which works even under small baselines, and reduces the motion estimation to a one-dimensional search problem, resulting in more accurate estimation. In moving regions the flow is treated as unconstrained, and computed with an existing optical flow method. The resulting Mostly-Rigid Flow (MR-Flow) method achieves state-of-the-art results on both the MPI-Sintel and KITTI-2015 benchmarks.Comment: 15 pages, 10 figures; accepted for publication at CVPR 201

Laser Rayleigh and Raman Diagnostics for Small Hydrogen/oxygen Rockets

Localized velocity, temperature, and species concentration measurements in rocket flow fields are needed to evaluate predictive computational fluid dynamics (CFD) codes and identify causes of poor rocket performance. Velocity, temperature, and total number density information have been successfully extracted from spectrally resolved Rayleigh scattering in the plume of small hydrogen/oxygen rockets. Light from a narrow band laser is scattered from the moving molecules with a Doppler shifted frequency. Two components of the velocity can be extracted by observing the scattered light from two directions. Thermal broadening of the scattered light provides a measure of the temperature, while the integrated scattering intensity is proportional to the number density. Spontaneous Raman scattering has been used to measure temperature and species concentration in similar plumes. Light from a dye laser is scattered by molecules in the rocket plume. Raman spectra scattered from major species are resolved by observing the inelastically scattered light with linear array mounted to a spectrometer. Temperature and oxygen concentrations have been extracted by fitting a model function to the measured Raman spectrum. Results of measurements on small rockets mounted inside a high altitude chamber using both diagnostic techniques are reported

Understanding the nature of FRII optical nuclei: a new diagnostic plane for radio galaxies

We extend our study of the nuclei of 3CR FR II radio galaxies through HST optical images up to z=0.3. In the majority of them an unresolved nucleus (central compact core, CCC) is found. We analyze their position in the plane formed by the radio and optical nuclear luminosities in relation to their optical spectral properties. The broad-lined objects (BLO) have the brightest nuclei: they are present only at optical luminosities nu L_nu > 4 X 10^42 erg s^-1 which we suggest might represent a threshold in the radiative efficiency combined to a small range of black hole masses. About 40 % of the high and low excitation galaxies (HEG and LEG) show CCC which resemble those previously detected in FR I galaxies, in apparent contrast to the unification model. The equivalent width of the [OIII] emission line (with respect to the nuclear luminosity) reveals the nature of these nuclei, indicating that the nuclei of HEG are obscured to our line of sight and only scattered radiation is observed. This implies that the population of FR II is composed by objects with different nuclear properties, and only a fraction of them can be unified with quasars.Comment: 11 pages, 6 figures, in press on Astronomy & Astrophysics, minor changes have been mad

The diversity of quasars unified by accretion and orientation

Quasars are rapidly accreting supermassive black holes at the center of massive galaxies. They display a broad range of properties across all wavelengths, reflecting the diversity in the physical conditions of the regions close to the central engine. These properties, however, are not random, but form well-defined trends. The dominant trend is known as Eigenvector 1, where many properties correlate with the strength of optical iron and [OIII] emission. The main physical driver of Eigenvector 1 has long been suspected to be the quasar luminosity normalized by the mass of the hole (the Eddington ratio), an important quantity of the black hole accretion process. But a definitive proof has been missing. Here we report an analysis of archival data that reveals that Eddington ratio indeed drives Eigenvector 1. We also find that orientation plays a significant role in determining the observed kinematics of the gas, implying a flattened, disklike geometry for the fast-moving clouds close to the hole. Our results show that most of the diversity of quasar phenomenology can be unified with two simple quantities, Eddington ratio and orientation.Comment: This is the author's version of the work; 18 pages including Supplementary Information; to appear in the 11 September 2014 issue of Nature at http://dx.doi.org/10.1038/nature1371

Bessel beam illumination reduces random and systematic errors in quantitative functional studies using light-sheet microscopy

Light-sheet microscopy (LSM), in combination with intrinsically transparent zebrafish larvae, is a choice method to observe brain function with high frame rates at cellular resolution. Inherently to LSM, however, residual opaque objects cause stripe artifacts, which obscure features of interest and, during functional imaging, modulate fluorescence variations related to neuronal activity. Here, we report how Bessel beams reduce streaking artifacts and produce high-fidelity quantitative data demonstrating a fivefold increase in sensitivity to calcium transients and a 20 fold increase in accuracy in the detection of activity correlations in functional imaging. Furthermore, using principal component analysis, we show that measurements obtained with Bessel beams are clean enough to reveal in one-shot experiments correlations that can not be averaged over trials after stimuli as is the case when studying spontaneous activity. Our results not only demonstrate the contamination of data by systematic and random errors through conventional Gaussian illumination and but,furthermore, quantify the increase in fidelity of such data when using Bessel beams