Real-time Halfway Domain Reconstruction of Motion and Geometry

We present a novel approach for real-time joint reconstruction of 3D scene motion and geometry from binocular stereo videos. Our approach is based on a novel variational halfway-domain scene flow formulation, which allows us to obtain highly accurate spatiotemporal reconstructions of shape and motion. We solve the underlying optimization problem at real-time frame rates using a novel data-parallel robust non-linear optimization strategy. Fast convergence and large displacement flows are achieved by employing a novel hierarchy that stores delta flows between hierarchy levels. High performance is obtained by the introduction of a coarser warp grid that decouples the number of unknowns from the input resolution of the images. We demonstrate our approach in a live setup that is based on two commodity webcams, as well as on publicly available video data. Our extensive experiments and evaluations show that our approach produces high-quality dense reconstructions of 3D geometry and scene flow at real-time frame rates, and compares favorably to the state of the art

Real-time Halfway Domain Reconstruction of Motion and Geometry

We present a novel approach for real-time joint reconstruction of 3D scene motion and geometry from binocular stereo videos. Our approach is based on a novel variational halfway-domain scene flow formulation, which allows us to obtain highly accurate spatiotemporal reconstructions of shape and motion. We solve the underlying optimization problem at real-time frame rates using a novel data-parallel robust non-linear optimization strategy. Fast convergence and large displacement flows are achieved by employing a novel hierarchy that stores delta flows between hierarchy levels. High performance is obtained by the introduction of a coarser warp grid that decouples the number of unknowns from the input resolution of the images. We demonstrate our approach in a live setup that is based on two commodity webcams, as well as on publicly available video data. Our extensive experiments and evaluations show that our approach produces high-quality dense reconstructions of 3D geometry and scene flow at real-time frame rates, and compares favorably to the state of the art.Comment: Proc. of the International Conference on 3D Vision 2016 (3DV 2016

Multiscale Representations for Manifold-Valued Data

We describe multiscale representations for data observed on equispaced grids and taking values in manifolds such as the sphere $S^2$, the special orthogonal group $SO(3)$, the positive definite matrices $SPD(n)$, and the Grassmann manifolds $G(n,k)$. The representations are based on the deployment of Deslauriers--Dubuc and average-interpolating pyramids "in the tangent plane" of such manifolds, using the $Exp$ and $Log$ maps of those manifolds. The representations provide "wavelet coefficients" which can be thresholded, quantized, and scaled in much the same way as traditional wavelet coefficients. Tasks such as compression, noise removal, contrast enhancement, and stochastic simulation are facilitated by this representation. The approach applies to general manifolds but is particularly suited to the manifolds we consider, i.e., Riemannian symmetric spaces, such as $S^{n-1}$, $SO(n)$, $G(n,k)$, where the $Exp$ and $Log$ maps are effectively computable. Applications to manifold-valued data sources of a geometric nature (motion, orientation, diffusion) seem particularly immediate. A software toolbox, SymmLab, can reproduce the results discussed in this paper

Koopman analysis of the long-term evolution in a turbulent convection cell

We analyse the long-time evolution of the three-dimensional flow in a closed cubic turbulent Rayleigh-B\'{e}nard convection cell via a Koopman eigenfunction analysis. A data-driven basis derived from diffusion kernels known in machine learning is employed here to represent a regularized generator of the unitary Koopman group in the sense of a Galerkin approximation. The resulting Koopman eigenfunctions can be grouped into subsets in accordance with the discrete symmetries in a cubic box. In particular, a projection of the velocity field onto the first group of eigenfunctions reveals the four stable large-scale circulation (LSC) states in the convection cell. We recapture the preferential circulation rolls in diagonal corners and the short-term switching through roll states parallel to the side faces which have also been seen in other simulations and experiments. The diagonal macroscopic flow states can last as long as a thousand convective free-fall time units. In addition, we find that specific pairs of Koopman eigenfunctions in the secondary subset obey enhanced oscillatory fluctuations for particular stable diagonal states of the LSC. The corresponding velocity field structures, such as corner vortices and swirls in the midplane, are also discussed via spatiotemporal reconstructions.Comment: 32 pages, 9 figures, article in press at Journal of Fluid Mechanic

A Demonstration of Spectral and Spatial Interferometry at THz Frequencies

A laboratory prototype spectral/spatial interferometer has been constructed to demonstrate the feasibility of the double Fourier technique at Far Infrared (FIR) wavelengths (0.15 - 1 THz). It is planned to use this demonstrator to investigate and validate important design features and data processing methods for future astronomical FIR interferometer instruments. In building this prototype we have had to address several key technologies to provide an end-end system demonstration of this double Fourier interferometer. We report on the first results taken when viewing single slit and double slit sources at the focus of a large collimator used to simulate real sources at infinity. The performance of the prototype instrument for these specific field geometries is analyzed to compare with the observed interferometric fringes and to demonstrate image reconstruction capabilities.Comment: Accepted for publication in Applied Optic

Compression, Modeling, and Real-Time Rendering of Realistic Materials and Objects

The realism of a scene basically depends on the quality of the geometry, the illumination and the materials that are used. Whereas many sources for the creation of three-dimensional geometry exist and numerous algorithms for the approximation of global illumination were presented, the acquisition and rendering of realistic materials remains a challenging problem. Realistic materials are very important in computer graphics, because they describe the reflectance properties of surfaces, which are based on the interaction of light and matter. In the real world, an enormous diversity of materials can be found, comprising very different properties. One important objective in computer graphics is to understand these processes, to formalize them and to finally simulate them. For this purpose various analytical models do already exist, but their parameterization remains difficult as the number of parameters is usually very high. Also, they fail for very complex materials that occur in the real world. Measured materials, on the other hand, are prone to long acquisition time and to huge input data size. Although very efficient statistical compression algorithms were presented, most of them do not allow for editability, such as altering the diffuse color or mesostructure. In this thesis, a material representation is introduced that makes it possible to edit these features. This makes it possible to re-use the acquisition results in order to easily and quickly create deviations of the original material. These deviations may be subtle, but also substantial, allowing for a wide spectrum of material appearances. The approach presented in this thesis is not based on compression, but on a decomposition of the surface into several materials with different reflection properties. Based on a microfacette model, the light-matter interaction is represented by a function that can be stored in an ordinary two-dimensional texture. Additionally, depth information, local rotations, and the diffuse color are stored in these textures. As a result of the decomposition, some of the original information is inevitably lost, therefore an algorithm for the efficient simulation of subsurface scattering is presented as well. Another contribution of this work is a novel perception-based simplification metric that includes the material of an object. This metric comprises features of the human visual system, for example trichromatic color perception or reduced resolution. The proposed metric allows for a more aggressive simplification in regions where geometric metrics do not simplif

TESS: A Relativistic Hydrodynamics Code on a Moving Voronoi Mesh

We have generalized a method for the numerical solution of hyperbolic systems of equations using a dynamic Voronoi tessellation of the computational domain. The Voronoi tessellation is used to generate moving computational meshes for the solution of multi-dimensional systems of conservation laws in finite-volume form. The mesh generating points are free to move with arbitrary velocity, with the choice of zero velocity resulting in an Eulerian formulation. Moving the points at the local fluid velocity makes the formulation effectively Lagrangian. We have written the TESS code to solve the equations of compressible hydrodynamics and magnetohydrodynamics for both relativistic and non-relativistic fluids on a dynamic Voronoi mesh. When run in Lagrangian mode, TESS is significantly less diffusive than fixed mesh codes and thus preserves contact discontinuities to high precision while also accurately capturing strong shock waves. TESS is written for Cartesian, spherical and cylindrical coordinates and is modular so that auxilliary physics solvers are readily integrated into the TESS framework and so that the TESS framework can be readily adapted to solve general systems of equations. We present results from a series of test problems to demonstrate the performance of TESS and to highlight some of the advantages of the dynamic tessellation method for solving challenging problems in astrophysical fluid dynamics.Comment: ApJS, 197, 1

Formalization of the General Video Temporal Synchronization Problem

In this work, we present a theoretical formalization of the temporal synchronization problem and a method to temporally synchronize multiple stationary video cameras with overlapping views of the same scene. The method uses a two stage approach that first approximates the synchronization by tracking moving objects and identifying curvature points. The method then proceeds to refine the estimate using a consensus based matching heuristic to find frames that best agree with the pre-computed camera geometries from stationary background image features. By using the fundamental matrix and the trifocal tensor in the second refinement step, we improve the estimation of the first step and handle a broader more generic range of input scenarios and camera conditions. The method is relatively simple compared to current techniques and is no harder than feature tracking in stage one and computing accurate geometries in stage two. We also provide a robust method to assist synchronization in the presence of inaccurate geometry computation, and a theoretical limit on the accuracy that can be expected from any synchronization syste