VConv-DAE: Deep Volumetric Shape Learning Without Object Labels

With the advent of affordable depth sensors, 3D capture becomes more and more ubiquitous and already has made its way into commercial products. Yet, capturing the geometry or complete shapes of everyday objects using scanning devices (e.g. Kinect) still comes with several challenges that result in noise or even incomplete shapes. Recent success in deep learning has shown how to learn complex shape distributions in a data-driven way from large scale 3D CAD Model collections and to utilize them for 3D processing on volumetric representations and thereby circumventing problems of topology and tessellation. Prior work has shown encouraging results on problems ranging from shape completion to recognition. We provide an analysis of such approaches and discover that training as well as the resulting representation are strongly and unnecessarily tied to the notion of object labels. Thus, we propose a full convolutional volumetric auto encoder that learns volumetric representation from noisy data by estimating the voxel occupancy grids. The proposed method outperforms prior work on challenging tasks like denoising and shape completion. We also show that the obtained deep embedding gives competitive performance when used for classification and promising results for shape interpolation

Evaluation of CNN-based Single-Image Depth Estimation Methods

While an increasing interest in deep models for single-image depth estimation methods can be observed, established schemes for their evaluation are still limited. We propose a set of novel quality criteria, allowing for a more detailed analysis by focusing on specific characteristics of depth maps. In particular, we address the preservation of edges and planar regions, depth consistency, and absolute distance accuracy. In order to employ these metrics to evaluate and compare state-of-the-art single-image depth estimation approaches, we provide a new high-quality RGB-D dataset. We used a DSLR camera together with a laser scanner to acquire high-resolution images and highly accurate depth maps. Experimental results show the validity of our proposed evaluation protocol

FPM: Fine Pose Parts-Based Model with 3D CAD Models

We introduce a novel approach to the problem of localizing objects in an image and estimating their fine-pose. Given exact CAD models, and a few real training images with aligned models, we propose to leverage the geometric information from CAD models and appearance information from real images to learn a model that can accurately estimate fine pose in real images. Specifically, we propose FPM, a fine pose parts-based model, that combines geometric information in the form of shared 3D parts in deformable part based models, and appearance information in the form of objectness to achieve both fast and accurate fine pose estimation. Our method significantly outperforms current state-of-the-art algorithms in both accuracy and speed

Depth Estimation via Affinity Learned with Convolutional Spatial Propagation Network

Depth estimation from a single image is a fundamental problem in computer vision. In this paper, we propose a simple yet effective convolutional spatial propagation network (CSPN) to learn the affinity matrix for depth prediction. Specifically, we adopt an efficient linear propagation model, where the propagation is performed with a manner of recurrent convolutional operation, and the affinity among neighboring pixels is learned through a deep convolutional neural network (CNN). We apply the designed CSPN to two depth estimation tasks given a single image: (1) To refine the depth output from state-of-the-art (SOTA) existing methods; and (2) to convert sparse depth samples to a dense depth map by embedding the depth samples within the propagation procedure. The second task is inspired by the availability of LIDARs that provides sparse but accurate depth measurements. We experimented the proposed CSPN over two popular benchmarks for depth estimation, i.e. NYU v2 and KITTI, where we show that our proposed approach improves in not only quality (e.g., 30% more reduction in depth error), but also speed (e.g., 2 to 5 times faster) than prior SOTA methods.Comment: 14 pages, 8 figures, ECCV 201

Assessing the Quality of Actions

While recent advances in computer vision have provided reliable methods to recognize actions in both images and videos, the problem of assessing how well people perform actions has been largely unexplored in computer vision. Since methods for assessing action quality have many real-world applications in healthcare, sports, and video retrieval, we believe the computer vision community should begin to tackle this challenging problem. To spur progress, we introduce a learning-based framework that takes steps towards assessing how well people perform actions in videos. Our approach works by training a regression model from spatiotemporal pose features to scores obtained from expert judges. Moreover, our approach can provide interpretable feedback on how people can improve their action. We evaluate our method on a new Olympic sports dataset, and our experiments suggest our framework is able to rank the athletes more accurately than a non-expert human. While promising, our method is still a long way to rivaling the performance of expert judges, indicating that there is significant opportunity in computer vision research to improve on this difficult yet important task.National Science Foundation (U.S.). Graduate Research FellowshipGoogle (Firm) (Research Award)United States. Office of Naval Research. Multidisciplinary University Research Initiative (N000141010933

Cosmological Density and Power Spectrum from Peculiar Velocities: Nonlinear Corrections and PCA

We allow for nonlinear effects in the likelihood analysis of galaxy peculiar velocities, and obtain ~35%-lower values for the cosmological density parameter Om and the amplitude of mass-density fluctuations. The power spectrum in the linear regime is assumed to be a flat LCDM model (h=0.65, n=1, COBE) with only Om as a free parameter. Since the likelihood is driven by the nonlinear regime, we "break" the power spectrum at k_b=0.2 h/Mpc and fit a power law at k>k_b. This allows for independent matching of the nonlinear behavior and an unbiased fit in the linear regime. The analysis assumes Gaussian fluctuations and errors, and a linear relation between velocity and density. Tests using proper mock catalogs demonstrate a reduced bias and a better fit. We find for the Mark3 and SFI data Om_m=0.32+-0.06 and 0.37+-0.09 respectively, with sigma_8*Om^0.6 = 0.49+-0.06 and 0.63+-0.08, in agreement with constraints from other data. The quoted 90% errors include cosmic variance. The improvement in likelihood due to the nonlinear correction is very significant for Mark3 and moderately so for SFI. When allowing deviations from LCDM, we find an indication for a wiggle in the power spectrum: an excess near k=0.05 and a deficiency at k=0.1 (cold flow). This may be related to the wiggle seen in the power spectrum from redshift surveys and the second peak in the CMB anisotropy. A chi^2 test applied to modes of a Principal Component Analysis (PCA) shows that the nonlinear procedure improves the goodness of fit and reduces a spatial gradient of concern in the linear analysis. The PCA allows addressing spatial features of the data and fine-tuning the theoretical and error models. It shows that the models used are appropriate for the cosmological parameter estimation performed. We address the potential for optimal data compression using PCA.Comment: 18 pages, LaTex, uses emulateapj.sty, ApJ in press (August 10, 2001), improvements to text and figures, updated reference

iPose: Instance-Aware 6D Pose Estimation of Partly Occluded Objects

We address the task of 6D pose estimation of known rigid objects from single input images in scenarios where the objects are partly occluded. Recent RGB-D-based methods are robust to moderate degrees of occlusion. For RGB inputs, no previous method works well for partly occluded objects. Our main contribution is to present the first deep learning-based system that estimates accurate poses for partly occluded objects from RGB-D and RGB input. We achieve this with a new instance-aware pipeline that decomposes 6D object pose estimation into a sequence of simpler steps, where each step removes specific aspects of the problem. The first step localizes all known objects in the image using an instance segmentation network, and hence eliminates surrounding clutter and occluders. The second step densely maps pixels to 3D object surface positions, so called object coordinates, using an encoder-decoder network, and hence eliminates object appearance. The third, and final, step predicts the 6D pose using geometric optimization. We demonstrate that we significantly outperform the state-of-the-art for pose estimation of partly occluded objects for both RGB and RGB-D input

Estimating Depth from RGB and Sparse Sensing

We present a deep model that can accurately produce dense depth maps given an RGB image with known depth at a very sparse set of pixels. The model works simultaneously for both indoor/outdoor scenes and produces state-of-the-art dense depth maps at nearly real-time speeds on both the NYUv2 and KITTI datasets. We surpass the state-of-the-art for monocular depth estimation even with depth values for only 1 out of every ~10000 image pixels, and we outperform other sparse-to-dense depth methods at all sparsity levels. With depth values for 1/256 of the image pixels, we achieve a mean absolute error of less than 1% of actual depth on indoor scenes, comparable to the performance of consumer-grade depth sensor hardware. Our experiments demonstrate that it would indeed be possible to efficiently transform sparse depth measurements obtained using e.g. lower-power depth sensors or SLAM systems into high-quality dense depth maps.Comment: European Conference on Computer Vision (ECCV) 2018. Updated to camera-ready version with additional experiment

The devil is in the decoder

Many machine vision applications require predictions for every pixel of the input image (for example semantic segmentation, boundary detection). Models for such problems usually consist of encoders which decreases spatial resolution while learning a high-dimensional representation, followed by decoders who recover the original input resolution and result in low-dimensional predictions. While encoders have been studied rigorously, relatively few studies address the decoder side. Therefore this paper presents an extensive comparison of a variety of decoders for a variety of pixel-wise prediction tasks. Our contributions are: (1) Decoders matter: we observe significant variance in results between different types of decoders on various problems. (2) We introduce a novel decoder: bilinear additive upsampling. (3) We introduce new residual-like connections for decoders. (4) We identify two decoder types which give a consistently high performance

Some open questions in "wave chaos"

The subject area referred to as "wave chaos", "quantum chaos" or "quantum chaology" has been investigated mostly by the theoretical physics community in the last 30 years. The questions it raises have more recently also attracted the attention of mathematicians and mathematical physicists, due to connections with number theory, graph theory, Riemannian, hyperbolic or complex geometry, classical dynamical systems, probability etc. After giving a rough account on "what is quantum chaos?", I intend to list some pending questions, some of them having been raised a long time ago, some others more recent