Weakly supervised 3D Reconstruction with Adversarial Constraint

Supervised 3D reconstruction has witnessed a significant progress through the use of deep neural networks. However, this increase in performance requires large scale annotations of 2D/3D data. In this paper, we explore inexpensive 2D supervision as an alternative for expensive 3D CAD annotation. Specifically, we use foreground masks as weak supervision through a raytrace pooling layer that enables perspective projection and backpropagation. Additionally, since the 3D reconstruction from masks is an ill posed problem, we propose to constrain the 3D reconstruction to the manifold of unlabeled realistic 3D shapes that match mask observations. We demonstrate that learning a log-barrier solution to this constrained optimization problem resembles the GAN objective, enabling the use of existing tools for training GANs. We evaluate and analyze the manifold constrained reconstruction on various datasets for single and multi-view reconstruction of both synthetic and real images

Multi-View Face Recognition From Single RGBD Models of the Faces

This work takes important steps towards solving the following problem of current interest: Assuming that each individual in a population can be modeled by a single frontal RGBD face image, is it possible to carry out face recognition for such a population using multiple 2D images captured from arbitrary viewpoints? Although the general problem as stated above is extremely challenging, it encompasses subproblems that can be addressed today. The subproblems addressed in this work relate to: (1) Generating a large set of viewpoint dependent face images from a single RGBD frontal image for each individual; (2) using hierarchical approaches based on view-partitioned subspaces to represent the training data; and (3) based on these hierarchical approaches, using a weighted voting algorithm to integrate the evidence collected from multiple images of the same face as recorded from different viewpoints. We evaluate our methods on three datasets: a dataset of 10 people that we created and two publicly available datasets which include a total of 48 people. In addition to providing important insights into the nature of this problem, our results show that we are able to successfully recognize faces with accuracies of 95% or higher, outperforming existing state-of-the-art face recognition approaches based on deep convolutional neural networks

Observability/Identifiability of Rigid Motion under Perspective Projection

The "visual motion" problem consists of estimating the motion of an object viewed under projection. In this paper we address the feasibility of such a problem. We will show that the model which defines the visual motion problem for feature points in the euclidean 3D space lacks of both linear and local (weak) observability. The locally observable manifold is covered with three levels of lie differentiations. Indeed, by imposing metric constraints on the state-space, it is possible to reduce the set of indistinguishable states. We will then analyze a model for visual motion estimation in terms of identification of an Exterior Differential System, with the parameters living on a topological manifold, called the "essential manifold", which includes explicitly in its definition the forementioned metric constraints. We will show that rigid motion is globally observable/identifiable under perspective projection with zero level of lie differentiation under some general position conditions. Such conditions hold when the viewer does not move on a quadric surface containing all the visible points

Identifying manifolds underlying group motion in Vicsek agents

Collective motion of animal groups often undergoes changes due to perturbations. In a topological sense, we describe these changes as switching between low-dimensional embedding manifolds underlying a group of evolving agents. To characterize such manifolds, first we introduce a simple mapping of agents between time-steps. Then, we construct a novel metric which is susceptible to variations in the collective motion, thus revealing distinct underlying manifolds. The method is validated through three sample scenarios simulated using a Vicsek model, namely switching of speed, coordination, and structure of a group. Combined with a dimensionality reduction technique that is used to infer the dimensionality of the embedding manifold, this approach provides an effective model-free framework for the analysis of collective behavior across animal species.Comment: 12 pages, 6 figures, journal articl