Structured Light-Based 3D Reconstruction System for Plants.

Camera-based 3D reconstruction of physical objects is one of the most popular computer vision trends in recent years. Many systems have been built to model different real-world subjects, but there is lack of a completely robust system for plants. This paper presents a full 3D reconstruction system that incorporates both hardware structures (including the proposed structured light system to enhance textures on object surfaces) and software algorithms (including the proposed 3D point cloud registration and plant feature measurement). This paper demonstrates the ability to produce 3D models of whole plants created from multiple pairs of stereo images taken at different viewing angles, without the need to destructively cut away any parts of a plant. The ability to accurately predict phenotyping features, such as the number of leaves, plant height, leaf size and internode distances, is also demonstrated. Experimental results show that, for plants having a range of leaf sizes and a distance between leaves appropriate for the hardware design, the algorithms successfully predict phenotyping features in the target crops, with a recall of 0.97 and a precision of 0.89 for leaf detection and less than a 13-mm error for plant size, leaf size and internode distance

Depth Super-Resolution Meets Uncalibrated Photometric Stereo

A novel depth super-resolution approach for RGB-D sensors is presented. It disambiguates depth super-resolution through high-resolution photometric clues and, symmetrically, it disambiguates uncalibrated photometric stereo through low-resolution depth cues. To this end, an RGB-D sequence is acquired from the same viewing angle, while illuminating the scene from various uncalibrated directions. This sequence is handled by a variational framework which fits high-resolution shape and reflectance, as well as lighting, to both the low-resolution depth measurements and the high-resolution RGB ones. The key novelty consists in a new PDE-based photometric stereo regularizer which implicitly ensures surface regularity. This allows to carry out depth super-resolution in a purely data-driven manner, without the need for any ad-hoc prior or material calibration. Real-world experiments are carried out using an out-of-the-box RGB-D sensor and a hand-held LED light source.Comment: International Conference on Computer Vision (ICCV) Workshop, 201

Feature Extraction for image super-resolution using finite rate of innovation principles

To understand a real-world scene from several multiview pictures, it is necessary to find the disparities existing between each pair of images so that they are correctly related to one another. This process, called image registration, requires the extraction of some specific information about the scene. This is achieved by taking features out of the acquired images. Thus, the quality of the registration depends largely on the accuracy of the extracted features. Feature extraction can be formulated as a sampling problem for which perfect re- construction of the desired features is wanted. The recent sampling theory for signals with finite rate of innovation (FRI) and the B-spline theory offer an appropriate new frame- work for the extraction of features in real images. This thesis first focuses on extending the sampling theory for FRI signals to a multichannel case and then presents exact sampling results for two different types of image features used for registration: moments and edges. In the first part, it is shown that the geometric moments of an observed scene can be retrieved exactly from sampled images and used as global features for registration. The second part describes how edges can also be retrieved perfectly from sampled images for registration purposes. The proposed feature extraction schemes therefore allow in theory the exact registration of images. Indeed, various simulations show that the proposed extraction/registration methods overcome traditional ones, especially at low-resolution. These characteristics make such feature extraction techniques very appropriate for applications like image super-resolution for which a very precise registration is needed. The quality of the super-resolved images obtained using the proposed feature extraction meth- ods is improved by comparison with other approaches. Finally, the notion of polyphase components is used to adapt the image acquisition model to the characteristics of real digital cameras in order to run super-resolution experiments on real images

High Resolution 3D Shape Texture from Multiple Videos

International audienceWe examine the problem of retrieving high resolution textures of objects observed in multiple videos under small object deformations. In the monocular case, the data redundancy necessary to reconstruct a high-resolution image stems from temporal accumulation. This has been vastly explored and is known as super-resolution. On the other hand, a handful of methods have considered the texture of a static 3D object observed from several cameras, where the data redundancy is obtained through the different viewpoints. We introduce a unified framework to leverage both possibilities for the estimation of a high resolution texture of an object. This framework uniformly deals with any related geometric variability introduced by the acquisition chain or by the evolution over time. To this goal we use 2D warps for all viewpoints and all temporal frames and a linear projection model from texture to image space. Despite its simplicity, the method is able to successfully handle different views over space and time. As shown experimentally, it demonstrates the interest of temporal information that improves the texture quality. Additionally, we also show that our method outperforms state of the art multi-view super-resolution methods that exist for the static case

Weakly supervised 3D Reconstruction with Adversarial Constraint

Supervised 3D reconstruction has witnessed a significant progress through the use of deep neural networks. However, this increase in performance requires large scale annotations of 2D/3D data. In this paper, we explore inexpensive 2D supervision as an alternative for expensive 3D CAD annotation. Specifically, we use foreground masks as weak supervision through a raytrace pooling layer that enables perspective projection and backpropagation. Additionally, since the 3D reconstruction from masks is an ill posed problem, we propose to constrain the 3D reconstruction to the manifold of unlabeled realistic 3D shapes that match mask observations. We demonstrate that learning a log-barrier solution to this constrained optimization problem resembles the GAN objective, enabling the use of existing tools for training GANs. We evaluate and analyze the manifold constrained reconstruction on various datasets for single and multi-view reconstruction of both synthetic and real images