Joint Image and Depth Estimation With Mask-Based Lensless Cameras

Mask-based lensless cameras replace the lens of a conventional camera with a custom mask. These cameras can potentially be very thin and even flexible. Recently, it has been demonstrated that such mask-based cameras can recover light intensity and depth information of a scene. Existing depth recovery algorithms either assume that the scene consists of a small number of depth planes or solve a sparse recovery problem over a large 3D volume. Both these approaches fail to recover the scenes with large depth variations. In this paper, we propose a new approach for depth estimation based on an alternating gradient descent algorithm that jointly estimates a continuous depth map and light distribution of the unknown scene from its lensless measurements. We present simulation results on image and depth reconstruction for a variety of 3D test scenes. A comparison between the proposed algorithm and other method shows that our algorithm is more robust for natural scenes with a large range of depths. We built a prototype lensless camera and present experimental results for reconstruction of intensity and depth maps of different real objects

Sparse representation-based synthetic aperture radar imaging

There is increasing interest in using synthetic aperture radar (SAR) images in automated target recognition and decision-making tasks. The success of such tasks depends on how well the reconstructed SAR images exhibit certain features of the underlying scene. Based on the observation that typical underlying scenes usually exhibit sparsity in terms of such features, we develop an image formation method which formulates the SAR imaging problem as a sparse signal representation problem. Sparse signal representation, which has mostly been exploited in real-valued problems, has many capabilities such as superresolution and feature enhancement for various reconstruction and recognition tasks. However, for problems of complex-valued nature, such as SAR, a key challenge is how to choose the dictionary and the representation scheme for effective sparse representation. Since we are usually interested in features of the magnitude of the SAR reflectivity field, our new approach is designed to sparsely represent the magnitude of the complex-valued scattered field. This turns the image reconstruction problem into a joint optimization problem over the representation of magnitude and phase of the underlying field reflectivities. We develop the mathematical framework for this method and propose an iterative solution for the corresponding joint optimization problem. Our experimental results demonstrate the superiority of this method over previous approaches in terms of both producing high quality SAR images as well as exhibiting robustness to uncertain or limited data

Sparse representation-based SAR imaging

There is increasing interest in using synthetic aperture radar (SAR) images in automated target recognition and decision-making tasks. The success of such tasks depends on how well the reconstructed SAR images exhibit certain features of the underlying scene. Based on the observation that typical underlying scenes usually exhibit sparsity in terms of such features, we develop an image formation method which formulates the SAR imaging problem as a sparse signal representation problem. Sparse signal representation, which has mostly been exploited in real-valued problems, has many capabilities such as superresolution and feature enhancement for various reconstruction and recognition tasks. However, for problems of complex-valued nature, such as SAR, a key challenge is how to choose the dictionary and the representation scheme for effective sparse representation. Since we are usually interested in features of the magnitude of the SAR reflectivity field, our new approach is designed to sparsely represent the magnitude of the complex-valued scattered field. This turns the image reconstruction problem into a joint optimization problem over the representation of magnitude and phase of the underlying field reflectivities. We develop the mathematical framework for this method and propose an iterative solution for the corresponding joint optimization problem. Our experimental results demonstrate the superiority of this method over previous approaches in terms of both producing high quality SAR images as well as exhibiting robustness to uncertain or limited data

Robust Dense Mapping for Large-Scale Dynamic Environments

We present a stereo-based dense mapping algorithm for large-scale dynamic urban environments. In contrast to other existing methods, we simultaneously reconstruct the static background, the moving objects, and the potentially moving but currently stationary objects separately, which is desirable for high-level mobile robotic tasks such as path planning in crowded environments. We use both instance-aware semantic segmentation and sparse scene flow to classify objects as either background, moving, or potentially moving, thereby ensuring that the system is able to model objects with the potential to transition from static to dynamic, such as parked cars. Given camera poses estimated from visual odometry, both the background and the (potentially) moving objects are reconstructed separately by fusing the depth maps computed from the stereo input. In addition to visual odometry, sparse scene flow is also used to estimate the 3D motions of the detected moving objects, in order to reconstruct them accurately. A map pruning technique is further developed to improve reconstruction accuracy and reduce memory consumption, leading to increased scalability. We evaluate our system thoroughly on the well-known KITTI dataset. Our system is capable of running on a PC at approximately 2.5Hz, with the primary bottleneck being the instance-aware semantic segmentation, which is a limitation we hope to address in future work. The source code is available from the project website (http://andreibarsan.github.io/dynslam).Comment: Presented at IEEE International Conference on Robotics and Automation (ICRA), 201

Joint space aspect reconstruction of wide-angle SAR exploiting sparsity

In this paper we present an algorithm for wide-angle synthetic aperture radar (SAR) image formation. Reconstruction of wide-angle SAR holds a promise of higher resolution and better information about a scene, but it also poses a number of challenges when compared to the traditional narrow-angle SAR. Most prominently, the isotropic point scattering model is no longer valid. We present an algorithm capable of producing high resolution reflectivity maps in both space and aspect, thus accounting for the anisotropic scattering behavior of targets. We pose the problem as a non-parametric three-dimensional inversion problem, with two constraints: magnitudes of the backscattered power are highly correlated across closely spaced look angles and the backscattered power originates from a small set of point scatterers. This approach considers jointly all scatterers in the scene across all azimuths, and exploits the sparsity of the underlying scattering field. We implement the algorithm and present reconstruction results on realistic data obtained from the XPatch Backhoe dataset

Multiple feature-enhanced synthetic aperture radar imaging

Non-quadratic regularization based image formation is a recently proposed framework for feature-enhanced radar imaging. Specific image formation techniques in this framework have so far focused on enhancing one type of feature, such as strong point scatterers, or smooth regions. However, many scenes contain a number of such features. We develop an image formation technique that simultaneously enhances multiple types of features by posing the problem as one of sparse signal representation based on overcomplete dictionaries. Due to the complex-valued nature of the reflectivities in SAR, our new approach is designed to sparsely represent the magnitude of the complex-valued scattered field in terms of multiple features, which turns the image reconstruction problem into a joint optimization problem over the representation of the magnitude and the phase of the underlying field reflectivities. We formulate the mathematical framework needed for this method and propose an iterative solution for the corresponding joint optimization problem. We demonstrate the effectiveness of this approach on various SAR images