Learning to Reconstruct Shapes from Unseen Classes

From a single image, humans are able to perceive the full 3D shape of an object by exploiting learned shape priors from everyday life. Contemporary single-image 3D reconstruction algorithms aim to solve this task in a similar fashion, but often end up with priors that are highly biased by training classes. Here we present an algorithm, Generalizable Reconstruction (GenRe), designed to capture more generic, class-agnostic shape priors. We achieve this with an inference network and training procedure that combine 2.5D representations of visible surfaces (depth and silhouette), spherical shape representations of both visible and non-visible surfaces, and 3D voxel-based representations, in a principled manner that exploits the causal structure of how 3D shapes give rise to 2D images. Experiments demonstrate that GenRe performs well on single-view shape reconstruction, and generalizes to diverse novel objects from categories not seen during training.Comment: NeurIPS 2018 (Oral). The first two authors contributed equally to this paper. Project page: http://genre.csail.mit.edu

Second-order Shape Optimization for Geometric Inverse Problems in Vision

We develop a method for optimization in shape spaces, i.e., sets of surfaces modulo re-parametrization. Unlike previously proposed gradient flows, we achieve superlinear convergence rates through a subtle approximation of the shape Hessian, which is generally hard to compute and suffers from a series of degeneracies. Our analysis highlights the role of mean curvature motion in comparison with first-order schemes: instead of surface area, our approach penalizes deformation, either by its Dirichlet energy or total variation. Latter regularizer sparks the development of an alternating direction method of multipliers on triangular meshes. Therein, a conjugate-gradients solver enables us to bypass formation of the Gaussian normal equations appearing in the course of the overall optimization. We combine all of the aforementioned ideas in a versatile geometric variation-regularized Levenberg-Marquardt-type method applicable to a variety of shape functionals, depending on intrinsic properties of the surface such as normal field and curvature as well as its embedding into space. Promising experimental results are reported

New Image Processing Methods for Ultrasound Musculoskeletal Applications

In the past few years, ultrasound (US) imaging modalities have received increasing interest as diagnostic tools for orthopedic applications. The goal for many of these novel ultrasonic methods is to be able to create three-dimensional (3D) bone visualization non-invasively, safely and with high accuracy and spatial resolution. Availability of accurate bone segmentation and 3D reconstruction methods would help correctly interpreting complex bone morphology as well as facilitate quantitative analysis. However, in vivo ultrasound images of bones may have poor quality due to uncontrollable motion, high ultrasonic attenuation and the presence of imaging artifacts, which can affect the quality of the bone segmentation and reconstruction results. In this study, we investigate the use of novel ultrasonic processing methods that can significantly improve bone visualization, segmentation and 3D reconstruction in ultrasound volumetric data acquired in applications in vivo. Specifically, in this study, we investigate the use of new elastography-based, Doppler-based and statistical shape model-based methods that can be applied to ultrasound bone imaging applications with the overall major goal of obtaining fast yet accurate 3D bone reconstructions. This study is composed to three projects, which all have the potential to significantly contribute to this major goal. The first project deals with the fast and accurate implementation of correlation-based elastography and poroelastography techniques for real-time assessment of the mechanical properties of musculoskeletal tissues. The rationale behind this project is that, iii in the future, elastography-based features can be used to reduce false positives in ultrasonic bone segmentation methods based on the differences between the mechanical properties of soft tissues and the mechanical properties of hard tissues. In this study, a hybrid computation model is designed, implemented and tested to achieve real time performance without compromise in elastographic image quality . In the second project, a Power Doppler-based signal enhancement method is designed and tested with the intent of increasing the contrast between soft tissue and bone while suppressing the contrast between soft tissue and connective tissue, which is often a cause of false positives in ultrasonic bone segmentation problems. Both in-vitro and in-vivo experiments are performed to statistically analyze the performance of this method. In the third project, a statistical shape model based bone surface segmentation method is proposed and investigated. This method uses statistical models to determine if a curve detected in a segmented ultrasound image belongs to a bone surface or not. Both in-vitro and in-vivo experiments are performed to statistically analyze the performance of this method. I conclude this Dissertation with a discussion on possible future work in the field of ultrasound bone imaging and assessment

Symmetry Shape Prior for Object Segmentation

Symmetry is a useful segmentation cue. We develop an algorithm for segmenting a single symmetric object from the background. Our algorithm is formulated in the principled global optimization framework. Thus we can incorporate all the useful segmentation cues in the global energy function, in addition to the symmetry shape prior. We use the standard cues of regular boundary and coherent object (background) appearance. Our algorithm consists of two stages. The first stage, based on seam carving, detects a set of symmetry axis candidates. Symmetry axis is detected by first finding image “seams” that are aligned with intensity gradients and then matching them based on pairwise symmetry. The second stage formulates symmetric object segmentation in discrete optimization framework. We choose the longest symmetry axis as the object axis. Object symmetry is encouraged through submodular long-range pairwise terms. These pairwise terms are submodular, so optimization with a graph cut is applicable. We demonstrate the effectiveness of symmetry cue on a new symmetric object dataset

Geometric spin Hall effect of light in tightly focused polarization tailored light beams

Recently, it was shown that a non-zero transverse angular momentum manifests itself in a polarization dependent intensity shift of the barycenter of a paraxial light beam [A. Aiello et al., Phys. Rev. Lett. 103, 100401 (2009)]. The underlying effect is phenomenologically similar to the spin Hall effect of light, but does not depend on the specific light-matter interaction and can be interpreted as a purely geometric effect. Thus, it was named the geometric spin Hall effect of light. Here, we experimentally investigate the appearance of this effect in tightly focused vector-beams. We use an experimental nano-probing technique in combination with a reconstruction algorithm to verify the relative shifts of the components of the electric energy density in the focal plane, which are linked to the intensity shift. By that, we experimentally demonstrate the geometric spin Hall effect of light in a focused light beam.Comment: 8 pages, 5 figure

Physics-based Simulation Models for EBSD: Advances and Challenges

EBSD has evolved into an effective tool for microstructure investigations in the scanning electron microscope. The purpose of this contribution is to give an overview of various simulation approaches for EBSD Kikuchi patterns and to discuss some of the underlying physical mechanisms