Probabilistic modeling and statistical inference for random fields and space-time processes

Author from publisher's list. Cover title.Final report for ONR Grant N00014-91-J-100

3D Shape Understanding and Generation

In recent years, Machine Learning techniques have revolutionized solutions to longstanding image-based problems, like image classification, generation, semantic segmentation, object detection and many others. However, if we want to be able to build agents that can successfully interact with the real world, those techniques need to be capable of reasoning about the world as it truly is: a tridimensional space. There are two main challenges while handling 3D information in machine learning models. First, it is not clear what is the best 3D representation. For images, convolutional neural networks (CNNs) operating on raster images yield the best results in virtually all image-based benchmarks. For 3D data, the best combination of model and representation is still an open question. Second, 3D data is not available on the same scale as images – taking pictures is a common procedure in our daily lives, whereas capturing 3D content is an activity usually restricted to specialized professionals. This thesis is focused on addressing both of these issues. Which model and representation should we use for generating and recognizing 3D data? What are efficient ways of learning 3D representations from a few examples? Is it possible to leverage image data to build models capable of reasoning about the world in 3D? Our research findings show that it is possible to build models that efficiently generate 3D shapes as irregularly structured representations. Those models require significantly less memory while generating higher quality shapes than the ones based on voxels and multi-view representations. We start by developing techniques to generate shapes represented as point clouds. This class of models leads to high quality reconstructions and better unsupervised feature learning. However, since point clouds are not amenable to editing and human manipulation, we also present models capable of generating shapes as sets of shape handles -- simpler primitives that summarize complex 3D shapes and were specifically designed for high-level tasks and user interaction. Despite their effectiveness, those approaches require some form of 3D supervision, which is scarce. We present multiple alternatives to this problem. First, we investigate how approximate convex decomposition techniques can be used as self-supervision to improve recognition models when only a limited number of labels are available. Second, we study how neural network architectures induce shape priors that can be used in multiple reconstruction tasks -- using both volumetric and manifold representations. In this regime, reconstruction is performed from a single example -- either a sparse point cloud or multiple silhouettes. Finally, we demonstrate how to train generative models of 3D shapes without using any 3D supervision by combining differentiable rendering techniques and Generative Adversarial Networks

Multiscale statistical methods for the segmentation of signals and images

Includes bibliographical references (p. 29-30).Supported by a National Science Foundation Graduate Fellowship, and by ONR. N00014-91-J-1004 Supported by AFOSR. F49620-95-1-0083 Supported by Boston University. GC123919NGN Supported by NIH. NINDS 1 R01 NS34189M.K. Schneider ... [et al.]

Error analysis of coarse-grained kinetic Monte Carlo method

In this paper we investigate the approximation properties of the coarse-graining procedure applied to kinetic Monte Carlo simulations of lattice stochastic dynamics. We provide both analytical and numerical evidence that the hierarchy of the coarse models is built in a systematic way that allows for error control in both transient and long-time simulations. We demonstrate that the numerical accuracy of the CGMC algorithm as an approximation of stochastic lattice spin flip dynamics is of order two in terms of the coarse-graining ratio and that the natural small parameter is the coarse-graining ratio over the range of particle/particle interactions. The error estimate is shown to hold in the weak convergence sense. We employ the derived analytical results to guide CGMC algorithms and we demonstrate a CPU speed-up in demanding computational regimes that involve nucleation, phase transitions and metastability.Comment: 30 page

Numerical Methods in Shape Spaces and Optimal Branching Patterns

The contribution of this thesis is twofold. The main part deals with numerical methods in the context of shape space analysis, where the shape space at hand is considered as a Riemannian manifold. In detail, we apply and extend the time-discrete geodesic calculus (established by Rumpf and Wirth [WBRS11, RW15]) to the space of discrete shells, i.e. triangular meshes with fixed connectivity. The essential building block is a variational time-discretization of geodesic curves, which is based on a local approximation of the squared Riemannian distance on the manifold. On physical shape spaces this approximation can be derived e.g. from a dissimilarity measure. The dissimilarity measure between two shell surfaces can naturally be defined as an elastic deformation energy capturing both membrane and bending distortions. Combined with a non-conforming discretization of a physically sound thin shell model the time-discrete geodesic calculus applied to the space of discrete shells is shown to be suitable to solve important problems in computer graphics and animation. To extend the existing calculus, we introduce a generalized spline functional based on the covariant derivative along a curve in shape space whose minimizers can be considered as Riemannian splines. We establish a corresponding time-discrete functional that fits perfectly into the framework of Rumpf and Wirth, and prove this discretization to be consistent. Several numerical simulations reveal that the optimization of the spline functional—subject to appropriate constraints—can be used to solve the multiple interpolation problem in shape space, e.g. to realize keyframe animation. Based on the spline functional, we further develop a simple regression model which generalizes linear regression to nonlinear shape spaces. Numerical examples based on real data from anatomy and botany show the capability of the model. Finally, we apply the statistical analysis of elastic shape spaces presented by Rumpf and Wirth [RW09, RW11] to the space of discrete shells. To this end, we compute a Fréchet mean within a class of shapes bearing highly nonlinear variations and perform a principal component analysis with respect to the metric induced by the Hessian of an elastic shell energy. The last part of this thesis deals with the optimization of microstructures arising e.g. at austenite-martensite interfaces in shape memory alloys. For a corresponding scalar problem, Kohn and Müller [KM92, KM94] proved existence of a minimizer and provided a lower and an upper bound for the optimal energy. To establish the upper bound, they studied a particular branching pattern generated by mixing two different martensite phases. We perform a finite element simulation based on subdivision surfaces that suggests a topologically different class of branching patterns to represent an optimal microstructure. Based on these observations we derive a novel, low dimensional family of patterns and show—numerically and analytically—that our new branching pattern results in a significantly better upper energy bound

Light field reconstruction from multi-view images

Kang Han studied recovering the 3D world from multi-view images. He proposed several algorithms to deal with occlusions in depth estimation and effective representations in view rendering. the proposed algorithms can be used for many innovative applications based on machine intelligence, such as autonomous driving and Metaverse

A New Pansharpening Approach for Hyperspectral Images

We first briefly review recent papers for pansharpening of hyperspectral (HS) images. We then present a recent pansharpening approach called hybrid color mapping (HCM). A few variants of HCM are then summarized. Using two hyperspectral images, we illustrate the advantages of HCM by comparing HCM with 10 state-of-the-art algorithms