Cubic-Spline Flows

A normalizing flow models a complex probability density as an invertible transformation of a simple density. The invertibility means that we can evaluate densities and generate samples from a flow. In practice, autoregressive flow-based models are slow to invert, making either density estimation or sample generation slow. Flows based on coupling transforms are fast for both tasks, but have previously performed less well at density estimation than autoregressive flows. We stack a new coupling transform, based on monotonic cubic splines, with LU-decomposed linear layers. The resulting cubic-spline flow retains an exact one-pass inverse, can be used to generate high-quality images, and closes the gap with autoregressive flows on a suite of density-estimation tasks.Comment: Appeared at the 1st Workshop on Invertible Neural Networks and Normalizing Flows at ICML 201

Neural Spline Flows

A normalizing flow models a complex probability density as an invertible transformation of a simple base density. Flows based on either coupling or autoregressive transforms both offer exact density evaluation and sampling, but rely on the parameterization of an easily invertible elementwise transformation, whose choice determines the flexibility of these models. Building upon recent work, we propose a fully-differentiable module based on monotonic rational-quadratic splines, which enhances the flexibility of both coupling and autoregressive transforms while retaining analytic invertibility. We demonstrate that neural spline flows improve density estimation, variational inference, and generative modeling of images.Comment: Published at the 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canad

Moment-Based Order-Independent Transparency

Compositing transparent surfaces rendered in an arbitrary order requires techniques for order-independent transparency. Each surface color needs to be multiplied by the appropriate transmittance to the eye to incorporate occlusion. Building upon moment shadow mapping, we present a moment-based method for compact storage and fast reconstruction of this depth-dependent function per pixel. We work with the logarithm of the transmittance such that the function may be accumulated additively rather than multiplicatively. Then an additive rendering pass for all transparent surfaces yields moments. Moment-based reconstruction algorithms provide approximations to the original function, which are used for compositing in a second additive pass. We utilize existing algorithms with four or six power moments and develop new algorithms using eight power moments or up to four trigonometric moments. The resulting techniques are completely order-independent, work well for participating media as well as transparent surfaces and come in many variants providing different tradeoffs. We also utilize the same approach for the closely related problem of computing shadows for transparent surfaces

Solving the nearest rotation matrix problem in three and four dimensions with applications in robotics

Aplicat embargament des de la data de defensa fins ei 31/5/2022Since the map from quaternions to rotation matrices is a 2-to-1 covering map, this map cannot be smoothly inverted. As a consequence, it is sometimes erroneously assumed that all inversions should necessarily contain singularities that arise in the form of quotients where the divisor can be arbitrarily small. This misconception was clarified when we found a new division-free conversion method. This result triggered the research work presented in this thesis. At first glance, the matrix to quaternion conversion does not seem to be a relevant problem. Actually, most researchers consider it as a well-solved problem whose revision is not likely to provide any new insight in any area of practical interest. Nevertheless, we show in this thesis how solving the nearest rotation matrix problem in Frobenius norm can be reduced to a matrix to quaternion conversion. Many problems, such as hand-eye calibration, camera pose estimation, location recognition, image stitching etc. require finding the nearest proper orthogonal matrix to a given matrix. Thus, the matrix to quaternion conversion becomes of paramount importance. While a rotation in 3D can be represented using a quaternion, a rotation in 4D can be represented using a double quaternion. As a consequence, the computation of the nearest rotation matrix in 4D, using our approach, essentially follow the same steps as in the 3D case. Although the 4D case might seem of theoretical interest only, we show in this thesis its practical relevance thanks to a little known mapping between 3D displacements and 4D rotations. In this thesis we focus our attention in obtaining closed-form solutions, in particular those that only require the four basic arithmetic operations because they can easily be implemented on microcomputers with limited computational resources. Moreover, closed-form methods are preferable for at least two reasons: they provide the most meaningful answer because they permit analyzing the influence of each variable on the result; and their computational cost, in terms of arithmetic operations, is fixed and assessable beforehand. We have actually derived closed-form methods specifically tailored for solving the hand-eye calibration and the pointcloud registration problems which outperform all previous approaches.Dado que la función que aplica a cada cuaternión su matrix de rotación correspondiente es 2 a 1, la inversa de esta función no es diferenciable en todo su dominio. Por consiguiente, a veces se asume erróneamente que todas las inversiones deben contener necesariamente singularidades que surgen en forma de cocientes donde el divisor puede ser arbitrariamente pequeño. Esta idea errónea se aclaró cuando encontramos un nuevo método de conversión sin división. Este resultado desencadenó el trabajo de investigación presentado en esta tesis. A primera vista, la conversión de matriz a cuaternión no parece un problema relevante. En realidad, la mayoría de los investigadores lo consideran un problema bien resuelto cuya revisión no es probable que proporcione nuevos resultados en ningún área de interés práctico. Sin embargo, mostramos en esta tesis cómo la resolución del problema de la matriz de rotación más cercana según la norma de Frobenius se puede reducir a una conversión de matriz a cuaternión. Muchos problemas, como el de la calibración mano-cámara, el de la estimación de la pose de una cámara, el de la identificación de una ubicación, el del solapamiento de imágenes, etc. requieren encontrar la matriz de rotación más cercana a una matriz dada. Por lo tanto, la conversión de matriz a cuaternión se vuelve de suma importancia. Mientras que una rotación en 3D se puede representar mediante un cuaternión, una rotación en 4D se puede representar mediante un cuaternión doble. Como consecuencia, el cálculo de la matriz de rotación más cercana en 4D, utilizando nuestro enfoque, sigue esencialmente los mismos pasos que en el caso 3D. Aunque el caso 4D pueda parecer de interés teórico únicamente, mostramos en esta tesis su relevancia práctica gracias a una función poco conocida que relaciona desplazamientos en 3D con rotaciones en 4D. En esta tesis nos centramos en la obtención de soluciones de forma cerrada, en particular aquellas que solo requieren las cuatro operaciones aritméticas básicas porque se pueden implementar fácilmente en microcomputadores con recursos computacionales limitados. Además, los métodos de forma cerrada son preferibles por al menos dos razones: proporcionan la respuesta más significativa porque permiten analizar la influencia de cada variable en el resultado; y su costo computacional, en términos de operaciones aritméticas, es fijo y evaluable de antemano. De hecho, hemos derivado nuevos métodos de forma cerrada diseñados específicamente para resolver el problema de la calibración mano-cámara y el del registro de nubes de puntos cuya eficiencia supera la de todos los métodos anteriores.Postprint (published version

Improving Filtering for Computer Graphics

When drawing images onto a computer screen, the information in the scene is typically more detailed than can be displayed. Most objects, however, will not be close to the camera, so details have to be filtered out, or anti-aliased, when the objects are drawn on the screen. I describe new methods for filtering images and shapes with high fidelity while using computational resources as efficiently as possible. Vector graphics are everywhere, from drawing 3D polygons to 2D text and maps for navigation software. Because of its numerous applications, having a fast, high-quality rasterizer is important. I developed a method for analytically rasterizing shapes using wavelets. This approach allows me to produce accurate 2D rasterizations of images and 3D voxelizations of objects, which is the first step in 3D printing. I later improved my method to handle more filters. The resulting algorithm creates higher-quality images than commercial software such as Adobe Acrobat and is several times faster than the most highly optimized commercial products. The quality of texture filtering also has a dramatic impact on the quality of a rendered image. Textures are images that are applied to 3D surfaces, which typically cannot be mapped to the 2D space of an image without introducing distortions. For situations in which it is impossible to change the rendering pipeline, I developed a method for precomputing image filters over 3D surfaces. If I can also change the pipeline, I show that it is possible to improve the quality of texture sampling significantly in real-time rendering while using the same memory bandwidth as used in traditional methods

Accurate and reliable probabilistic modeling with high-dimensional data

Machine learning studies algorithms for learning from data. Probabilistic modeling and reasoning define a principled framework for machine learning, where probability theory is used to represent and manipulate knowledge. In this thesis we focus on two fundamental tasks in probabilistic machine learning: probabilistic prediction and density estimation. We study reliability of probabilistic predictive models, propose flexible models for density estimation, and propose a novel training regime for densities with low-dimensional structure. Neural networks demonstrate state-of-the-art performance in many different prediction tasks. At the same time, modern neural networks trained by maximum likelihood have poorly calibrated predictive uncertainties and suffer from adversarial examples. We hypothesize that careful probabilistic treatment of neural networks would make them better calibrated and more robust. However, Bayesian neural networks have to rely on uninformative priors and crude approximations, which makes it difficult to test this hypothesis. In this thesis we take a step back and study adversarial robustness of a simple, linear model, demonstrating that it no longer suffers from calibration errors on adversarial points when the approximate inference method is accurate and the prior is chosen carefully. Classic density estimation methods do not scale to complex, high-dimensional data like natural images. Normalizing flows model the target density as an invertible transformation of a simple base density, and demonstrate good results in high-dimensional density estimation tasks. State-of-the-art normalizing flow architectures rely on parametrizations of univariate invertible functions. Simple additive/affine parametrizations are often used, stacking many layers to express complex transformations. In this thesis we propose novel parametrizations based on cubic and rational-quadratic splines. The proposed flows demonstrate improved parameter-efficiency and advance state-of-the-art on several density estimation benchmarks. The manifold hypothesis says that the data are likely to lie on a lower-dimensional manifold. This assumption is built into many machine learning models, but using it with density models like normalizing flows is difficult: the standard likelihood-based training objective becomes ill-defined. Injective normalizing flows can be implemented, but their training objective is no longer tractable, requiring approximations or heuristic alternatives. In this thesis we propose a novel training objective that uses nested dropout to align the latent space of a normalizing flow, allowing us to extract a sequence of manifold densities from the trained model. Our experiments demonstrate that the manifolds fit by the method match the data well

Neural distribution estimation as a two-part problem

Given a dataset of examples, distribution estimation is the task of approximating the assumed underlying probability distribution from which those samples were drawn. Neural distribution estimation relies on the powerful function approximation capabilities of deep neural networks to build models for this purpose, and excels when data is high-dimensional and exhibits complex, nonlinear dependencies. In this thesis, we explore several approaches to neural distribution estimation, and present a unified perspective for these methods based on a two-part design principle. In particular, we examine how many models iteratively break down the task of distribution estimation into a series of tractable sub-tasks, before fitting a multi-step generative process which combines solutions to these sub-tasks in order to approximate the data distribution of interest. Framing distribution estimation as a two-part problem provides a shared language in which to compare and contrast prevalent models in the literature, and also allows for discussion of alternative approaches which do not follow this structure. We first present the Autoregressive Energy Machine, an energy-based model which is trained by approximate maximum likelihood through an autoregressive decomposition. The method demonstrates the flexibility of an energy-based model over an explicitly normalized model, and the novel application of autoregressive importance sampling highlights the benefit of an autoregressive approach to distribution estimation which recursively transforms the problem into a series of univariate tasks. Next, we present Neural Spline Flows, a class of normalizing flow models based on monotonic spline transformations which admit both an explicit inverse and a tractable Jacobian determinant. Normalizing flows tackle distribution estimation by searching for an invertible map between the data distribution and a more tractable base distribution, and this map is typically constructed as the composition of a series of invertible building blocks. We demonstrate that spline flows can be used to enhance density estimation of tabular data, variational inference in latent variable models, and generative modeling of natural images. The third chapter presents Maximum Likelihood Training of Score-Based Diffusion Models. Generative models based on estimation of the gradient of the logarithm of the probability density---or score function---have recently gained traction as a powerful modeling paradigm, in which the data distribution is gradually transformed toward a tractable base distribution by means of a stochastic process. The paper illustrates how this class of models can be trained by maximum likelihood, resulting in a model which is functionally equivalent to a continuous normalizing flow, and which bridges the gap between two branches of the literature. We also discuss latent-variable generative models more broadly, of which diffusion models are a structured special case. Finally, we present On Contrastive Learning for Likelihood-Free Inference, a unifying perspective for likelihood-free inference methods which perform Bayesian inference using either density estimation or density-ratio estimation. Likelihood-free inference focuses on inference in stochastic simulator models where the likelihood of parameters given observations is computationally intractable, and traditional inference methods fall short. In addition to illustrating the power of normalizing flows as generic tools for density estimation, this chapter also gives us the opportunity to discuss likelihood-free models more broadly. These so-called implicit generative models form a large part of the distribution estimation literature under the umbrella of generative adversarial networks, and are distinct in how they treat distribution estimation as a one-part problem