Out-of-Distribution Detection of Melanoma using Normalizing Flows

Generative modelling has been a topic at the forefront of machine learning research for a substantial amount of time. With the recent success in the field of machine learning, especially in deep learning, there has been an increased interest in explainable and interpretable machine learning. The ability to model distributions and provide insight in the density estimation and exact data likelihood is an example of such a feature. Normalizing Flows (NFs), a relatively new research field of generative modelling, has received substantial attention since it is able to do exactly this at a relatively low cost whilst enabling competitive generative results. While the generative abilities of NFs are typically explored, we focus on exploring the data distribution modelling for Out-of-Distribution (OOD) detection. Using one of the state-of-the-art NF models, GLOW, we attempt to detect OOD examples in the ISIC dataset. We notice that this model under performs in conform related research. To improve the OOD detection, we explore the masking methods to inhibit co-adaptation of the coupling layers however find no substantial improvement. Furthermore, we utilize Wavelet Flow which uses wavelets that can filter particular frequency components, thus simplifying the modeling process to data-driven conditional wavelet coefficients instead of complete images. This enables us to efficiently model larger resolution images in the hopes that it would capture more relevant features for OOD. The paper that introduced Wavelet Flow mainly focuses on its ability of sampling high resolution images and did not treat OOD detection. We present the results and propose several ideas for improvement such as controlling frequency components, using different wavelets and using other state-of-the-art NF architectures

Normalising Flows for Bayesian Gravity Inversion

Gravity inversion is a commonly applied data analysis technique in the field of geophysics. While machine learning methods have previously been explored for the problem of gravity inversion, these are deterministic approaches returning a single solution deemed most appropriate by the algorithm. The method presented here takes a different approach, where gravity inversion is reformulated as a Bayesian parameter inference problem. Samples from the posterior probability distribution of source model parameters are obtained via the implementation of a generative neural network architecture known as Normalising Flows. Due to its probabilistic nature, this framework provides the user with a range of source parameters and uncertainties instead of a single solution, and is inherently robust against instrumental noise. The performance of the Normalising Flow is compared to that of an established Bayesian method called Nested Sampling. It is shown that the new method returns results with comparable accuracy 200 times faster than standard sampling methods, which makes Normalising Flows a suitable method for real-time inversion in the field. When applied to data sets with high dimensionality, standard sampling methods can become impractical due to long computation times. It is shown that inversion using Normalising Flows remains tractable even at 512 dimensions and once the network is trained, the results can be obtained in $O(10)$ seconds.Comment: 14 pages, 6 figures, submitted for publication in Computers & Geosciences Journa

SurVAE Flows: Surjections to Bridge the Gap between VAEs and Flows

Normalizing flows and variational autoencoders are powerful generative models that can represent complicated density functions. However, they both impose constraints on the models: Normalizing flows use bijective transformations to model densities whereas VAEs learn stochastic transformations that are non-invertible and thus typically do not provide tractable estimates of the marginal likelihood. In this paper, we introduce SurVAE Flows: A modular framework of composable transformations that encompasses VAEs and normalizing flows. SurVAE Flows bridge the gap between normalizing flows and VAEs with surjective transformations, wherein the transformations are deterministic in one direction -- thereby allowing exact likelihood computation, and stochastic in the reverse direction -- hence providing a lower bound on the corresponding likelihood. We show that several recently proposed methods, including dequantization and augmented normalizing flows, can be expressed as SurVAE Flows. Finally, we introduce common operations such as the max value, the absolute value, sorting and stochastic permutation as composable layers in SurVAE Flows