GANs schön kompliziert: Applications of Generative Adversarial Networks

Scientific research progresses via model-building. Researchers attempt to build realistic models of real-world phenomena, ranging from bacterial growth to galactic motion, and study these models as a means of understanding these phenomena. However, making these models as realistic as possible often involves fitting them to experimentally measured data. Recent advances in experimental methods have allowed for the collection of large-scale datasets. Simultaneously, advancements in computational capacity have allowed for more complex model-building. The confluence of these two factors accounts for the rise of machine learning methods as powerful tools, both for building models and fitting these models to large scale datasets. In this thesis, we use a particular machine learning technique: generative adversarial networks (GANs). GANs are a flexible and powerful tool, capable of fitting a wide variety of models. We explore the properties of GANs that underpin this flexibility, and show how we can capitalize on them in different scientific applications, beyond the image- and text-generating applications they are well-known for. Here we present three different applications of GANs. First, we show how GANs can be used as generative models of neural spike trains, and how they are capable of capturing more features of these spike trains compared to other approaches. We also show how this could enable insight into how information about stimuli are encoded in the spike trains. Second, we demonstrate how GANs can be used as density estimators for extending simulation-based Bayesian inference to high-dimensional parameter spaces. In this form, we also show how GANs bridge Bayesian inference methods and variational inference with autoencoders and use them to fit complex climate models to data. Finally, we use GANs to infer synaptic plasticity rules for biological rate networks directly from data. We then show how GANs be used to test the robustness of the inferred rules to differences in data and network initialisation. Overall, we repurpose GANs in new ways for a variety of scientific domains, and show that they confer specific advantages over the state-of-the-art methods in each of these domains

Stochastic ICA contrast maximisation using OJA's nonlinear PCA algorithm.

Independent Component Analysis (ICA) is an important extension of linear Principal Component Analysis (PCA). PCA performs a data transformation to provide independence to second order, that is, decorrelation. ICA transforms data to provide approximate independence up to and beyond second order yielding transformed data with fully factorable probability densities. The linear ICA transformation has been applied to the classical statistical signal-processing problem of Blind Separation of Sources (BSS), that is, separating unknown original source signals from a mixture whose mode of mixing is undetermined. In this paper it is shown that Oja's Nonlinear PCA algorithm performs a general stochastic online adaptive ICA. This analysis is corroborated with three simulations. The first separates unknown mixtures of original natural images, which have sub-Gaussian densities, the second separates linear mixtures of natural speech whose densities are super-Gaussian. Finally unknown mixtures of original images, which have both sub- and super-Gaussian densities are separated