Learned SVD: solving inverse problems via hybrid autoencoding

Our world is full of physics-driven data where effective mappings between data manifolds are desired. There is an increasing demand for understanding combined model-based and data-driven methods. We propose a nonlinear, learned singular value decomposition (L-SVD), which combines autoencoders that simultaneously learn and connect latent codes for desired signals and given measurements. We provide a convergence analysis for a specifically structured L-SVD that acts as a regularisation method. In a more general setting, we investigate the topic of model reduction via data dimensionality reduction to obtain a regularised inversion. We present a promising direction for solving inverse problems in cases where the underlying physics are not fully understood or have very complex behaviour. We show that the building blocks of learned inversion maps can be obtained automatically, with improved performance upon classical methods and better interpretability than black-box methods

Structured recognition for generative models with explaining away

A key goal of unsupervised learning is to go beyond density estimation and sample generation to reveal the structure inherent within observed data. Such structure can be expressed in the pattern of interactions between explanatory latent variables captured through a probabilistic graphical model. Although the learning of structured graphical models has a long history, much recent work in unsupervised modelling has instead emphasised flexible deep-network-based generation, either transforming independent latent generators to model complex data or assuming that distinct observed variables are derived from different latent nodes. Here, we extend amortised variational inference to incorporate structured factors over multiple variables, able to capture the observation-induced posterior dependence between latents that results from “explaining away” and thus allow complex observations to depend on multiple nodes of a structured graph. We show that appropriately parametrised factors can be combined efficiently with variational message passing in rich graphical structures. We instantiate the framework in nonlinear Gaussian Process Factor Analysis, evaluating the structured recognition framework using synthetic data from known generative processes. We fit the GPFA model to high-dimensional neural spike data from the hippocampus of freely moving rodents, where the model successfully identifies latent signals that correlate with behavioural covariates

Noise Reduction in EEG Signals using Convolutional Autoencoding Techniques

The presence of noise in electroencephalography (EEG) signals can significantly reduce the accuracy of the analysis of the signal. This study assesses to what extent stacked autoencoders designed using one-dimensional convolutional neural network layers can reduce noise in EEG signals. The EEG signals, obtained from 81 people, were processed by a two-layer one-dimensional convolutional autoencoder (CAE), whom performed 3 independent button pressing tasks. The signal-to-noise ratios (SNRs) of the signals before and after processing were calculated and the distributions of the SNRs were compared. The performance of the model was compared to noise reduction performance of Principal Component Analysis, with 95% explained variance, by comparing the Harrell-Davis decile differences between the SNR distributions of both methods and the raw signal SNR distribution for each task. It was found that the CAE outperformed PCA for the full dataset across all three tasks, however the CAE did not outperform PCA for the person specific datasets in any of the three tasks. The results indicate that CAEs can perform better than PCA for noise reduction in EEG signals, but performance of the model may be training size dependent

End-to-end Sinkhorn Autoencoder with Noise Generator

In this work, we propose a novel end-to-end sinkhorn autoencoder with noise generator for efficient data collection simulation. Simulating processes that aim at collecting experimental data is crucial for multiple real-life applications, including nuclear medicine, astronomy and high energy physics. Contemporary methods, such as Monte Carlo algorithms, provide high-fidelity results at a price of high computational cost. Multiple attempts are taken to reduce this burden, e.g. using generative approaches based on Generative Adversarial Networks or Variational Autoencoders. Although such methods are much faster, they are often unstable in training and do not allow sampling from an entire data distribution. To address these shortcomings, we introduce a novel method dubbed end-to-end Sinkhorn Autoencoder, that leverages sinkhorn algorithm to explicitly align distribution of encoded real data examples and generated noise. More precisely, we extend autoencoder architecture by adding a deterministic neural network trained to map noise from a known distribution onto autoencoder latent space representing data distribution. We optimise the entire model jointly. Our method outperforms competing approaches on a challenging dataset of simulation data from Zero Degree Calorimeters of ALICE experiment in LHC. as well as standard benchmarks, such as MNIST and CelebA