Neural Network Operations and Susuki-Trotter evolution of Neural Network States

It was recently proposed to leverage the representational power of artificial neural networks, in particular Restricted Boltzmann Machines, in order to model complex quantum states of many-body systems [Science, 355(6325), 2017]. States represented in this way, called Neural Network States (NNSs), were shown to display interesting properties like the ability to efficiently capture long-range quantum correlations. However, identifying an optimal neural network representation of a given state might be challenging, and so far this problem has been addressed with stochastic optimization techniques. In this work we explore a different direction. We study how the action of elementary quantum operations modifies NNSs. We parametrize a family of many body quantum operations that can be directly applied to states represented by Unrestricted Boltzmann Machines, by just adding hidden nodes and updating the network parameters. We show that this parametrization contains a set of universal quantum gates, from which it follows that the state prepared by any quantum circuit can be expressed as a Neural Network State with a number of hidden nodes that grows linearly with the number of elementary operations in the circuit. This is a powerful representation theorem (which was recently obtained with different methods) but that is not directly useful, since there is no general and efficient way to extract information from this unrestricted description of quantum states. To circumvent this problem, we propose a step-wise procedure based on the projection of Unrestricted quantum states to Restricted quantum states. In turn, two approximate methods to perform this projection are discussed. In this way, we show that it is in principle possible to approximately optimize or evolve Neural Network States without relying on stochastic methods such as Variational Monte Carlo, which are computationally expensive

Molecular associative memory: An associative memory framework with exponential storage capacity for DNA computing

Associative memory problem: Find the closest stored vector (in Hamming distance) to a given query vector. There are different ways to implement an associative memory, including the neural networks and DNA strands. Using neural networks, connection weights are adjusted in order to perform association. Recall procedure is iterative and relies on simple neural operations. In this case, the design criteria is maximizing the number of stored patterns C while having some noise tolerance. The molecular implementation is based on synthesizing C DNA strands as stored vectors. Recall procedure is usually done in one shot via chemical reactions and relies on highly parallelism of DNA computing. Here, the design criteria: finding proper DNA sequences to minimize probability of error during the recall phase. Current molecular associative memories are either low in storage capacity, if implemented using molecular realizations of neural networks, or very complex to implement, if all the stored sequences have to be synthesized. We introduce an associative memory framework with exponential storage capacity based on transcriptional networks of DNA switches. The advantages of the proposed approach over current methods are: 1. Exponential storage capacities with current neural network-based approaches can not be achieved. 2. For other methods, although having exponential storage capacities is possible, it is very complex as it requires synthesizing an extraordinarily large number of DNA strands

Learning a Mixture of Deep Networks for Single Image Super-Resolution

Single image super-resolution (SR) is an ill-posed problem which aims to recover high-resolution (HR) images from their low-resolution (LR) observations. The crux of this problem lies in learning the complex mapping between low-resolution patches and the corresponding high-resolution patches. Prior arts have used either a mixture of simple regression models or a single non-linear neural network for this propose. This paper proposes the method of learning a mixture of SR inference modules in a unified framework to tackle this problem. Specifically, a number of SR inference modules specialized in different image local patterns are first independently applied on the LR image to obtain various HR estimates, and the resultant HR estimates are adaptively aggregated to form the final HR image. By selecting neural networks as the SR inference module, the whole procedure can be incorporated into a unified network and be optimized jointly. Extensive experiments are conducted to investigate the relation between restoration performance and different network architectures. Compared with other current image SR approaches, our proposed method achieves state-of-the-arts restoration results on a wide range of images consistently while allowing more flexible design choices. The source codes are available in http://www.ifp.illinois.edu/~dingliu2/accv2016

Metric Gaussian variational inference

One main result of this dissertation is the development of Metric Gaussian Variational Inference (MGVI), a method to perform approximate inference in extremely high dimensions and for complex probabilistic models. The problem with high-dimensional and complex models is twofold. Fist, to capture the true posterior distribution accurately, a sufficiently rich approximation for it is required. Second, the number of parameters to express this richness scales dramatically with the number of model parameters. For example, explicitly expressing the correlation between all model parameters requires their squared number of correlation coefficients. In settings with millions of model parameter, this is unfeasible. MGVI overcomes this limitation by replacing the explicit covariance with an implicit approximation, which does not have to be stored and is accessed via samples. This procedure scales linearly with the problem size and allows to account for the full correlations in even extremely large problems. This makes it also applicable to significantly more complex setups. MGVI enabled a series of ambitious signal reconstructions by me and others, which will be showcased. This involves a time- and frequency-resolved reconstruction of the shadow around the black hole M87* using data provided by the Event Horizon Telescope Collaboration, a three-dimensional tomographic reconstruction of interstellar dust within 300pc around the sun from Gaia starlight-absorption and parallax data, novel medical imaging methods for computed tomography, an all-sky Faraday rotation map, combining distinct data sources, and simultaneous calibration and imaging with a radio-interferometer. The second main result is an an approach to use several, independently trained and deep neural networks to reason on complex tasks. Deep learning allows to capture abstract concepts by extracting them from large amounts of training data, which alleviates the necessity of an explicit mathematical formulation. Here a generative neural network is used as a prior distribution and certain properties are imposed via classification and regression networks. The inference is then performed in terms of the latent variables of the generator, which is done using MGVI and other methods. This allows to flexibly answer novel questions without having to re-train any neural network and to come up with novel answers through Bayesian reasoning. This novel approach of Bayesian reasoning with neural networks can also be combined with conventional measurement data

An automated model reduction method for biochemical reaction networks

We propose a new approach to the model reduction of biochemical reaction networks governed by various types of enzyme kinetics rate laws with non-autocatalytic reactions, each of which can be reversible or irreversible. This method extends the approach for model reduction previously proposed by Rao et al. which proceeds by the step-wise reduction in the number of complexes by Kron reduction of the weighted Laplacian corresponding to the complex graph of the network. The main idea in the current manuscript is based on rewriting the mathematical model of a reaction network as a model of a network consisting of linkage classes that contain more than one reaction. It is done by joining certain distinct linkage classes into a single linkage class by using the conservation laws of the network. We show that this adjustment improves the extent of applicability of the method proposed by Rao et al. We automate the entire reduction procedure using Matlab. We test our automated model reduction to two real-life reaction networks, namely, a model of neural stem cell regulation and a model of hedgehog signaling pathway. We apply our reduction approach to meaningfully reduce the number of complexes in the complex graph corresponding to these networks. When the number of species' concentrations in the model of neural stem cell regulation is reduced by 33.33%, the difference between the dynamics of the original model and the reduced model, quantified by an error integral, is only 4.85%. Likewise, when the number of species' concentrations is reduced by 33.33% in the model of hedgehog signaling pathway, the difference between the dynamics of the original model and the reduced model is only 6.59%