Nonparametric Weight Initialization of Neural Networks via Integral Representation

A new initialization method for hidden parameters in a neural network is proposed. Derived from the integral representation of the neural network, a nonparametric probability distribution of hidden parameters is introduced. In this proposal, hidden parameters are initialized by samples drawn from this distribution, and output parameters are fitted by ordinary linear regression. Numerical experiments show that backpropagation with proposed initialization converges faster than uniformly random initialization. Also it is shown that the proposed method achieves enough accuracy by itself without backpropagation in some cases.Comment: For ICLR2014, revised into 9 pages; revised into 12 pages (with supplements

Precision Learning: Towards Use of Known Operators in Neural Networks

In this paper, we consider the use of prior knowledge within neural networks. In particular, we investigate the effect of a known transform within the mapping from input data space to the output domain. We demonstrate that use of known transforms is able to change maximal error bounds. In order to explore the effect further, we consider the problem of X-ray material decomposition as an example to incorporate additional prior knowledge. We demonstrate that inclusion of a non-linear function known from the physical properties of the system is able to reduce prediction errors therewith improving prediction quality from SSIM values of 0.54 to 0.88. This approach is applicable to a wide set of applications in physics and signal processing that provide prior knowledge on such transforms. Also maximal error estimation and network understanding could be facilitated within the context of precision learning.Comment: accepted on ICPR 201

Quantum algorithm for non-homogeneous linear partial differential equations

We describe a quantum algorithm for preparing states that encode solutions of non-homogeneous linear partial differential equations. The algorithm is a continuous-variable version of matrix inversion: it efficiently inverts differential operators that are polynomials in the variables and their partial derivatives. The output is a quantum state whose wavefunction is proportional to a specific solution of the non-homogeneous differential equation, which can be measured to reveal features of the solution. The algorithm consists of three stages: preparing fixed resource states in ancillary systems, performing Hamiltonian simulation, and measuring the ancilla systems. The algorithm can be carried out using standard methods for gate decompositions, but we improve this in two ways. First, we show that for a wide class of differential operators, it is possible to derive exact decompositions for the gates employed in Hamiltonian simulation. This avoids the need for costly commutator approximations, reducing gate counts by orders of magnitude. Additionally, we employ methods from machine learning to find explicit circuits that prepare the required resource states. We conclude by studying an example application of the algorithm: solving Poisson's equation in electrostatics.Comment: 9 pages, 6 figure