767 research outputs found
Acceleration of Levenberg-Marquardt Training of Neural Networks with Variable Decay Rate
In the application of the standard Levenherg-Marquardt training process of neural networks, error oscillations are frequently observed and they usually aggravate on approaching the required accuracy. In this paper, a modified Levenberg-Marquardt method based on variable decay rate in each iteration is proposed in order to reduce such error oscillations. Through a certain variation of the decay rate, the time required for training of neural networks is cut down to less than half of that required in the standard Levenberg-Marquardt method. Several numerical examples are given to show the effectiveness of the proposed method.published_or_final_versio
Neural networks in geophysical applications
Neural networks are increasingly popular in geophysics.
Because they are universal approximators, these
tools can approximate any continuous function with an
arbitrary precision. Hence, they may yield important
contributions to finding solutions to a variety of geophysical applications.
However, knowledge of many methods and techniques
recently developed to increase the performance
and to facilitate the use of neural networks does not seem
to be widespread in the geophysical community. Therefore,
the power of these tools has not yet been explored to
their full extent. In this paper, techniques are described
for faster training, better overall performance, i.e., generalization,and the automatic estimation of network size
and architecture
Efficient Neural Network Approaches for Conditional Optimal Transport with Applications in Bayesian Inference
We present two neural network approaches that approximate the solutions of
static and dynamic conditional optimal transport (COT) problems, respectively.
Both approaches enable sampling and density estimation of conditional
probability distributions, which are core tasks in Bayesian inference. Our
methods represent the target conditional distributions as transformations of a
tractable reference distribution and, therefore, fall into the framework of
measure transport. COT maps are a canonical choice within this framework, with
desirable properties such as uniqueness and monotonicity. However, the
associated COT problems are computationally challenging, even in moderate
dimensions. To improve the scalability, our numerical algorithms leverage
neural networks to parameterize COT maps. Our methods exploit the structure of
the static and dynamic formulations of the COT problem. PCP-Map models
conditional transport maps as the gradient of a partially input convex neural
network (PICNN) and uses a novel numerical implementation to increase
computational efficiency compared to state-of-the-art alternatives. COT-Flow
models conditional transports via the flow of a regularized neural ODE; it is
slower to train but offers faster sampling. We demonstrate their effectiveness
and efficiency by comparing them with state-of-the-art approaches using
benchmark datasets and Bayesian inverse problems.Comment: 25 pages, 7 tables, 8 figure
- …