11,949 research outputs found
Universal Approximation Property of Hamiltonian Deep Neural Networks
This paper investigates the universal approximation capabilities of
Hamiltonian Deep Neural Networks (HDNNs) that arise from the discretization of
Hamiltonian Neural Ordinary Differential Equations. Recently, it has been shown
that HDNNs enjoy, by design, non-vanishing gradients, which provide numerical
stability during training. However, although HDNNs have demonstrated
state-of-the-art performance in several applications, a comprehensive study to
quantify their expressivity is missing. In this regard, we provide a universal
approximation theorem for HDNNs and prove that a portion of the flow of HDNNs
can approximate arbitrary well any continuous function over a compact domain.
This result provides a solid theoretical foundation for the practical use of
HDNNs
Introduction to neural ordinary differential equations
This thesis aims to provide a comprehensive overview of Neural Networks (NN) and Neural Ordinary Differential Equations (NODEs) from a mathematical standpoint and insights into their training methods and approximation capabilities. The first chapter covers the basics of NNs, including the mathematics of gradient descent and Universal Approximation theorems (UA), as well as an introduction to residual NNs. The second chapter dives into the world of NODEs, which can be thought of as continuous idealisations of residual NNs. Then we explore some UA theorems for NODEs and examine three different training methods: direct backpropagation, continuous adjoint, and the adaptative checkpoint adjoint method (ACA). Additionally, we explore some applications of NODEs, such as image classification, and provide a PyTorch code example that trains a NODE to approximate the trajectories of a Lorenz System using ACA
Training Echo State Networks with Regularization through Dimensionality Reduction
In this paper we introduce a new framework to train an Echo State Network to
predict real valued time-series. The method consists in projecting the output
of the internal layer of the network on a space with lower dimensionality,
before training the output layer to learn the target task. Notably, we enforce
a regularization constraint that leads to better generalization capabilities.
We evaluate the performances of our approach on several benchmark tests, using
different techniques to train the readout of the network, achieving superior
predictive performance when using the proposed framework. Finally, we provide
an insight on the effectiveness of the implemented mechanics through a
visualization of the trajectory in the phase space and relying on the
methodologies of nonlinear time-series analysis. By applying our method on well
known chaotic systems, we provide evidence that the lower dimensional embedding
retains the dynamical properties of the underlying system better than the
full-dimensional internal states of the network
Neural Networks in Nonlinear Aircraft Control
Recent research indicates that artificial neural networks offer interesting learning or adaptive capabilities. The current research focuses on the potential for application of neural networks in a nonlinear aircraft control law. The current work has been to determine which networks are suitable for such an application and how they will fit into a nonlinear control law
Artificial Neural Network Methods in Quantum Mechanics
In a previous article we have shown how one can employ Artificial Neural
Networks (ANNs) in order to solve non-homogeneous ordinary and partial
differential equations. In the present work we consider the solution of
eigenvalue problems for differential and integrodifferential operators, using
ANNs. We start by considering the Schr\"odinger equation for the Morse
potential that has an analytically known solution, to test the accuracy of the
method. We then proceed with the Schr\"odinger and the Dirac equations for a
muonic atom, as well as with a non-local Schr\"odinger integrodifferential
equation that models the system in the framework of the resonating
group method. In two dimensions we consider the well studied Henon-Heiles
Hamiltonian and in three dimensions the model problem of three coupled
anharmonic oscillators. The method in all of the treated cases proved to be
highly accurate, robust and efficient. Hence it is a promising tool for
tackling problems of higher complexity and dimensionality.Comment: Latex file, 29pages, 11 psfigs, submitted in CP
Machine Learning for Fluid Mechanics
The field of fluid mechanics is rapidly advancing, driven by unprecedented
volumes of data from field measurements, experiments and large-scale
simulations at multiple spatiotemporal scales. Machine learning offers a wealth
of techniques to extract information from data that could be translated into
knowledge about the underlying fluid mechanics. Moreover, machine learning
algorithms can augment domain knowledge and automate tasks related to flow
control and optimization. This article presents an overview of past history,
current developments, and emerging opportunities of machine learning for fluid
mechanics. It outlines fundamental machine learning methodologies and discusses
their uses for understanding, modeling, optimizing, and controlling fluid
flows. The strengths and limitations of these methods are addressed from the
perspective of scientific inquiry that considers data as an inherent part of
modeling, experimentation, and simulation. Machine learning provides a powerful
information processing framework that can enrich, and possibly even transform,
current lines of fluid mechanics research and industrial applications.Comment: To appear in the Annual Reviews of Fluid Mechanics, 202
- …