13 research outputs found
Representation Equivalent Neural Operators: a Framework for Alias-free Operator Learning
Recently, operator learning, or learning mappings between
infinite-dimensional function spaces, has garnered significant attention,
notably in relation to learning partial differential equations from data.
Conceptually clear when outlined on paper, neural operators necessitate
discretization in the transition to computer implementations. This step can
compromise their integrity, often causing them to deviate from the underlying
operators. This research offers a fresh take on neural operators with a
framework Representation equivalent Neural Operators (ReNO) designed to address
these issues. At its core is the concept of operator aliasing, which measures
inconsistency between neural operators and their discrete representations. We
explore this for widely-used operator learning techniques. Our findings detail
how aliasing introduces errors when handling different discretizations and
grids and loss of crucial continuous structures. More generally, this framework
not only sheds light on existing challenges but, given its constructive and
broad nature, also potentially offers tools for developing new neural
operators.Comment: 28 page
Convolutional Neural Operators for robust and accurate learning of PDEs
Although very successfully used in conventional machine learning, convolution
based neural network architectures -- believed to be inconsistent in function
space -- have been largely ignored in the context of learning solution
operators of PDEs. Here, we present novel adaptations for convolutional neural
networks to demonstrate that they are indeed able to process functions as
inputs and outputs. The resulting architecture, termed as convolutional neural
operators (CNOs), is designed specifically to preserve its underlying
continuous nature, even when implemented in a discretized form on a computer.
We prove a universality theorem to show that CNOs can approximate operators
arising in PDEs to desired accuracy. CNOs are tested on a novel suite of
benchmarks, encompassing a diverse set of PDEs with possibly multi-scale
solutions and are observed to significantly outperform baselines, paving the
way for an alternative framework for robust and accurate operator learning
Augmenting Physical Models with Deep Networks for Complex Dynamics Forecasting
Forecasting complex dynamical phenomena in settings where only partial
knowledge of their dynamics is available is a prevalent problem across various
scientific fields. While purely data-driven approaches are arguably
insufficient in this context, standard physical modeling based approaches tend
to be over-simplistic, inducing non-negligible errors. In this work, we
introduce the APHYNITY framework, a principled approach for augmenting
incomplete physical dynamics described by differential equations with deep
data-driven models. It consists in decomposing the dynamics into two
components: a physical component accounting for the dynamics for which we have
some prior knowledge, and a data-driven component accounting for errors of the
physical model. The learning problem is carefully formulated such that the
physical model explains as much of the data as possible, while the data-driven
component only describes information that cannot be captured by the physical
model, no more, no less. This not only provides the existence and uniqueness
for this decomposition, but also ensures interpretability and benefits
generalization. Experiments made on three important use cases, each
representative of a different family of phenomena, i.e. reaction-diffusion
equations, wave equations and the non-linear damped pendulum, show that
APHYNITY can efficiently leverage approximate physical models to accurately
forecast the evolution of the system and correctly identify relevant physical
parameters
Modélisation de Processus Physique avec de l'Apprentissage Profond
Deep Learning has emerged as a predominant tool for AI, and has alreadymany applications in fields where data is abundant and access to prior knowledge is difficult. This is not necessarily the case for natural sciences, and in particular, for physical processes. Indeed, these have been the object of study since centuries, a vast amount of knowledge has been acquired, and elaborate algorithms and methods have been developed. Thus, this thesis has two main objectives. The first considers the study of the role that deep learning has to play in this vast ecosystem of knowledge, theory and tools. We will attempt to answer this general question through a concrete problem: the modelling complex physical processes, leveraging deep learning methods in order to make up for lacking prior knowledge. The second objective is somewhat its dual: it focuses on how perspectives, insights and tools from the field of study of physical processes and dynamical systems can be applied in the context of deep learning, in order to gain a better understanding and develop novel algorithms.L'apprentissage profond s'impose comme un outil prédominant pour l'IA, avec de nombreuses applications fructueuses pour des tâches où les données sont abondantes et l'accès aux connaissances préalables est difficile. Cependant ce n'est pas encore le cas dans le domaine des sciences naturelles, et encore moins pour l'étude des systèmes dynamiques. En effet, ceux-ci font l'objet d'études depuis des siècles, une quantité considérable de connaissances a ainsi été acquise, et des algorithmes et des méthodes ingénieux ont été développés. Cette thèse a donc deux objectifs principaux. Le premier concerne l'étude du rôle que l'apprentissage profond doit jouer dans ce vaste écosystème de connaissances, de théories et d'outils. Nous tenterons de répondre à cette question générale à travers un problème concret: la modélisation de processus physiques complexes à l'aide de l'apprentissage profond. Le deuxième objectif est en quelque sorte son dual; il concerne l'analyse des algorithmes d'apprentissage profond à travers le prisme des systèmes dynamiques et des processus physiques, dans le but d'acquérir une meilleure compréhension et de développer de nouveaux algorithmes pour ce domaine
Learning Dynamical Systems from Partial Observations
We consider the problem of forecasting complex, nonlinear space-time processes when observations provide only partial information of on the system's state. We propose a natural data-driven framework, where the system's dynamics are modelled by an unknown time-varying differential equation, and the evolution term is estimated from the data, using a neural network. Any future state can then be computed by placing the associated differential equation in an ODE solver. We first evaluate our approach on shallow water and Euler simulations. We find that our method not only demonstrates high quality long-term forecasts, but also learns to produce hidden states closely resembling the true states of the system, without direct supervision on the latter. Additional experiments conducted on challenging, state of the art ocean simulations further validate our findings, while exhibiting notable improvements over classical baselines
Unifying GANs and Score-Based Diffusion as Generative Particle Models
International audienceParticle-based deep generative models, such as gradient flows and score-based diffusion models, have recently gained traction thanks to their striking performance. Their principle of displacing particle distributions using differential equations is conventionally seen as opposed to the previously widespread generative adversarial networks (GANs), which involve training a pushforward generator network. In this paper we challenge this interpretation, and propose a novel framework that unifies particle and adversarial generative models by framing generator training as a generalization of particle models. This suggests that a generator is an optional addition to any such generative model. Consequently, integrating a generator into a score-based diffusion model and training a GAN without a generator naturally emerge from our framework. We empirically test the viability of these original models as proofs of concepts of potential applications of our framework