Representation Equivalent Neural Operators: a Framework for Alias-free Operator Learning

Recently, operator learning, or learning mappings between infinite-dimensional function spaces, has garnered significant attention, notably in relation to learning partial differential equations from data. Conceptually clear when outlined on paper, neural operators necessitate discretization in the transition to computer implementations. This step can compromise their integrity, often causing them to deviate from the underlying operators. This research offers a fresh take on neural operators with a framework Representation equivalent Neural Operators (ReNO) designed to address these issues. At its core is the concept of operator aliasing, which measures inconsistency between neural operators and their discrete representations. We explore this for widely-used operator learning techniques. Our findings detail how aliasing introduces errors when handling different discretizations and grids and loss of crucial continuous structures. More generally, this framework not only sheds light on existing challenges but, given its constructive and broad nature, also potentially offers tools for developing new neural operators.Comment: 28 page

Convolutional Neural Operators for robust and accurate learning of PDEs

Although very successfully used in conventional machine learning, convolution based neural network architectures -- believed to be inconsistent in function space -- have been largely ignored in the context of learning solution operators of PDEs. Here, we present novel adaptations for convolutional neural networks to demonstrate that they are indeed able to process functions as inputs and outputs. The resulting architecture, termed as convolutional neural operators (CNOs), is designed specifically to preserve its underlying continuous nature, even when implemented in a discretized form on a computer. We prove a universality theorem to show that CNOs can approximate operators arising in PDEs to desired accuracy. CNOs are tested on a novel suite of benchmarks, encompassing a diverse set of PDEs with possibly multi-scale solutions and are observed to significantly outperform baselines, paving the way for an alternative framework for robust and accurate operator learning

Augmenting Physical Models with Deep Networks for Complex Dynamics Forecasting

Forecasting complex dynamical phenomena in settings where only partial knowledge of their dynamics is available is a prevalent problem across various scientific fields. While purely data-driven approaches are arguably insufficient in this context, standard physical modeling based approaches tend to be over-simplistic, inducing non-negligible errors. In this work, we introduce the APHYNITY framework, a principled approach for augmenting incomplete physical dynamics described by differential equations with deep data-driven models. It consists in decomposing the dynamics into two components: a physical component accounting for the dynamics for which we have some prior knowledge, and a data-driven component accounting for errors of the physical model. The learning problem is carefully formulated such that the physical model explains as much of the data as possible, while the data-driven component only describes information that cannot be captured by the physical model, no more, no less. This not only provides the existence and uniqueness for this decomposition, but also ensures interpretability and benefits generalization. Experiments made on three important use cases, each representative of a different family of phenomena, i.e. reaction-diffusion equations, wave equations and the non-linear damped pendulum, show that APHYNITY can efficiently leverage approximate physical models to accurately forecast the evolution of the system and correctly identify relevant physical parameters

Modélisation de Processus Physique avec de l'Apprentissage Profond

Deep Learning has emerged as a predominant tool for AI, and has alreadymany applications in fields where data is abundant and access to prior knowledge is difficult. This is not necessarily the case for natural sciences, and in particular, for physical processes. Indeed, these have been the object of study since centuries, a vast amount of knowledge has been acquired, and elaborate algorithms and methods have been developed. Thus, this thesis has two main objectives. The first considers the study of the role that deep learning has to play in this vast ecosystem of knowledge, theory and tools. We will attempt to answer this general question through a concrete problem: the modelling complex physical processes, leveraging deep learning methods in order to make up for lacking prior knowledge. The second objective is somewhat its dual: it focuses on how perspectives, insights and tools from the field of study of physical processes and dynamical systems can be applied in the context of deep learning, in order to gain a better understanding and develop novel algorithms.L'apprentissage profond s'impose comme un outil prédominant pour l'IA, avec de nombreuses applications fructueuses pour des tâches où les données sont abondantes et l'accès aux connaissances préalables est difficile. Cependant ce n'est pas encore le cas dans le domaine des sciences naturelles, et encore moins pour l'étude des systèmes dynamiques. En effet, ceux-ci font l'objet d'études depuis des siècles, une quantité considérable de connaissances a ainsi été acquise, et des algorithmes et des méthodes ingénieux ont été développés. Cette thèse a donc deux objectifs principaux. Le premier concerne l'étude du rôle que l'apprentissage profond doit jouer dans ce vaste écosystème de connaissances, de théories et d'outils. Nous tenterons de répondre à cette question générale à travers un problème concret: la modélisation de processus physiques complexes à l'aide de l'apprentissage profond. Le deuxième objectif est en quelque sorte son dual; il concerne l'analyse des algorithmes d'apprentissage profond à travers le prisme des systèmes dynamiques et des processus physiques, dans le but d'acquérir une meilleure compréhension et de développer de nouveaux algorithmes pour ce domaine

Learning Dynamical Systems from Partial Observations

We consider the problem of forecasting complex, nonlinear space-time processes when observations provide only partial information of on the system's state. We propose a natural data-driven framework, where the system's dynamics are modelled by an unknown time-varying differential equation, and the evolution term is estimated from the data, using a neural network. Any future state can then be computed by placing the associated differential equation in an ODE solver. We first evaluate our approach on shallow water and Euler simulations. We find that our method not only demonstrates high quality long-term forecasts, but also learns to produce hidden states closely resembling the true states of the system, without direct supervision on the latter. Additional experiments conducted on challenging, state of the art ocean simulations further validate our findings, while exhibiting notable improvements over classical baselines

Unifying GANs and Score-Based Diffusion as Generative Particle Models

International audienceParticle-based deep generative models, such as gradient flows and score-based diffusion models, have recently gained traction thanks to their striking performance. Their principle of displacing particle distributions using differential equations is conventionally seen as opposed to the previously widespread generative adversarial networks (GANs), which involve training a pushforward generator network. In this paper we challenge this interpretation, and propose a novel framework that unifies particle and adversarial generative models by framing generator training as a generalization of particle models. This suggests that a generator is an optional addition to any such generative model. Consequently, integrating a generator into a score-based diffusion model and training a GAN without a generator naturally emerge from our framework. We empirically test the viability of these original models as proofs of concepts of potential applications of our framework