18 research outputs found
Fourier Neural Operator Networks: A Fast and General Solver for the Photoacoustic Wave Equation
Simulation tools for photoacoustic wave propagation have played a key role in
advancing photoacoustic imaging by providing quantitative and qualitative
insights into parameters affecting image quality. Classical methods for
numerically solving the photoacoustic wave equation relies on a fine
discretization of space and can become computationally expensive for large
computational grids. In this work, we apply Fourier Neural Operator (FNO)
networks as a fast data-driven deep learning method for solving the 2D
photoacoustic wave equation in a homogeneous medium. Comparisons between the
FNO network and pseudo-spectral time domain approach demonstrated that the FNO
network generated comparable simulations with small errors and was several
orders of magnitude faster. Moreover, the FNO network was generalizable and can
generate simulations not observed in the training data
Semi-Implicit Neural Solver for Time-dependent Partial Differential Equations
Fast and accurate solutions of time-dependent partial differential equations (PDEs) are of pivotal interest to many research fields, including physics, engineering, and biology. Generally, implicit/semi-implicit schemes are preferred over explicit ones to improve stability and correctness. However, existing semi-implicit methods are usually iterative and employ a general-purpose solver, which may be sub-optimal for a specific class of PDEs. In this paper, we propose a neural solver to learn an optimal iterative scheme in a data-driven fashion for any class of PDEs. Specifically, we modify a single iteration of a semi-implicit solver using a deep neural network. We provide theoretical guarantees for the correctness and convergence of neural solvers analogous to conventional iterative solvers. In addition to the commonly used Dirichlet boundary condition, we adopt a diffuse domain approach to incorporate a diverse type of boundary conditions, e.g., Neumann. We show that the proposed neural solver can go beyond linear PDEs and applies to a class of non-linear PDEs, where the non-linear component is non-stiff. We demonstrate the efficacy of our method on 2D and 3D scenarios. To this end, we show how our model generalizes to parameter settings, which are different from training; and achieves faster convergence than semi-implicit schemes
PDETime: Rethinking Long-Term Multivariate Time Series Forecasting from the perspective of partial differential equations
Recent advancements in deep learning have led to the development of various
models for long-term multivariate time-series forecasting (LMTF), many of which
have shown promising results. Generally, the focus has been on
historical-value-based models, which rely on past observations to predict
future series. Notably, a new trend has emerged with time-index-based models,
offering a more nuanced understanding of the continuous dynamics underlying
time series. Unlike these two types of models that aggregate the information of
spatial domains or temporal domains, in this paper, we consider multivariate
time series as spatiotemporal data regularly sampled from a continuous
dynamical system, which can be represented by partial differential equations
(PDEs), with the spatial domain being fixed. Building on this perspective, we
present PDETime, a novel LMTF model inspired by the principles of Neural PDE
solvers, following the encoding-integration-decoding operations. Our extensive
experimentation across seven diverse real-world LMTF datasets reveals that
PDETime not only adapts effectively to the intrinsic spatiotemporal nature of
the data but also sets new benchmarks, achieving state-of-the-art result
Deep Generative Models that Solve PDEs: Distributed Computing for Training Large Data-Free Models
Recent progress in scientific machine learning (SciML) has opened up the
possibility of training novel neural network architectures that solve complex
partial differential equations (PDEs). Several (nearly data free) approaches
have been recently reported that successfully solve PDEs, with examples
including deep feed forward networks, generative networks, and deep
encoder-decoder networks. However, practical adoption of these approaches is
limited by the difficulty in training these models, especially to make
predictions at large output resolutions (). Here we
report on a software framework for data parallel distributed deep learning that
resolves the twin challenges of training these large SciML models - training in
reasonable time as well as distributing the storage requirements. Our framework
provides several out of the box functionality including (a) loss integrity
independent of number of processes, (b) synchronized batch normalization, and
(c) distributed higher-order optimization methods. We show excellent
scalability of this framework on both cloud as well as HPC clusters, and report
on the interplay between bandwidth, network topology and bare metal vs cloud.
We deploy this approach to train generative models of sizes hitherto not
possible, showing that neural PDE solvers can be viably trained for practical
applications. We also demonstrate that distributed higher-order optimization
methods are faster than stochastic gradient-based methods and
provide minimal convergence drift with higher batch-size.Comment: 10 pages, 18 figure
A Deep Learning algorithm to accelerate Algebraic Multigrid methods in Finite Element solvers of 3D elliptic PDEs
Algebraic multigrid (AMG) methods are among the most efficient solvers for
linear systems of equations and they are widely used for the solution of
problems stemming from the discretization of Partial Differential Equations
(PDEs). The most severe limitation of AMG methods is the dependence on
parameters that require to be fine-tuned. In particular, the strong threshold
parameter is the most relevant since it stands at the basis of the construction
of successively coarser grids needed by the AMG methods. We introduce a novel
Deep Learning algorithm that minimizes the computational cost of the AMG method
when used as a finite element solver. We show that our algorithm requires
minimal changes to any existing code. The proposed Artificial Neural Network
(ANN) tunes the value of the strong threshold parameter by interpreting the
sparse matrix of the linear system as a black-and-white image and exploiting a
pooling operator to transform it into a small multi-channel image. We
experimentally prove that the pooling successfully reduces the computational
cost of processing a large sparse matrix and preserves the features needed for
the regression task at hand. We train the proposed algorithm on a large dataset
containing problems with a highly heterogeneous diffusion coefficient defined
in different three-dimensional geometries and discretized with unstructured
grids and linear elasticity problems with a highly heterogeneous Young's
modulus. When tested on problems with coefficients or geometries not present in
the training dataset, our approach reduces the computational time by up to 30%