30,920 research outputs found
ParticleWNN: a Novel Neural Networks Framework for Solving Partial Differential Equations
Deep neural networks (DNNs) have been widely used to solve partial
differential equations (PDEs) in recent years. In this work, a novel deep
learning-based framework named Particle Weak-form based Neural Networks
(ParticleWNN) is developed for solving PDEs in the weak form. In this
framework, the trial space is chosen as the space of DNNs, and the test space
is constructed by functions compactly supported in extremely small regions
whose centers are particles. To train the neural networks, an R-adaptive
strategy is designed to adaptively modify the radius of regions during
training. The ParticleWNN inherits the advantages of weak/variational
formulation, such as requiring less regularity of the solution and a small
number of quadrature points for computing the integrals. Moreover, due to the
special construction of the test functions, the ParticleWNN allows local
training of networks, parallel implementation, and integral calculations only
in extremely small regions. The framework is particularly desirable for solving
problems with high-dimensional and complex domains. The efficiency and accuracy
of the ParticleWNN are demonstrated with several numerical examples. The
numerical results show clear advantages of the ParticleWNN over the
state-of-the-art methods
Closed-form continuous-time neural networks
Continuous-time neural networks are a class of machine learning systems that can tackle representation learning on spatiotemporal decision-making tasks. These models are typically represented by continuous differential equations. However, their expressive power when they are deployed on computers is bottlenecked by numerical differential equation solvers. This limitation has notably slowed down the scaling and understanding of numerous natural physical phenomena such as the dynamics of nervous systems. Ideally, we would circumvent this bottleneck by solving the given dynamical system in closed form. This is known to be intractable in general. Here, we show that it is possible to closely approximate the interaction between neurons and synapses—the building blocks of natural and artificial neural networks—constructed by liquid time-constant networks efficiently in closed form. To this end, we compute a tightly bounded approximation of the solution of an integral appearing in liquid time-constant dynamics that has had no known closed-form solution so far. This closed-form solution impacts the design of continuous-time and continuous-depth neural models. For instance, since time appears explicitly in closed form, the formulation relaxes the need for complex numerical solvers. Consequently, we obtain models that are between one and five orders of magnitude faster in training and inference compared with differential equation-based counterparts. More importantly, in contrast to ordinary differential equation-based continuous networks, closed-form networks can scale remarkably well compared with other deep learning instances. Lastly, as these models are derived from liquid networks, they show good performance in time-series modelling compared with advanced recurrent neural network models
Multipole Graph Neural Operator for Parametric Partial Differential Equations
One of the main challenges in using deep learning-based methods for simulating physical systems and solving partial differential equations (PDEs) is formulating physics-based data in the desired structure for neural networks. Graph neural networks (GNNs) have gained popularity in this area since graphs offer a natural way of modeling particle interactions and provide a clear way of discretizing the continuum models. However, the graphs constructed for approximating such tasks usually ignore long-range interactions due to unfavorable scaling of the computational complexity with respect to the number of nodes. The errors due to these approximations scale with the discretization of the system, thereby not allowing for generalization under mesh-refinement. Inspired by the classical multipole methods, we purpose a novel multi-level graph neural network framework that captures interaction at all ranges with only linear complexity. Our multi-level formulation is equivalent to recursively adding inducing points to the kernel matrix, unifying GNNs with multi-resolution matrix factorization of the kernel. Experiments confirm our multi-graph network learns discretization-invariant solution operators to PDEs and can be evaluated in linear time
Multipole Graph Neural Operator for Parametric Partial Differential Equations
One of the main challenges in using deep learning-based methods for
simulating physical systems and solving partial differential equations (PDEs)
is formulating physics-based data in the desired structure for neural networks.
Graph neural networks (GNNs) have gained popularity in this area since graphs
offer a natural way of modeling particle interactions and provide a clear way
of discretizing the continuum models. However, the graphs constructed for
approximating such tasks usually ignore long-range interactions due to
unfavorable scaling of the computational complexity with respect to the number
of nodes. The errors due to these approximations scale with the discretization
of the system, thereby not allowing for generalization under mesh-refinement.
Inspired by the classical multipole methods, we propose a novel multi-level
graph neural network framework that captures interaction at all ranges with
only linear complexity. Our multi-level formulation is equivalent to
recursively adding inducing points to the kernel matrix, unifying GNNs with
multi-resolution matrix factorization of the kernel. Experiments confirm our
multi-graph network learns discretization-invariant solution operators to PDEs
and can be evaluated in linear time
Fast Neural Network Predictions from Constrained Aerodynamics Datasets
Incorporating computational fluid dynamics in the design process of jets,
spacecraft, or gas turbine engines is often challenged by the required
computational resources and simulation time, which depend on the chosen
physics-based computational models and grid resolutions. An ongoing problem in
the field is how to simulate these systems faster but with sufficient accuracy.
While many approaches involve simplified models of the underlying physics,
others are model-free and make predictions based only on existing simulation
data. We present a novel model-free approach in which we reformulate the
simulation problem to effectively increase the size of constrained pre-computed
datasets and introduce a novel neural network architecture (called a cluster
network) with an inductive bias well-suited to highly nonlinear computational
fluid dynamics solutions. Compared to the state-of-the-art in model-based
approximations, we show that our approach is nearly as accurate, an order of
magnitude faster, and easier to apply. Furthermore, we show that our method
outperforms other model-free approaches
- …