360 research outputs found
MgNO: Efficient Parameterization of Linear Operators via Multigrid
In this work, we propose a concise neural operator architecture for operator
learning. Drawing an analogy with a conventional fully connected neural
network, we define the neural operator as follows: the output of the -th
neuron in a nonlinear operator layer is defined by . Here,
denotes the bounded linear operator connecting -th input
neuron to -th output neuron, and the bias takes the form
of a function rather than a scalar. Given its new universal approximation
property, the efficient parameterization of the bounded linear operators
between two neurons (Banach spaces) plays a critical role. As a result, we
introduce MgNO, utilizing multigrid structures to parameterize these linear
operators between neurons. This approach offers both mathematical rigor and
practical expressivity. Additionally, MgNO obviates the need for conventional
lifting and projecting operators typically required in previous neural
operators. Moreover, it seamlessly accommodates diverse boundary conditions.
Our empirical observations reveal that MgNO exhibits superior ease of training
compared to other CNN-based models, while also displaying a reduced
susceptibility to overfitting when contrasted with spectral-type neural
operators. We demonstrate the efficiency and accuracy of our method with
consistently state-of-the-art performance on different types of partial
differential equations (PDEs)
Mitigating spectral bias for the multiscale operator learning with hierarchical attention
Neural operators have emerged as a powerful tool for learning the mapping
between infinite-dimensional parameter and solution spaces of partial
differential equations (PDEs). In this work, we focus on multiscale PDEs that
have important applications such as reservoir modeling and turbulence
prediction. We demonstrate that for such PDEs, the spectral bias towards
low-frequency components presents a significant challenge for existing neural
operators. To address this challenge, we propose a hierarchical attention
neural operator (HANO) inspired by the hierarchical matrix approach. HANO
features a scale-adaptive interaction range and self-attentions over a
hierarchy of levels, enabling nested feature computation with controllable
linear cost and encoding/decoding of multiscale solution space. We also
incorporate an empirical loss function to enhance the learning of
high-frequency components. Our numerical experiments demonstrate that HANO
outperforms state-of-the-art (SOTA) methods for representative multiscale
problems
- …