Neural Network Learning for PDEs with Oscillatory Solutions and Causal Operators

In this thesis, we focus on developing neural networks algorithms for scientific computing. First, we proposed a phase shift deep neural network (PhaseDNN), which provides a uniform wideband convergence in approximating high frequency functions and solutions of wave equations. Several linearized learning schemes have been proposed for neural networks solving nonlinear Navier-Stokes equations. We also proposed a causality deep neural network (Causality-DeepONet) to learn the causal response of a physical system. An extension of the Causality-DeepONet to time-dependent PDE systems is also proposed. The PhaseDNN makes use of the fact that common DNNs often achieve convergence in the low frequency range first, and constructs a series of moderately-sized DNNs trained for selected high frequency ranges. With the help of phase shifts in the frequency domain, each of the DNNs will be trained to approximate the function’s specific high frequency range at the speed of learning for low frequency. As a result, the proposed PhaseDNN is able to convert high frequency learning to low frequency one, allowing a uniform learning to wideband functions. To solve the stationary nonlinear Navier-Stokes(NS) equation with deep neural networks, we integrate linearization of the nonlinear convection term in the NS equation into the training process of multi-scale deep neural network (DNN) approximations of the NS solution. Four forms of linearization are considered. We solve highly oscillating stationary flows in complex domains utilizing the proposed linearized learning with multiscale neural networks. The theorem of universal approximations to nonlinear operators proposed by Chen et al. [11] is extended to operators with causalities, and the proposed Causality-DeepONet implements the physical causality in its framework. The proposed Causality-DeepONet considers causality (the state of the system at the current time is not affected by that of the future, but only by its current state and past history) and uses a convolution-type weight in its design. To demonstrate its effectiveness in handling the causal response of a physical system, the Causality-DeepONet is applied to learn the operator representing the response of a building due to earthquake ground accelerations. Finally, we proposed a deep neural network approximation to the evolution operator for time dependent PDE systems over long time period by recursively using one single neural network propagator, in the form of POD-DeepONet with built-in causality feature, for a small-time interval