24 research outputs found
Exploring datasets to solve partial differential equations with TensorFlow
The version of record is available online at: http://dx.doi.org/10.1007/978-3-030-57802-2_42This paper proposes a way of approximating the solution of partial differential equations (PDE) using Deep Neural Networks (DNN) based on Keras and TensorFlow, that is capable of running on a conventional laptop, which is relatively fast for different network architectures. We analyze the performance of our method using a well known PDE, the heat equation with Dirichlet boundary conditions for a non-derivable non-continuous initial function. We have tried the use of different families of functions as training datasets as well as different time spreadings aiming at the best possible performance. The code is easily modifiable and can be adapted to solve PDE problems in more complex scenarios by changing the activation functions of the different layers.This work has been partially supported by the Spanish Ministry of Science, Innovation and Universities, Gobierno de España, under Contracts No. PGC2018-093854-BI00, and ICMAT Severo Ochoa SEV-2015-0554, and from the People Programme (Marie Curie Actions) of the European Unionâs Horizon 2020 Research and Innovation Program under Grant No. 734557.Peer ReviewedPostprint (published version
Avoiding overfitting of multilayer perceptrons by training derivatives
Resistance to overfitting is observed for neural networks trained with
extended backpropagation algorithm. In addition to target values, its cost
function uses derivatives of those up to the order. For
common applications of neural networks, high order derivatives are not readily
available, so simpler cases are considered: training network to approximate
analytical function inside 2D and 5D domains and solving Poisson equation
inside a 2D circle. For function approximation, the cost is a sum of squared
differences between output and target as well as their derivatives with respect
to the input. Differential equations are usually solved by putting a multilayer
perceptron in place of unknown function and training its weights, so that
equation holds within some margin of error. Commonly used cost is the
equation's residual squared. Added terms are squared derivatives of said
residual with respect to the independent variables. To investigate overfitting,
the cost is minimized for points of regular grids with various spacing, and its
root mean is compared with its value on much denser test set. Fully connected
perceptrons with six hidden layers and , and
weights in total are trained with Rprop until cost changes by
less than 10% for last 1000 epochs, or when the epoch is
reached. Training the network with weights to represent simple
2D function using 10 points with 8 extra derivatives in each produces cost test
to train ratio of , whereas for classical backpropagation in comparable
conditions this ratio is