13 research outputs found
Fourier Neural Network Approximation of Transition Densities in Finance
This paper introduces FourNet, a novel single-layer feed-forward neural
network (FFNN) method designed to approximate transition densities for which
closed-form expressions of their Fourier transforms, i.e. characteristic
functions, are available. A unique feature of FourNet lies in its use of a
Gaussian activation function, enabling exact Fourier and inverse Fourier
transformations and drawing analogies with the Gaussian mixture model. We
mathematically establish FourNet's capacity to approximate transition densities
in the -sense arbitrarily well with finite number of neurons. The
parameters of FourNet are learned by minimizing a loss function derived from
the known characteristic function and the Fourier transform of the FFNN,
complemented by a strategic sampling approach to enhance training. Through a
rigorous and comprehensive error analysis, we derive informative bounds for the
estimation error and the potential (pointwise) loss of nonnegativity in
the estimated densities. FourNet's accuracy and versatility are demonstrated
through a wide range of dynamics common in quantitative finance, including
L\'{e}vy processes and the Heston stochastic volatility models-including those
augmented with the self-exciting Queue-Hawkes jump process.Comment: 27 pages, 5 figure
Iterative Averaging in the Quest for Best Test Error
We analyse and explain the increased generalisation performance of iterate
averaging using a Gaussian process perturbation model between the true and
batch risk surface on the high dimensional quadratic. We derive three phenomena
\latestEdits{from our theoretical results:} (1) The importance of combining
iterate averaging (IA) with large learning rates and regularisation for
improved regularisation. (2) Justification for less frequent averaging. (3)
That we expect adaptive gradient methods to work equally well, or better, with
iterate averaging than their non-adaptive counterparts. Inspired by these
results\latestEdits{, together with} empirical investigations of the importance
of appropriate regularisation for the solution diversity of the iterates, we
propose two adaptive algorithms with iterate averaging. These give
significantly better results compared to stochastic gradient descent (SGD),
require less tuning and do not require early stopping or validation set
monitoring. We showcase the efficacy of our approach on the CIFAR-10/100,
ImageNet and Penn Treebank datasets on a variety of modern and classical
network architectures