10,944 research outputs found
Modeling Financial Time Series with Artificial Neural Networks
Financial time series convey the decisions and actions of a population of human actors over time. Econometric and regressive models have been developed in the past decades for analyzing these time series. More recently, biologically inspired artificial neural network models have been shown to overcome some of the main challenges of traditional techniques by better exploiting the non-linear, non-stationary, and oscillatory nature of noisy, chaotic human interactions. This review paper explores the options, benefits, and weaknesses of the various forms of artificial neural networks as compared with regression techniques in the field of financial time series analysis.CELEST, a National Science Foundation Science of Learning Center (SBE-0354378); SyNAPSE program of the Defense Advanced Research Project Agency (HR001109-03-0001
Deep Learning Applied to the Asteroseismic Modeling of Stars with Coherent Oscillation Modes
We develop a novel method based on machine learning principles to achieve
optimal initiation of CPU-intensive computations for forward asteroseismic
modeling in a multi-D parameter space. A deep neural network is trained on a
precomputed asteroseismology grid containing about 62 million coherent
oscillation-mode frequencies derived from stellar evolution models. These
models are representative of the core-hydrogen burning stage of
intermediate-mass and high-mass stars. The evolution models constitute a 6D
parameter space and their predicted low-degree pressure- and gravity-mode
oscillations are scanned, using a genetic algorithm. A software pipeline is
created to find the best fitting stellar parameters for a given set of observed
oscillation frequencies. The proposed method finds the optimal regions in the
6D parameters space in less than a minute, hence providing the optimal starting
point for further and more detailed forward asteroseismic modeling in a
high-dimensional context. We test and apply the method to seven pulsating stars
that were previously modeled asteroseismically by classical grid-based forward
modeling based on a statistic and obtain good agreement with past
results. Our deep learning methodology opens up the application of
asteroseismic modeling in +6D parameter space for thousands of stars pulsating
in coherent modes with long lifetimes observed by the space telescope
and to be discovered with the TESS and PLATO space missions, while applications
so far were done star-by-star for only a handful of cases. Our method is open
source and can be used by anyone freely.Comment: Accepted for publication in PASP Speciale Volume on Machine Learnin
Comparative performance of some popular ANN algorithms on benchmark and function approximation problems
We report an inter-comparison of some popular algorithms within the
artificial neural network domain (viz., Local search algorithms, global search
algorithms, higher order algorithms and the hybrid algorithms) by applying them
to the standard benchmarking problems like the IRIS data, XOR/N-Bit parity and
Two Spiral. Apart from giving a brief description of these algorithms, the
results obtained for the above benchmark problems are presented in the paper.
The results suggest that while Levenberg-Marquardt algorithm yields the lowest
RMS error for the N-bit Parity and the Two Spiral problems, Higher Order
Neurons algorithm gives the best results for the IRIS data problem. The best
results for the XOR problem are obtained with the Neuro Fuzzy algorithm. The
above algorithms were also applied for solving several regression problems such
as cos(x) and a few special functions like the Gamma function, the
complimentary Error function and the upper tail cumulative
-distribution function. The results of these regression problems
indicate that, among all the ANN algorithms used in the present study,
Levenberg-Marquardt algorithm yields the best results. Keeping in view the
highly non-linear behaviour and the wide dynamic range of these functions, it
is suggested that these functions can be also considered as standard benchmark
problems for function approximation using artificial neural networks.Comment: 18 pages 5 figures. Accepted in Pramana- Journal of Physic
Theoretical Interpretations and Applications of Radial Basis Function Networks
Medical applications usually used Radial Basis Function Networks just as Artificial Neural Networks. However, RBFNs are Knowledge-Based Networks that can be interpreted in several way: Artificial Neural Networks, Regularization Networks, Support Vector Machines, Wavelet Networks, Fuzzy Controllers, Kernel Estimators, Instanced-Based Learners. A survey of their interpretations and of their corresponding learning algorithms is provided as well as a brief survey on dynamic learning algorithms. RBFNs' interpretations can suggest applications that are particularly interesting in medical domains
Temporal evolution of generalization during learning in linear networks
We study generalization in a simple framework of feedforward linear networks with n inputs and n outputs, trained from examples by gradient descent on the usual quadratic error function. We derive analytical results on the behavior of the validation function corresponding to the LMS error function calculated on a set of validation patterns. We show that the behavior of the validation function depends critically on the initial conditions and on the characteristics of the noise. Under certain simple assumptions, if the initial weights are sufficiently small, the validation function has a unique minimum corresponding to an optimal stopping time for training for which simple bounds can be calculated. There exists also situations where the validation function can have more complicated and somewhat unexpected behavior such as multiple local minima (at most n) of variable depth and long but finite plateau effects. Additional results and possible extensions are briefly discussed
Effect of Depth and Width on Local Minima in Deep Learning
In this paper, we analyze the effects of depth and width on the quality of
local minima, without strong over-parameterization and simplification
assumptions in the literature. Without any simplification assumption, for deep
nonlinear neural networks with the squared loss, we theoretically show that the
quality of local minima tends to improve towards the global minimum value as
depth and width increase. Furthermore, with a locally-induced structure on deep
nonlinear neural networks, the values of local minima of neural networks are
theoretically proven to be no worse than the globally optimal values of
corresponding classical machine learning models. We empirically support our
theoretical observation with a synthetic dataset as well as MNIST, CIFAR-10 and
SVHN datasets. When compared to previous studies with strong
over-parameterization assumptions, the results in this paper do not require
over-parameterization, and instead show the gradual effects of
over-parameterization as consequences of general results
A practical Bayesian framework for backpropagation networks
A quantitative and practical Bayesian framework is described for learning of mappings in feedforward networks. The framework makes possible (1) objective comparisons between solutions using alternative network architectures, (2) objective stopping rules for network pruning or growing procedures, (3) objective choice of magnitude and type of weight decay terms or additive regularizers (for penalizing large weights, etc.), (4) a measure of the effective number of well-determined parameters in a model, (5) quantified estimates of the error bars on network parameters and on network output, and (6) objective comparisons with alternative learning and interpolation models such as splines and radial basis functions. The Bayesian "evidence" automatically embodies "Occam's razor," penalizing overflexible and overcomplex models. The Bayesian approach helps detect poor underlying assumptions in learning models. For learning models well matched to a problem, a good correlation between generalization ability and the Bayesian evidence is obtained
- …