593 research outputs found
Encog: Library of Interchangeable Machine Learning Models for Java and C#
This paper introduces the Encog library for Java and C#, a scalable,
adaptable, multiplatform machine learning framework that was 1st released in
2008. Encog allows a variety of machine learning models to be applied to
datasets using regression, classification, and clustering. Various supported
machine learning models can be used interchangeably with minimal recoding.
Encog uses efficient multithreaded code to reduce training time by exploiting
modern multicore processors. The current version of Encog can be downloaded
from http://www.encog.org
Training feedforward neural networks using orthogonal iteration of the Hessian eigenvectors
Introduction
Training algorithms for Multilayer Perceptrons optimize the set of W weights and biases, w, so as to minimize an
error function, E, applied to a set of N training patterns. The well-known back propagation algorithm combines an
efficient method of estimating the gradient of the error function in weight space, DE=g, with a simple gradient
descent procedure to adjust the weights, Dw = -hg. More efficient algorithms maintain the gradient estimation
procedure, but replace the update step with a faster non-linear optimization strategy [1].
Efficient non-linear optimization algorithms are based upon second order approximation [2]. When sufficiently
close to a minimum the error surface is approximately quadratic, the shape being determined by the Hessian matrix.
Bishop [1] presents a detailed discussion of the properties and significance of the Hessian matrix. In principle, if
sufficiently close to a minimum it is possible to move directly to the minimum using the Newton step, -H-1g.
In practice, the Newton step is not used as H-1 is very expensive to evaluate; in addition, when not sufficiently close
to a minimum, the Newton step may cause a disastrously poor step to be taken. Second order algorithms either build
up an approximation to H-1, or construct a search strategy that implicitly exploits its structure without evaluating it;
they also either take precautions to prevent steps that lead to a deterioration in error, or explicitly reject such steps.
In applying non-linear optimization algorithms to neural networks, a key consideration is the high-dimensional
nature of the search space. Neural networks with thousands of weights are not uncommon. Some algorithms have
O(W2) or O(W3) memory or execution times, and are hence impracticable in such cases. It is desirable to identify
algorithms that have limited memory requirements, particularly algorithms where one may trade memory usage
against convergence speed.
The paper describes a new training algorithm that has scalable memory requirements, which may range from O(W)
to O(W2), although in practice the useful range is limited to lower complexity levels. The algorithm is based upon a
novel iterative estimation of the principal eigen-subspace of the Hessian, together with a quadratic step estimation
procedure.
It is shown that the new algorithm has convergence time comparable to conjugate gradient descent, and may be
preferable if early stopping is used as it converges more quickly during the initial phases.
Section 2 overviews the principles of second order training algorithms. Section 3 introduces the new algorithm.
Second 4 discusses some experiments to confirm the algorithm's performance; section 5 concludes the paper
Psychology well- being at workplaces
Psychological well-being indicates the existence of positive relationships with others, personal personality, autonomy, meaning in life, and personal growth and development. According to Ronald (2017), psychology well-being is a balance between challenging life events and rewarding, and is achieved through a balance of affected life events
Intrusion Detection Systems Using Adaptive Regression Splines
Past few years have witnessed a growing recognition of intelligent techniques
for the construction of efficient and reliable intrusion detection systems. Due
to increasing incidents of cyber attacks, building effective intrusion
detection systems (IDS) are essential for protecting information systems
security, and yet it remains an elusive goal and a great challenge. In this
paper, we report a performance analysis between Multivariate Adaptive
Regression Splines (MARS), neural networks and support vector machines. The
MARS procedure builds flexible regression models by fitting separate splines to
distinct intervals of the predictor variables. A brief comparison of different
neural network learning algorithms is also given
О нейросетевом подходе решения дифференциальных уравнений
Рассматривается задача решения уравнений математической физики с использованием нейронных сетей. Приведены существующие подходы к решению уравнений нейросетевым методом. Разработана универсальная архитектура нейронной сети, имеющая преимущества по сравнению с существующими подходами. Приведены данные о погрешности и скорости вычислени
Long-term prediction of discharges in Manwan Reservoir using artificial neural network models
Author name used in this publication: Kwokwing Chau2004-2005 > Academic research: refereed > Publication in refereed journalAccepted ManuscriptPublishe
- …