452 research outputs found
Learning the Pseudoinverse Solution to Network Weights
The last decade has seen the parallel emergence in computational neuroscience
and machine learning of neural network structures which spread the input signal
randomly to a higher dimensional space; perform a nonlinear activation; and
then solve for a regression or classification output by means of a mathematical
pseudoinverse operation. In the field of neuromorphic engineering, these
methods are increasingly popular for synthesizing biologically plausible neural
networks, but the "learning method" - computation of the pseudoinverse by
singular value decomposition - is problematic both for biological plausibility
and because it is not an online or an adaptive method. We present an online or
incremental method of computing the pseudoinverse, which we argue is
biologically plausible as a learning method, and which can be made adaptable
for non-stationary data streams. The method is significantly more
memory-efficient than the conventional computation of pseudoinverses by
singular value decomposition.Comment: 13 pages, 3 figures; in submission to Neural Network
OCReP: An Optimally Conditioned Regularization for Pseudoinversion Based Neural Training
In this paper we consider the training of single hidden layer neural networks
by pseudoinversion, which, in spite of its popularity, is sometimes affected by
numerical instability issues. Regularization is known to be effective in such
cases, so that we introduce, in the framework of Tikhonov regularization, a
matricial reformulation of the problem which allows us to use the condition
number as a diagnostic tool for identification of instability. By imposing
well-conditioning requirements on the relevant matrices, our theoretical
analysis allows the identification of an optimal value for the regularization
parameter from the standpoint of stability. We compare with the value derived
by cross-validation for overfitting control and optimisation of the
generalization performance. We test our method for both regression and
classification tasks. The proposed method is quite effective in terms of
predictivity, often with some improvement on performance with respect to the
reference cases considered. This approach, due to analytical determination of
the regularization parameter, dramatically reduces the computational load
required by many other techniques.Comment: Published on Neural Network
Parallel methods for linear systems solution in extreme learning machines: an overview
This paper aims to present an updated review of parallel algorithms for solving
square and rectangular single and double precision matrix linear systems using multi-core central
processing units and graphic processing units. A brief description of the methods for the solution
of linear systems based on operations, factorization and iterations was made. The methodology
implemented, in this article, is a documentary and it was based on the review of about 17
papers reported in the literature during the last five years (2016-2020). The disclosed findings
demonstrate the potential of parallelism to significantly decrease extreme learning machines
training times for problems with large amounts of data given the calculation of the Moore
Penrose pseudo inverse. The implementation of parallel algorithms in the calculation of the
pseudo-inverse will allow to contribute significantly in the applications of diversifying areas,
since it can accelerate the training time of the extreme learning machines with optimal results
Probabilistic Inference from Arbitrary Uncertainty using Mixtures of Factorized Generalized Gaussians
This paper presents a general and efficient framework for probabilistic
inference and learning from arbitrary uncertain information. It exploits the
calculation properties of finite mixture models, conjugate families and
factorization. Both the joint probability density of the variables and the
likelihood function of the (objective or subjective) observation are
approximated by a special mixture model, in such a way that any desired
conditional distribution can be directly obtained without numerical
integration. We have developed an extended version of the expectation
maximization (EM) algorithm to estimate the parameters of mixture models from
uncertain training examples (indirect observations). As a consequence, any
piece of exact or uncertain information about both input and output values is
consistently handled in the inference and learning stages. This ability,
extremely useful in certain situations, is not found in most alternative
methods. The proposed framework is formally justified from standard
probabilistic principles and illustrative examples are provided in the fields
of nonparametric pattern classification, nonlinear regression and pattern
completion. Finally, experiments on a real application and comparative results
over standard databases provide empirical evidence of the utility of the method
in a wide range of applications
A Theory of Networks for Appxoimation and Learning
Learning an input-output mapping from a set of examples, of the type that many neural networks have been constructed to perform, can be regarded as synthesizing an approximation of a multi-dimensional function, that is solving the problem of hypersurface reconstruction. From this point of view, this form of learning is closely related to classical approximation techniques, such as generalized splines and regularization theory. This paper considers the problems of an exact representation and, in more detail, of the approximation of linear and nolinear mappings in terms of simpler functions of fewer variables. Kolmogorov's theorem concerning the representation of functions of several variables in terms of functions of one variable turns out to be almost irrelevant in the context of networks for learning. We develop a theoretical framework for approximation based on regularization techniques that leads to a class of three-layer networks that we call Generalized Radial Basis Functions (GRBF), since they are mathematically related to the well-known Radial Basis Functions, mainly used for strict interpolation tasks. GRBF networks are not only equivalent to generalized splines, but are also closely related to pattern recognition methods such as Parzen windows and potential functions and to several neural network algorithms, such as Kanerva's associative memory, backpropagation and Kohonen's topology preserving map. They also have an interesting interpretation in terms of prototypes that are synthesized and optimally combined during the learning stage. The paper introduces several extensions and applications of the technique and discusses intriguing analogies with neurobiological data
An artificial neural network for redundant robot inverse kinematics computation
A redundant manipulator can be defined as a manipulator that has more degrees of freedom than necessary to determine the position and orientation of the end effector. Such a manipulator has dexterity, flexibility, and the ability to maneuver in presence of obstacles. One important and necessary step in utilizing a redundant robot is to relate the joint coordinates of the manipulator with the position and orientation of the end-effector. This specification is termed as the direct kinematics problem and can be written as x = f(q) where x is a vector representing the position and orientation of the end-effector, q is the Joint vector, and f is a continuous non-linear function defined by the design of the manipulator. The inverse kinematics problem can be stated as: Given a position and orientation of the end-effector, determine the joint vector that specifies this position a q = f -1(x). and orientation. That is, For non-trivial designs, f -1 cannot be expressed analytically. This paper presents a solution to the inverse kinematics problem for a redundant robot based on multilayer feed-forward artificial neural network
- …