1,416 research outputs found
OCReP: An Optimally Conditioned Regularization for Pseudoinversion Based Neural Training
In this paper we consider the training of single hidden layer neural networks
by pseudoinversion, which, in spite of its popularity, is sometimes affected by
numerical instability issues. Regularization is known to be effective in such
cases, so that we introduce, in the framework of Tikhonov regularization, a
matricial reformulation of the problem which allows us to use the condition
number as a diagnostic tool for identification of instability. By imposing
well-conditioning requirements on the relevant matrices, our theoretical
analysis allows the identification of an optimal value for the regularization
parameter from the standpoint of stability. We compare with the value derived
by cross-validation for overfitting control and optimisation of the
generalization performance. We test our method for both regression and
classification tasks. The proposed method is quite effective in terms of
predictivity, often with some improvement on performance with respect to the
reference cases considered. This approach, due to analytical determination of
the regularization parameter, dramatically reduces the computational load
required by many other techniques.Comment: Published on Neural Network
Parallel methods for linear systems solution in extreme learning machines: an overview
This paper aims to present an updated review of parallel algorithms for solving
square and rectangular single and double precision matrix linear systems using multi-core central
processing units and graphic processing units. A brief description of the methods for the solution
of linear systems based on operations, factorization and iterations was made. The methodology
implemented, in this article, is a documentary and it was based on the review of about 17
papers reported in the literature during the last five years (2016-2020). The disclosed findings
demonstrate the potential of parallelism to significantly decrease extreme learning machines
training times for problems with large amounts of data given the calculation of the Moore
Penrose pseudo inverse. The implementation of parallel algorithms in the calculation of the
pseudo-inverse will allow to contribute significantly in the applications of diversifying areas,
since it can accelerate the training time of the extreme learning machines with optimal results
ADEPOS: Anomaly Detection based Power Saving for Predictive Maintenance using Edge Computing
In industry 4.0, predictive maintenance(PM) is one of the most important
applications pertaining to the Internet of Things(IoT). Machine learning is
used to predict the possible failure of a machine before the actual event
occurs. However, the main challenges in PM are (a) lack of enough data from
failing machines, and (b) paucity of power and bandwidth to transmit sensor
data to cloud throughout the lifetime of the machine. Alternatively, edge
computing approaches reduce data transmission and consume low energy. In this
paper, we propose Anomaly Detection based Power Saving(ADEPOS) scheme using
approximate computing through the lifetime of the machine. In the beginning of
the machines life, low accuracy computations are used when the machine is
healthy. However, on the detection of anomalies, as time progresses, the system
is switched to higher accuracy modes. We show using the NASA bearing dataset
that using ADEPOS, we need 8.8X less neurons on average and based on
post-layout results, the resultant energy savings are 6.4 to 6.65XComment: Submitted to ASP-DAC 2019, Japa
Hamiltonian Monte Carlo Acceleration Using Surrogate Functions with Random Bases
For big data analysis, high computational cost for Bayesian methods often
limits their applications in practice. In recent years, there have been many
attempts to improve computational efficiency of Bayesian inference. Here we
propose an efficient and scalable computational technique for a
state-of-the-art Markov Chain Monte Carlo (MCMC) methods, namely, Hamiltonian
Monte Carlo (HMC). The key idea is to explore and exploit the structure and
regularity in parameter space for the underlying probabilistic model to
construct an effective approximation of its geometric properties. To this end,
we build a surrogate function to approximate the target distribution using
properly chosen random bases and an efficient optimization process. The
resulting method provides a flexible, scalable, and efficient sampling
algorithm, which converges to the correct target distribution. We show that by
choosing the basis functions and optimization process differently, our method
can be related to other approaches for the construction of surrogate functions
such as generalized additive models or Gaussian process models. Experiments
based on simulated and real data show that our approach leads to substantially
more efficient sampling algorithms compared to existing state-of-the art
methods
- …