28,357 research outputs found
Constrained Extreme Learning Machines: A Study on Classification Cases
Extreme learning machine (ELM) is an extremely fast learning method and has a
powerful performance for pattern recognition tasks proven by enormous
researches and engineers. However, its good generalization ability is built on
large numbers of hidden neurons, which is not beneficial to real time response
in the test process. In this paper, we proposed new ways, named "constrained
extreme learning machines" (CELMs), to randomly select hidden neurons based on
sample distribution. Compared to completely random selection of hidden nodes in
ELM, the CELMs randomly select hidden nodes from the constrained vector space
containing some basic combinations of original sample vectors. The experimental
results show that the CELMs have better generalization ability than traditional
ELM, SVM and some other related methods. Additionally, the CELMs have a similar
fast learning speed as ELM.Comment: 14 pages, 6 figure, journe
A Comparison Study of Credit Card Fraud Detection: Supervised versus Unsupervised
Credit card has become popular mode of payment for both online and offline
purchase, which leads to increasing daily fraud transactions. An Efficient
fraud detection methodology is therefore essential to maintain the reliability
of the payment system. In this study, we perform a comparison study of credit
card fraud detection by using various supervised and unsupervised approaches.
Specifically, 6 supervised classification models, i.e., Logistic Regression
(LR), K-Nearest Neighbors (KNN), Support Vector Machines (SVM), Decision Tree
(DT), Random Forest (RF), Extreme Gradient Boosting (XGB), as well as 4
unsupervised anomaly detection models, i.e., One-Class SVM (OCSVM),
Auto-Encoder (AE), Restricted Boltzmann Machine (RBM), and Generative
Adversarial Networks (GAN), are explored in this study. We train all these
models on a public credit card transaction dataset from Kaggle website, which
contains 492 frauds out of 284,807 transactions. The labels of the transactions
are used for supervised learning models only. The performance of each model is
evaluated through 5-fold cross validation in terms of Area Under the Receiver
Operating Curves (AUROC). Within supervised approaches, XGB and RF obtain the
best performance with AUROC = 0.989 and AUROC = 0.988, respectively. While for
unsupervised approaches, RBM achieves the best performance with AUROC = 0.961,
followed by GAN with AUROC = 0.954. The experimental results show that
supervised models perform slightly better than unsupervised models in this
study. Anyway, unsupervised approaches are still promising for credit card
fraud transaction detection due to the insufficient annotation and the data
imbalance issue in real-world applications
Genesis of Basic and Multi-Layer Echo State Network Recurrent Autoencoders for Efficient Data Representations
It is a widely accepted fact that data representations intervene noticeably
in machine learning tools. The more they are well defined the better the
performance results are. Feature extraction-based methods such as autoencoders
are conceived for finding more accurate data representations from the original
ones. They efficiently perform on a specific task in terms of 1) high accuracy,
2) large short term memory and 3) low execution time. Echo State Network (ESN)
is a recent specific kind of Recurrent Neural Network which presents very rich
dynamics thanks to its reservoir-based hidden layer. It is widely used in
dealing with complex non-linear problems and it has outperformed classical
approaches in a number of tasks including regression, classification, etc. In
this paper, the noticeable dynamism and the large memory provided by ESN and
the strength of Autoencoders in feature extraction are gathered within an ESN
Recurrent Autoencoder (ESN-RAE). In order to bring up sturdier alternative to
conventional reservoir-based networks, not only single layer basic ESN is used
as an autoencoder, but also Multi-Layer ESN (ML-ESN-RAE). The new features,
once extracted from ESN's hidden layer, are applied to classification tasks.
The classification rates rise considerably compared to those obtained when
applying the original data features. An accuracy-based comparison is performed
between the proposed recurrent AEs and two variants of an ELM feed-forward AEs
(Basic and ML) in both of noise free and noisy environments. The empirical
study reveals the main contribution of recurrent connections in improving the
classification performance results.Comment: 13 pages, 9 figure
Identity Crisis: Memorization and Generalization under Extreme Overparameterization
We study the interplay between memorization and generalization of
overparameterized networks in the extreme case of a single training example and
an identity-mapping task. We examine fully-connected and convolutional networks
(FCN and CNN), both linear and nonlinear, initialized randomly and then trained
to minimize the reconstruction error. The trained networks stereotypically take
one of two forms: the constant function (memorization) and the identity
function (generalization). We formally characterize generalization in
single-layer FCNs and CNNs. We show empirically that different architectures
exhibit strikingly different inductive biases. For example, CNNs of up to 10
layers are able to generalize from a single example, whereas FCNs cannot learn
the identity function reliably from 60k examples. Deeper CNNs often fail, but
nonetheless do astonishing work to memorize the training output: because CNN
biases are location invariant, the model must progressively grow an output
pattern from the image boundaries via the coordination of many layers. Our work
helps to quantify and visualize the sensitivity of inductive biases to
architectural choices such as depth, kernel width, and number of channels.Comment: ICLR 202
Deep Learning with the Random Neural Network and its Applications
The random neural network (RNN) is a mathematical model for an "integrate and
fire" spiking network that closely resembles the stochastic behaviour of
neurons in mammalian brains. Since its proposal in 1989, there have been
numerous investigations into the RNN's applications and learning algorithms.
Deep learning (DL) has achieved great success in machine learning. Recently,
the properties of the RNN for DL have been investigated, in order to combine
their power. Recent results demonstrate that the gap between RNNs and DL can be
bridged and the DL tools based on the RNN are faster and can potentially be
used with less energy expenditure than existing methods.Comment: 23 pages, 19 figure
Feature Learning for Fault Detection in High-Dimensional Condition-Monitoring Signals
Complex industrial systems are continuously monitored by a large number of
heterogeneous sensors. The diversity of their operating conditions and the
possible fault types make it impossible to collect enough data for learning all
the possible fault patterns. The paper proposes an integrated automatic
unsupervised feature learning and one-class classification for fault detection
that uses data on healthy conditions only for its training. The approach is
based on stacked Extreme Learning Machines (namely Hierarchical, or HELM) and
comprises an autoencoder, performing unsupervised feature learning, stacked
with a one-class classifier monitoring the distance of the test data to the
training healthy class, thereby assessing the health of the system.
This study provides a comprehensive evaluation of HELM fault detection
capability compared to other machine learning approaches, such as stand-alone
one-class classifiers (ELM and SVM), these same one-class classifiers combined
with traditional dimensionality reduction methods (PCA) and a Deep Belief
Network. The performance is first evaluated on a synthetic dataset that
encompasses typical characteristics of condition monitoring data. Subsequently,
the approach is evaluated on a real case study of a power plant fault. The
proposed algorithm for fault detection, combining feature learning with the
one-class classifier, demonstrates a better performance, particularly in cases
where condition monitoring data contain several non-informative signals
Semi-Trained Memristive Crossbar Computing Engine with In-Situ Learning Accelerator
On-device intelligence is gaining significant attention recently as it offers
local data processing and low power consumption. In this research, an on-device
training circuitry for threshold-current memristors integrated in a crossbar
structure is proposed. Furthermore, alternate approaches of mapping the
synaptic weights into fully-trained and semi-trained crossbars are
investigated. In a semi-trained crossbar a confined subset of memristors are
tuned and the remaining subset of memristors are not programmed. This
translates to optimal resource utilization and power consumption, compared to a
fully programmed crossbar. The semi-trained crossbar architecture is applicable
to a broad class of neural networks. System level verification is performed
with an extreme learning machine for binomial and multinomial classification.
The total power for a single 4x4 layer network, when implemented in IBM 65nm
node, is estimated to be ~ 42.16uW and the area is estimated to be 26.48um x
22.35um
Restricted Boltzmann machine to determine the input weights for extreme learning machines
The Extreme Learning Machine (ELM) is a single-hidden layer feedforward
neural network (SLFN) learning algorithm that can learn effectively and
quickly. The ELM training phase assigns the input weights and bias randomly and
does not change them in the whole process. Although the network works well, the
random weights in the input layer can make the algorithm less effective and
impact on its performance. Therefore, we propose a new approach to determine
the input weights and bias for the ELM using the restricted Boltzmann machine
(RBM), which we call RBM-ELM. We compare our new approach with a well-known
approach to improve the ELM and a state of the art algorithm to select the
weights for the ELM. The results show that the RBM-ELM outperforms both
methodologies and achieve a better performance than the ELM.Comment: 14 pages, 7 figures and 5 table
Enhanced Image Classification With a Fast-Learning Shallow Convolutional Neural Network
We present a neural network architecture and training method designed to
enable very rapid training and low implementation complexity. Due to its
training speed and very few tunable parameters, the method has strong potential
for applications requiring frequent retraining or online training. The approach
is characterized by (a) convolutional filters based on biologically inspired
visual processing filters, (b) randomly-valued classifier-stage input weights,
(c) use of least squares regression to train the classifier output weights in a
single batch, and (d) linear classifier-stage output units. We demonstrate the
efficacy of the method by applying it to image classification. Our results
match existing state-of-the-art results on the MNIST (0.37% error) and
NORB-small (2.2% error) image classification databases, but with very fast
training times compared to standard deep network approaches. The network's
performance on the Google Street View House Number (SVHN) (4% error) database
is also competitive with state-of-the art methods.Comment: 7 pages, 2 figures, Paper at IJCNN 2015 (International Joint
Conference on Neural Networks, 2015
A smartphone application to detection and classification of coffee leaf miner and coffee leaf rust
Generally, the identification and classification of plant diseases and/or
pests are performed by an expert . One of the problems facing coffee farmers in
Brazil is crop infestation, particularly by leaf rust Hemileia vastatrix and
leaf miner Leucoptera coffeella. The progression of the diseases and or pests
occurs spatially and temporarily. So, it is very important to automatically
identify the degree of severity. The main goal of this article consists on the
development of a method and its i implementation as an App that allow the
detection of the foliar damages from images of coffee leaf that are captured
using a smartphone, and identify whether it is rust or leaf miner, and in turn
the calculation of its severity degree. The method consists of identifying a
leaf from the image and separates it from the background with the use of a
segmentation algorithm. In the segmentation process, various types of
backgrounds for the image using the HSV and YCbCr color spaces are tested. In
the segmentation of foliar damages, the Otsu algorithm and the iterative
threshold algorithm, in the YCgCr color space, have been used and compared to
k-means. Next, features of the segmented foliar damages are calculated. For the
classification, artificial neural network trained with extreme learning machine
have been used. The results obtained shows the feasibility and effectiveness of
the approach to identify and classify foliar damages, and the automatic
calculation of the severity. The results obtained are very promising according
to experts
- …