27 research outputs found
Spatio-temporal Learning with Arrays of Analog Nanosynapses
Emerging nanodevices such as resistive memories are being considered for
hardware realizations of a variety of artificial neural networks (ANNs),
including highly promising online variants of the learning approaches known as
reservoir computing (RC) and the extreme learning machine (ELM). We propose an
RC/ELM inspired learning system built with nanosynapses that performs both
on-chip projection and regression operations. To address time-dynamic tasks,
the hidden neurons of our system perform spatio-temporal integration and can be
further enhanced with variable sampling or multiple activation windows. We
detail the system and show its use in conjunction with a highly analog
nanosynapse device on a standard task with intrinsic timing dynamics- the TI-46
battery of spoken digits. The system achieves nearly perfect (99%) accuracy at
sufficient hidden layer size, which compares favorably with software results.
In addition, the model is extended to a larger dataset, the MNIST database of
handwritten digits. By translating the database into the time domain and using
variable integration windows, up to 95% classification accuracy is achieved. In
addition to an intrinsically low-power programming style, the proposed
architecture learns very quickly and can easily be converted into a spiking
system with negligible loss in performance- all features that confer
significant energy efficiency.Comment: 6 pages, 3 figures. Presented at 2017 IEEE/ACM Symposium on Nanoscale
architectures (NANOARCH
An Adaptive Locally Connected Neuron Model: Focusing Neuron
This paper presents a new artificial neuron model capable of learning its
receptive field in the topological domain of inputs. The model provides
adaptive and differentiable local connectivity (plasticity) applicable to any
domain. It requires no other tool than the backpropagation algorithm to learn
its parameters which control the receptive field locations and apertures. This
research explores whether this ability makes the neuron focus on informative
inputs and yields any advantage over fully connected neurons. The experiments
include tests of focusing neuron networks of one or two hidden layers on
synthetic and well-known image recognition data sets. The results demonstrated
that the focusing neurons can move their receptive fields towards more
informative inputs. In the simple two-hidden layer networks, the focusing
layers outperformed the dense layers in the classification of the 2D spatial
data sets. Moreover, the focusing networks performed better than the dense
networks even when 70 of the weights were pruned. The tests on
convolutional networks revealed that using focusing layers instead of dense
layers for the classification of convolutional features may work better in some
data sets.Comment: 45 pages, a national patent filed, submitted to Turkish Patent
Office, No: -2017/17601, Date: 09.11.201
Benchmarking different deep regression models for predicting image rotation angle and robot’s end effector’s position
Deep visual regression models have an important role to find how much the learning model fits the relationship between the visual data (images) and the predicted continuous output. Recently, deep visual regression has been utilized in different applications such as age prediction, digital holography, and head-pose estimation. Deep learning has recently been cutting-edge research. Most of the research papers have focused on utilizing deep learning in classification tasks. There is still a lack of research that use deep learning for regression. This paper utilizes different deep learning models for two regression tasks. The first one is the prediction of the image rotation angle. The second task is to predict the position of the robot’s end-effector in 2D space. Efficient features were learned or extracted in order to perform good regression. The paper demonstrates and compares various models such as a local Receptive Field-Extreme Learning Machine (LRF-ELM), Hierarchical ELM, Supervised Convolutional Neural Network (CNN), and pre-trained CNN such as AlexNet. Each model was trained to learn or extract features and map them to specific continuous output. The results show that all models gave good performance in terms of RMSE and accuracy. H-ELM was found to outperform other models in term of training speed
Aero Engine Fault Diagnosis Using an Optimized Extreme Learning Machine
A new extreme learning machine optimized by quantum-behaved particle swarm optimization (QPSO) is developed in this paper. It uses QPSO to select optimal network parameters including the number of hidden layer neurons according to both the root mean square error on validation data set and the norm of output weights. The proposed Q-ELM was applied to real-world classification applications and a gas turbine fan engine diagnostic problem and was compared with two other optimized ELM methods and original ELM, SVM, and BP method. Results show that the proposed Q-ELM is a more reliable and suitable method than conventional neural network and other ELM methods for the defect diagnosis of the gas turbine engine