Are probabilistic spiking neural networks suitable for reservoir computing?

This study employs networks of stochastic spiking neurons as reservoirs for liquid state machines (LSM). We experimentally investigate the separation property of these reservoirs and show their ability to generalize classes of input signals. Similar to traditional LSM, probabilistic LSM (pLSM) have the separation property enabling them to distinguish between different classes of input stimuli. Furthermore, our results indicate some potential advantages of non-deterministic LSM by improving upon the separation ability of the liquid. Three non-deterministic neural models are considered and for each of them several parameter configurations are explored. We demonstrate some of the characteristics of pLSM and compare them to their deterministic counterparts. pLSM offer more flexibility due to the probabilistic parameters resulting in a better performance for some values of these parameters

Liquid State Machine with Dendritically Enhanced Readout for Low-power, Neuromorphic VLSI Implementations

In this paper, we describe a new neuro-inspired, hardware-friendly readout stage for the liquid state machine (LSM), a popular model for reservoir computing. Compared to the parallel perceptron architecture trained by the p-delta algorithm, which is the state of the art in terms of performance of readout stages, our readout architecture and learning algorithm can attain better performance with significantly less synaptic resources making it attractive for VLSI implementation. Inspired by the nonlinear properties of dendrites in biological neurons, our readout stage incorporates neurons having multiple dendrites with a lumped nonlinearity. The number of synaptic connections on each branch is significantly lower than the total number of connections from the liquid neurons and the learning algorithm tries to find the best 'combination' of input connections on each branch to reduce the error. Hence, the learning involves network rewiring (NRW) of the readout network similar to structural plasticity observed in its biological counterparts. We show that compared to a single perceptron using analog weights, this architecture for the readout can attain, even by using the same number of binary valued synapses, up to 3.3 times less error for a two-class spike train classification problem and 2.4 times less error for an input rate approximation task. Even with 60 times larger synapses, a group of 60 parallel perceptrons cannot attain the performance of the proposed dendritically enhanced readout. An additional advantage of this method for hardware implementations is that the 'choice' of connectivity can be easily implemented exploiting address event representation (AER) protocols commonly used in current neuromorphic systems where the connection matrix is stored in memory. Also, due to the use of binary synapses, our proposed method is more robust against statistical variations.Comment: 14 pages, 19 figures, Journa

Extreme sparse multinomial logistic regression : a fast and robust framework for hyperspectral image classification

Although the sparse multinomial logistic regression (SMLR) has provided a useful tool for sparse classification, it suffers from inefficacy in dealing with high dimensional features and manually set initial regressor values. This has significantly constrained its applications for hyperspectral image (HSI) classification. In order to tackle these two drawbacks, an extreme sparse multinomial logistic regression (ESMLR) is proposed for effective classification of HSI. First, the HSI dataset is projected to a new feature space with randomly generated weight and bias. Second, an optimization model is established by the Lagrange multiplier method and the dual principle to automatically determine a good initial regressor for SMLR via minimizing the training error and the regressor value. Furthermore, the extended multi-attribute profiles (EMAPs) are utilized for extracting both the spectral and spatial features. A combinational linear multiple features learning (MFL) method is proposed to further enhance the features extracted by ESMLR and EMAPs. Finally, the logistic regression via the variable splitting and the augmented Lagrangian (LORSAL) is adopted in the proposed framework for reducing the computational time. Experiments are conducted on two well-known HSI datasets, namely the Indian Pines dataset and the Pavia University dataset, which have shown the fast and robust performance of the proposed ESMLR framework

On Improving the Computing Capacity of Dynamical Systems

Reservoir Computing has emerged as a practical approach for solving temporal pattern recognition problems. The procedure of preparing the system for pattern recognition is simple, provided that the dynamical system (reservoir) used for computation is complex enough. However, to achieve a sufficient reservoir complexity, one has to use many interacting elements. We propose a novel method to reduce the number of reservoir elements without reducing the computing capacity of the device. It is shown that if an auxiliary input channel can be engineered, the drive, advantageous correlations between the signal one wishes to analyse and the state of the reservoir can emerge, increasing the intelligence of the system. The method has been illustrated on the problem of electrocardiogram (ECG) signal classification. By using a reservoir with only one element, and an optimised drive, more than 93% of the signals have been correctly labelled

Extreme sparse multinomial logistic regression: a fast and robust framework for hyperspectral image classification.

Although sparse multinomial logistic regression (SMLR) has provided a useful tool for sparse classification, it suffers from inefficacy in dealing with high dimensional features and manually set initial regressor values. This has significantly constrained its applications for hyperspectral image (HSI) classification. In order to tackle these two drawbacks, an extreme sparse multinomial logistic regression (ESMLR) is proposed for effective classification of HSI. First, the HSI dataset is projected to a new feature space with randomly generated weight and bias. Second, an optimization model is established by the Lagrange multiplier method and the dual principle to automatically determine a good initial regressor for SMLR via minimizing the training error and the regressor value. Furthermore, the extended multi-attribute profiles (EMAPs) are utilized for extracting both the spectral and spatial features. A combinational linear multiple features learning (MFL) method is proposed to further enhance the features extracted by ESMLR and EMAPs. Finally, the logistic regression via the variable splitting and the augmented Lagrangian (LORSAL) is adopted in the proposed framework for reducing the computational time. Experiments are conducted on two well-known HSI datasets, namely the Indian Pines dataset and the Pavia University dataset, which have shown the fast and robust performance of the proposed ESMLR framework

Máquina de estado líquido para previsão de séries temporais contínuas: aplicação na demanda de energia elétrica

Among of several aspects of the natural intelligence is its ability to process temporal information. One of major challenges to be addresses is how to efficiently develop intelligent systems that integrate the complexities of human behavior. In this context, appear the Liquid State Machines (LSMs), a pulsed neural architecture (liquid) that projects the input data in a high-dimensional dynamical space and therefore makes the analysis of input data all through a classical neural network (readout). Thus, this thesis presents an innovative solution for forecasting continuous time series through LSMs with reset mechanism and analog inputs, applied to the electric energy demand. The methodology was applied in the short-term and long-term forecasting of electrical energy demand. Results are promising, considering the high error to stop training the readout, the low number of iterations of training of the readout, and that no strategy of seasonal adjustment or preprocessing of input data was achieved. So far, it can be notice that the LSMs have been studied as a new and promising approach in the Artificial Neural Networks paradigm, emergent from cognitive science.CAPESUm dos aspectos fundamentais da inteligência natural é sua aptidão no processamento de informações temporais. O grande desafio proposto é o de desenvolver sistemas inteligentes que mapeiem essa aptidão do comportamento humano. Neste contexto, aportam as Máquinas de Estado Líquido (LSMs), uma arquitetura neural pulsada (meio líquido) que projeta os dados de entrada em um espaço dinâmico de alta dimensão e, por conseguinte, realiza a análise do conjunto de dados de entrada através de uma rede neural clássica (unidade de leitura). Desta maneira, esta tese apresenta uma solução inovadora para a previsão de séries temporais contínuas através das LSMs com mecanismo de reinicialização e entradas analógicas, contemplando a área da demanda de energia elétrica. A metodologia desenvolvida foi aplicada no horizonte de previsão a curto prazo e a longo prazo. Os resultados obtidos são promissores, considerando o alto erro estabelecido para parada do treinamento da unidade de leitura, o baixo número de iterações do treinamento da unidade de leitura e que nenhuma estratégia de ajustamento sazonal, ou pré-processamento, sob os dados de entrada foi realizado. Até o momento, percebe-se que as LSMs têm despontado como uma nova e promissora abordagem dentro do paradigma conexionista, emergente da ciência cognitiva