Supervised learning in Spiking Neural Networks with Limited Precision: SNN/LP

A new supervised learning algorithm, SNN/LP, is proposed for Spiking Neural Networks. This novel algorithm uses limited precision for both synaptic weights and synaptic delays; 3 bits in each case. Also a genetic algorithm is used for the supervised training. The results are comparable or better than previously published work. The results are applicable to the realization of large scale hardware neural networks. One of the trained networks is implemented in programmable hardware.Comment: 7 pages, originally submitted to IJCNN 201

The Hasty Wisdom of the Mob: How Market Sentiment Predicts Stock Market Behavior

We explore the ability of sentiment metrics, extracted from micro-blogging sites, to predict stock markets. We also address sentiments’ predictive time-horizons. The data concern bloggers’ feelings about five major stocks. Taking independent bullish and bearish sentiment metrics, granular to two minute intervals, we model their ability to forecast stock price direction, volatility, and traded volume. We find evidence of a causal link from sentiments to stock price returns, volatility and volume. The predictive time-horizon is minutes, rather than hours or days. We argue that diverse and high volume sentiment is more predictive of price volatility and traded volume than near-consensus is predictive of price direction. Causality is ephemeral. In this sense, the crowd is more a hasty mob than a source of wisdom

Training Algorithms for Networks of Spiking Neurons

Neural networks represent a type of computing that is based on the way that the brain performs computations. Neural networks are good at fitting non-linear functions and recognizing patterns. It is believed that biological neurons work similar to spiking neurons that process temporal information. In 2002, Bohte derived a backpropagation training algorithm (dubbbed as SpikeProp) for spiking neural networks (SNNs) containing temporal information as firing time of first spike. SpikeProp algorithm and its different variations were subject of many publications in the last decade. SpikeProp algorithm works for continuous weight SNNs. Implementing continuous parameters on hardware is a difficult task. On the other hand implementing digital logic on hardware is more straightforward because of many available tools. Training SNN with discrete weights is tricky because smallest change allowed in weights is a discrete step. And this discrete step might affect the accuracy of the network by huge amount. Previous works have been done for Artificial Neural Networks (ANNs) with discrete weights but there is no research in the area of training SNNs with discrete weights. New algorithms have been proposed as part of this thesis work. These algorithms work well for training discrete weights in a spiking neural network. These new algorithms use SpikeProp algorithm for choosing weights that are to be updated. Several standard classification datasets have been used to demonstrate the efficacy of proposed algorithms. It is shown that one of the proposed algorithms (Multiple Weights Multiple Steps) takes less execution time to train and the results are comparable to continuous weight SNNs in terms of accuracy

A Predictive Fuzzy-Neural Autopilot for the Guidance of Small Motorised Marine Craft

This thesis investigates the design and evaluation of a control system, that is able to adapt quickly to changes in environment and steering characteristics. This type of controller is particularly suited for applications with wide-ranging working conditions such as those experienced by small motorised craft. A small motorised craft is assumed to be highly agile and prone to disturbances, being thrown off-course very easily when travelling at high speed 'but rather heavy and sluggish at low speeds. Unlike large vessels, the steering characteristics of the craft will change tremendously with a change in forward speed. Any new design of autopilot needs to be to compensate for these changes in dynamic characteristics to maintain near optimal levels of performance. This study identities the problems that need to be overcome and the variables involved. A self-organising fuzzy logic controller is developed and tested in simulation. This type of controller learns on-line but has certain performance limitations. The major original contribution of this research investigation is the development of an improved self-adaptive and predictive control concept, the Predictive Self-organising Fuzzy Logic Controller (PSoFLC). The novel feature of the control algorithm is that is uses a neural network as a predictive simulator of the boat's future response and this network is then incorporated into the control loop to improve the course changing, as well as course keeping capabilities of the autopilot investigated. The autopilot is tested in simulation to validate the working principle of the concept and to demonstrate the self-tuning of the control parameters. Further work is required to establish the suitability of the proposed novel concept to other control

Dynamics of neural systems with time delays

Complex networks are ubiquitous in nature. Numerous neurological diseases, such as Alzheimer's, Parkinson's, epilepsy are caused by the abnormal collective behaviour of neurons in the brain. In particular, there is a strong evidence that Parkinson's disease is caused by the synchronisation of neurons, and understanding how and why such synchronisation occurs will bring scientists closer to the design and implementation of appropriate control to support desynchronisation required for the normal functioning of the brain. In order to study the emergence of (de)synchronisation, it is necessary first to understand how the dynamical behaviour of the system under consideration depends on the changes in systems parameters. This can be done using a powerful mathematical method, called bifurcation analysis, which allows one to identify and classify different dynamical regimes, such as, for example, stable/unstable steady states, Hopf and fold bifurcations, and find periodic solutions by varying parameters of the nonlinear system. In real-world systems, interactions between elements do not happen instantaneously due to a finite time of signal propagation, reaction times of individual elements, etc. Moreover, time delays are normally non-constant and may vary with time. This means that it is vital to introduce time delays in any realistic model of neural networks. In this thesis, I consider four different models. First, in order to analyse the fundamental properties of neural networks with time-delayed connections, I consider a system of four coupled nonlinear delay differential equations. This model represents a neural network, where one subsystem receives a delayed input from another subsystem. The exciting feature of this model is the combination of both discrete and distributed time delays, where distributed time delays represent the neural feedback between the two sub-systems, and the discrete delays describe neural interactions within each of the two subsystems. Stability properties are investigated for different commonly used distribution kernels, and the results are compared to the corresponding stability results for networks with no distributed delays. It is shown how approximations to the boundary of stability region of an equilibrium point can be obtained analytically for the cases of delta, uniform, and gamma delay distributions. Numerical techniques are used to investigate stability properties of the fully nonlinear system and confirm our analytical findings. In the second part of this thesis, I consider a globally coupled network composed of active (oscillatory) and inactive (non-oscillatory) oscillators with distributed time delayed coupling. Analytical conditions for the amplitude death, where the oscillations are quenched, are obtained in terms of the coupling strength, the ratio of inactive oscillators, the width of the uniformly distributed delay and the mean time delay for gamma distribution. The results show that for uniform distribution, by increasing both the width of the delay distribution and the ratio of inactive oscillators, the amplitude death region increases in the mean time delay and the coupling strength parameter space. In the case of the gamma distribution kernel, we find the amplitude death region in the space of the ratio of inactive oscillators, the mean time delay for gamma distribution, and the coupling strength for both weak and strong gamma distribution kernels. Furthermore, I analyse a model of the subthalamic nucleus (STN)-globus palidus (GP) network with three different transmission delays. A time-shift transformation reduces the model to a system with two time delays, for which the existence of a unique steady state is established. Conditions for stability of the steady state are derived in terms of system parameters and the time delays. Numerical stability analysis is performed using traceDDE and DDE-BIFTOOL in Matlab to investigate different dynamical regimes in the STN-GP model, and to obtain critical stability boundaries separating stable (healthy) and oscillatory (Parkinsonian-like) neural ring. Direct numerical simulations of the fully nonlinear system are performed to confirm analytical findings, and to illustrate different dynamical behaviours of the system. Finally, I consider a ring of n neurons coupled through the discrete and distributed time delays. I show that the amplitude death occurs in the symmetric (asymmetric) region depending on the even (odd) number of neurons in the ring neural system. Analytical conditions for linear stability of the trivial steady state are represented in a parameter space of the synaptic weight of the self-feedback and the coupling strength between the connected neurons, as well as in the space of the delayed self-feedback and the coupling strength between the neurons. It is shown that both Hopf and steady-state bifurcations may occur when the steady state loses its stability. Stability properties are also investigated for different commonly used distribution kernels, such as delta function and weak gamma distributions. Moreover, the obtained analytical results are confirmed by the numerical simulations of the fully nonlinear system