Random Neural Networks and Optimisation

In this thesis we introduce new models and learning algorithms for the Random Neural Network (RNN), and we develop RNN-based and other approaches for the solution of emergency management optimisation problems. With respect to RNN developments, two novel supervised learning algorithms are proposed. The first, is a gradient descent algorithm for an RNN extension model that we have introduced, the RNN with synchronised interactions (RNNSI), which was inspired from the synchronised firing activity observed in brain neural circuits. The second algorithm is based on modelling the signal-flow equations in RNN as a nonnegative least squares (NNLS) problem. NNLS is solved using a limited-memory quasi-Newton algorithm specifically designed for the RNN case. Regarding the investigation of emergency management optimisation problems, we examine combinatorial assignment problems that require fast, distributed and close to optimal solution, under information uncertainty. We consider three different problems with the above characteristics associated with the assignment of emergency units to incidents with injured civilians (AEUI), the assignment of assets to tasks under execution uncertainty (ATAU), and the deployment of a robotic network to establish communication with trapped civilians (DRNCTC). AEUI is solved by training an RNN tool with instances of the optimisation problem and then using the trained RNN for decision making; training is achieved using the developed learning algorithms. For the solution of ATAU problem, we introduce two different approaches. The first is based on mapping parameters of the optimisation problem to RNN parameters, and the second on solving a sequence of minimum cost flow problems on appropriately constructed networks with estimated arc costs. For the exact solution of DRNCTC problem, we develop a mixed-integer linear programming formulation, which is based on network flows. Finally, we design and implement distributed heuristic algorithms for the deployment of robots when the civilian locations are known or uncertain

Encoding strategies and mechanisms underpinning adaptation to stimulus statistics in the rat barrel cortex

It is well established that, following adaptation, cells adjust their sensitivity to reflect the global stimulus conditions. Two recent studies in guinea pig inferior colliculus (IC, Dean, Harper & McAlpine 2005) and rat barrel cortex (Garcia-Lazaro, Ho, Nair & Schnupp 2007) found that neural stimulus-response functions were displaced laterally in a manner that was dependent on the mean adapting stimulus. However, the direction of gain change, following adaptation to variance, was in contradiction to Information Theory, which predicts a decrease in gain with increased stimulus variance. On further analysis of the experimental data, presented within this thesis, it was revealed that the adaptive gain changes to global stimulus variance were, in fact, in the direction predicted by Information Theory. However, following adaptation to global mean amplitude, neural threshold was displaced to centre the SRF on inputs that were located on the edge of the stimulus distribution. It was found that adaptation scaled neural output such that the relationship between firing rate and local, as opposed to global, differences in stimulus amplitude was maintained; with the majority of cells responding to large differences in stimulus amplitude, on the 40ms scale. A small majority of cells responded to step-size differences, in amplitude, of either direction and were classed as novelty preferring. Adaptation to global mean was replicated in model neuron with spike-rate adaptation and tonic inhibition, which increased with stimulus mean. Adaptation to stimulus variance was replicated in three models 1: By increasing, in proportion to stimulus variance, background, excitatory and inhibitory firing rates in a balanced manner (Chance, Abbott & Reyes 2002), 2: A model of asymmetric synaptic depression (Chelaru & Dragoi 2008) and 3: a model combining non-linear input with synaptic depression. The results presented, within this thesis, demonstrate that neurons change their coding strategies depending upon the global levels of mean and variance within the sensory input. Under low noise conditions, neurons act as deviation detectors, i.e. are primed to respond to large changes in the stimulus on the tens of millisecond; however, under conditions of increased noise switch their encoding strategy in order to compute the full range of the stimulus distribution through adjusting neural gain.EPSR

Evolutionary robotics in high altitude wind energy applications

Recent years have seen the development of wind energy conversion systems that can exploit the superior wind resource that exists at altitudes above current wind turbine technology. One class of these systems incorporates a flying wing tethered to the ground which drives a winch at ground level. The wings often resemble sports kites, being composed of a combination of fabric and stiffening elements. Such wings are subject to load dependent deformation which makes them particularly difficult to model and control. Here we apply the techniques of evolutionary robotics i.e. evolution of neural network controllers using genetic algorithms, to the task of controlling a steerable kite. We introduce a multibody kite simulation that is used in an evolutionary process in which the kite is subject to deformation. We demonstrate how discrete time recurrent neural networks that are evolved to maximise line tension fly the kite in repeated looping trajectories similar to those seen using other methods. We show that these controllers are robust to limited environmental variation but show poor generalisation and occasional failure even after extended evolution. We show that continuous time recurrent neural networks (CTRNNs) can be evolved that are capable of flying appropriate repeated trajectories even when the length of the flying lines are changing. We also show that CTRNNs can be evolved that stabilise kites with a wide range of physical attributes at a given position in the sky, and systematically add noise to the simulated task in order to maximise the transferability of the behaviour to a real world system. We demonstrate how the difficulty of the task must be increased during the evolutionary process to deal with this extreme variability in small increments. We describe the development of a real world testing platform on which the evolved neurocontrollers can be tested

Self-organisation of internal models in autonomous robots

Internal Models (IMs) play a significant role in autonomous robotics. They are mechanisms able to represent the input-output characteristics of the sensorimotor loop. In developmental robotics, open-ended learning of skills and knowledge serves the purpose of reaction to unexpected inputs, to explore the environment and to acquire new behaviours. The development of the robot includes self-exploration of the state-action space and learning of the environmental dynamics. In this dissertation, we explore the properties and benefits of the self-organisation of robot behaviour based on the homeokinetic learning paradigm. A homeokinetic robot explores the environment in a coherent way without prior knowledge of its configuration or the environment itself. First, we propose a novel approach to self-organisation of behaviour by artificial curiosity in the sensorimotor loop. Second, we study how different forward models settings alter the behaviour of both exploratory and goal-oriented robots. Diverse complexity, size and learning rules are compared to assess the importance in the robot’s exploratory behaviour. We define the self-organised behaviour performance in terms of simultaneous environment coverage and best prediction of future sensori inputs. Among the findings, we have encountered that models with a fast response and a minimisation of the prediction error by local gradients achieve the best performance. Third, we study how self-organisation of behaviour can be exploited to learn IMs for goal-oriented tasks. An IM acquires coherent self-organised behaviours that are then used to achieve high-level goals by reinforcement learning (RL). Our results demonstrate that learning of an inverse model in this context yields faster reward maximisation and a higher final reward. We show that an initial exploration of the environment in a goal-less yet coherent way improves learning. In the same context, we analyse the self-organisation of central pattern generators (CPG) by reward maximisation. Our results show that CPGs can learn favourable reward behaviour on high-dimensional robots using the self-organised interaction between degrees of freedom. Finally, we examine an on-line dual control architecture where we combine an Actor-Critic RL and the homeokinetic controller. With this configuration, the probing signal is generated by the exertion of the embodied robot experience with the environment. This set-up solves the problem of designing task-dependant probing signals by the emergence of intrinsically motivated comprehensible behaviour. Faster improvement of the reward signal compared to classic RL is achievable with this configuration

Entropy-based parametric estimation of spike train statistics

We consider the evolution of a network of neurons, focusing on the asymptotic behavior of spikes dynamics instead of membrane potential dynamics. The spike response is not sought as a deterministic response in this context, but as a conditional probability : "Reading out the code" consists of inferring such a probability. This probability is computed from empirical raster plots, by using the framework of thermodynamic formalism in ergodic theory. This gives us a parametric statistical model where the probability has the form of a Gibbs distribution. In this respect, this approach generalizes the seminal and profound work of Schneidman and collaborators. A minimal presentation of the formalism is reviewed here, while a general algorithmic estimation method is proposed yielding fast convergent implementations. It is also made explicit how several spike observables (entropy, rate, synchronizations, correlations) are given in closed-form from the parametric estimation. This paradigm does not only allow us to estimate the spike statistics, given a design choice, but also to compare different models, thus answering comparative questions about the neural code such as : "are correlations (or time synchrony or a given set of spike patterns, ..) significant with respect to rate coding only ?" A numerical validation of the method is proposed and the perspectives regarding spike-train code analysis are also discussed.Comment: 37 pages, 8 figures, submitte

The Internet of Humans: Optimal Resource Allocation and Wireless Channel Prediction

Recent advances in information and communications technologies (ICT) have accelerated the realization of the Internet of Humans (IoH). Among the many IoH applications, Wireless Body Area Networks (BANs) are a remarkable solution that are revolutionising the health care industry. However, many challenges must be addressed, including: a) unavoidable inter-BAN interference severely degrading system performance. b) The non-stationarity and atypical dynamics of BAN channels make it extremely challenging to apply predictive transmit power control that improves the energy efficiency of the network. In this context, this thesis investigates the use of intelligent and adaptive resource allocation algorithms and effective channel prediction to achieve reliable, energy-efficient communications in BAN-enabled IoH. Firstly, we investigate the problem of co-channel interference amongst coexisting BANs by proposing a socially optimal finite repeated non-cooperative transmit power control game. The proposed method improves throughput, reduces overall power consumption and suppress interference. The game is shown to have a unique Nash equilibrium. We also prove that the aggregate outcome of the game is socially efficient across all players at the unique Nash equilibrium, given reasonable constraints for both static and slowly time-varying channels. Secondly, we address the problem of overlapping transmissions among non-coordinated BANs with multiple access schemes through intelligent link resource allocation methods. We present two non-cooperative games, employed with a time-division multiple access (TDMA) based MAC layer scheme that has a novel back-off mechanism. The Link Adaptation game jointly adjusts the sensor node's transmit power and data rate, which provides robust transmission under strong inter-BAN interference. Moreover, by adaptively tuning contention windows size an alternative game, namely a Contention Window game is developed, which significantly reduces latency. The uniqueness and existence of the games' Nash Equilibrium (NE) over the action space are proved using discrete concavity. The NE solution is further analysed and shown to be socially efficient. Motivated by the emergence of deep learning technology, we address the challenge of long-term channel predictions in BANs by using neural networks. Specifically, we propose Long Short-term Memory (LSTM)-based neural network (NN) prediction methods that provide long-term accurate channel gain prediction of up to 2s over non-stationary BAN on-body channels. An incremental learning scheme, which provides continuous and robust predictions, is also developed. We also propose a lightweight NN predictor, namely 'LiteLSTM', that has a compact structure and higher computational efficiency. When implemented on hand-held devices, 'LiteLSTM' remains functional with comparable performance. Finally, we explore the theoretical connections between BAN on-body channels' characteristics and the performance of NN-based power control. To analyse wide-sense stationarity (WSS) characteristics, different stationarity tests are performed for a range of window lengths for on-body channels. Following from this, we develop test benches for NN-based methods at corresponding window lengths using empirical channel measurements. It is observed that WSS characteristics of the BAN on-body channels have a significant impact on the performance of NN-based methods

Time- and value-continuous explainable affect estimation in-the-wild

Today, the relevance of Affective Computing, i.e., of making computers recognise and simulate human emotions, cannot be overstated. All technology giants (from manufacturers of laptops to mobile phones to smart speakers) are in a fierce competition to make their devices understand not only what is being said, but also how it is being said to recognise user’s emotions. The goals have evolved from predicting the basic emotions (e.g., happy, sad) to now the more nuanced affective states (e.g., relaxed, bored) real-time. The databases used in such research too have evolved, from earlier featuring the acted behaviours to now spontaneous behaviours. There is a more powerful shift lately, called in-the-wild affect recognition, i.e., taking the research out of the laboratory, into the uncontrolled real-world. This thesis discusses, for the very first time, affect recognition for two unique in-the-wild audiovisual databases, GRAS2 and SEWA. The GRAS2 is the only database till date with time- and value-continuous affect annotations for Labov effect-free affective behaviours, i.e., without the participant’s awareness of being recorded (which otherwise is known to affect the naturalness of one’s affective behaviour). The SEWA features participants from six different cultural backgrounds, conversing using a video-calling platform. Thus, SEWA features in-the-wild recordings further corrupted by unpredictable artifacts, such as the network-induced delays, frame-freezing and echoes. The two databases present a unique opportunity to study time- and value-continuous affect estimation that is truly in-the-wild. A novel ‘Evaluator Weighted Estimation’ formulation is proposed to generate a gold standard sequence from several annotations. An illustration is presented demonstrating that the moving bag-of-words (BoW) representation better preserves the temporal context of the features, yet remaining more robust against the outliers compared to other statistical summaries, e.g., moving average. A novel, data-independent randomised codebook is proposed for the BoW representation; especially useful for cross-corpus model generalisation testing when the feature-spaces of the databases differ drastically. Various deep learning models and support vector regressors are used to predict affect dimensions time- and value-continuously. Better generalisability of the models trained on GRAS2 , despite the smaller training size, makes a strong case for the collection and use of Labov effect-free data. A further foundational contribution is the discovery of the missing many-to-many mapping between the mean square error (MSE) and the concordance correlation coefficient (CCC), i.e., between two of the most popular utility functions till date. The newly invented cost function |MSE_{XY}/σ_{XY}| has been evaluated in the experiments aimed at demystifying the inner workings of a well-performing, simple, low-cost neural network effectively utilising the BoW text features. Also proposed herein is the shallowest-possible convolutional neural network (CNN) that uses the facial action unit (FAU) features. The CNN exploits sequential context, but unlike RNNs, also inherently allows data- and process-parallelism. Interestingly, for the most part, these white-box AI models have shown to utilise the provided features consistent with the human perception of emotion expression