The Role of Synaptic Tagging and Capture for Memory Dynamics in Spiking Neural Networks

Memory serves to process and store information about experiences such that this information can be used in future situations. The transfer from transient storage into long-term memory, which retains information for hours, days, and even years, is called consolidation. In brains, information is primarily stored via alteration of synapses, so-called synaptic plasticity. While these changes are at first in a transient early phase, they can be transferred to a late phase, meaning that they become stabilized over the course of several hours. This stabilization has been explained by so-called synaptic tagging and capture (STC) mechanisms. To store and recall memory representations, emergent dynamics arise from the synaptic structure of recurrent networks of neurons. This happens through so-called cell assemblies, which feature particularly strong synapses. It has been proposed that the stabilization of such cell assemblies by STC corresponds to so-called synaptic consolidation, which is observed in humans and other animals in the first hours after acquiring a new memory. The exact connection between the physiological mechanisms of STC and memory consolidation remains, however, unclear. It is equally unknown which influence STC mechanisms exert on further cognitive functions that guide behavior. On timescales of minutes to hours (that means, the timescales of STC) such functions include memory improvement, modification of memories, interference and enhancement of similar memories, and transient priming of certain memories. Thus, diverse memory dynamics may be linked to STC, which can be investigated by employing theoretical methods based on experimental data from the neuronal and the behavioral level. In this thesis, we present a theoretical model of STC-based memory consolidation in recurrent networks of spiking neurons, which are particularly suited to reproduce biologically realistic dynamics. Furthermore, we combine the STC mechanisms with calcium dynamics, which have been found to guide the major processes of early-phase synaptic plasticity in vivo. In three included research articles as well as additional sections, we develop this model and investigate how it can account for a variety of behavioral effects. We find that the model enables the robust implementation of the cognitive memory functions mentioned above. The main steps to this are: 1. demonstrating the formation, consolidation, and improvement of memories represented by cell assemblies, 2. showing that neuromodulator-dependent STC can retroactively control whether information is stored in a temporal or rate-based neural code, and 3. examining interaction of multiple cell assemblies with transient and attractor dynamics in different organizational paradigms. In summary, we demonstrate several ways by which STC controls the late-phase synaptic structure of cell assemblies. Linking these structures to functional dynamics, we show that our STC-based model implements functionality that can be related to long-term memory. Thereby, we provide a basis for the mechanistic explanation of various neuropsychological effects.2021-09-0

Formation control of mobile robots and unmanned aerial vehicles

In this dissertation, the nonlinear control of nonholonomic mobile robot formations and unmanned aerial vehicle (UAV) formations is undertaken and presented in six papers. In the first paper, an asymptotically stable combined kinematic/torque control law is developed for leader-follower based formation control of mobile robots using backstepping. A neural network (NN) is introduced along with robust integral of the sign of the error (RISE) feedback to approximate the dynamics of the follower as well as its leader using online weight tuning. Subsequently, in the second paper, a novel NN observer is designed to estimate the linear and angular velocities of both the follower and its leader robot and a NN output feedback control law is developed. On the other hand, in the third paper, a NN-based output feedback control law is presented for the control of an underactuated quad rotor UAV, and a NN virtual control input scheme is proposed which allows all six degrees of freedom to be controlled using only four control inputs. The results of this paper are extended to include the control of quadrotor UAV formations, and a novel three-dimensional leader-follower framework is proposed in the fourth paper. Next, in the fifth paper, the discrete-time nonlinear optimal control is undertaken using two online approximators (OLA\u27s) to solve the infinite horizon Hamilton-Jacobi-Bellman (HJB) equation forward-in-time to achieve nearly optimal regulation and tracking control. In contrast, paper six utilizes a single OLA to solve the infinite horizon HJB and Hamilton-Jacobi-Isaacs (HJI) equations forward-intime for the near optimal regulation and tracking control of continuous affine nonlinear systems. The effectiveness of the optimal tracking controllers proposed in the fifth and sixth papers are then demonstrated using nonholonomic mobile robot formation control --Abstract, page iv

Health Management and Adaptive Control of Distributed Spacecraft Systems

As the development of challenging missions like on-orbit construction and collaborative inspection that involve multi-spacecraft systems increases, the requirements needed to improve post-failure safety to maintain the mission performance also increases, especially when operating under uncertain conditions. In particular, space missions that involve Distributed Spacecraft Systems (e.g, inspection, repairing, assembling, or deployment of space assets) are susceptible to failures and threats that are detrimental to the overall mission performance. This research applies a distributed Health Management System that uses a bio-inspired mechanism based on the Artificial Immune System coupled with a Support Vector Machine to obtain an optimized health monitoring system capable of detecting nominal and off-nominal system conditions. A simulation environment is developed for a fleet of spacecraft performing a low-Earth orbit inspection within close proximity of a target space asset, where the spacecraft observers follow stable relative orbits with respect to the target asset, allowing dynamics to be expressed using the Clohessy-Wiltshire-Hill equations. Additionally, based on desired points of inspection, the observers have specific attitude requirements that are achieved using Reaction Wheels as the control moment device. An adaptive control based on Deep Reinforcement Learning using an Actor-Critic-Adverse architecture is implemented to achieve high levels of mission protection, especially under disturbances that might lead to performance degradation. Numerical simulations to evaluate the capabilities of the health management architecture when the spacecraft network is subjected to failures are performed. A comparison of different attitude controllers such as Nonlinear Dynamic Inversion and Pole Placement against Deep Reinforcement Learning based controller is presented. The Dynamic Inversion controller showed better tracking performance but large control effort, while the Deep Reinforcement controller showed satisfactory tracking performance with minimal control effort. Numerical simulations successfully demonstrated the potential of both the bioinspired Health Monitoring System architecture and the controller, to detect and identify failures and overcome bounded disturbances, respectively

Advances in Reinforcement Learning

Reinforcement Learning (RL) is a very dynamic area in terms of theory and application. This book brings together many different aspects of the current research on several fields associated to RL which has been growing rapidly, producing a wide variety of learning algorithms for different applications. Based on 24 Chapters, it covers a very broad variety of topics in RL and their application in autonomous systems. A set of chapters in this book provide a general overview of RL while other chapters focus mostly on the applications of RL paradigms: Game Theory, Multi-Agent Theory, Robotic, Networking Technologies, Vehicular Navigation, Medicine and Industrial Logistic

Biometrics

Biometrics-Unique and Diverse Applications in Nature, Science, and Technology provides a unique sampling of the diverse ways in which biometrics is integrated into our lives and our technology. From time immemorial, we as humans have been intrigued by, perplexed by, and entertained by observing and analyzing ourselves and the natural world around us. Science and technology have evolved to a point where we can empirically record a measure of a biological or behavioral feature and use it for recognizing patterns, trends, and or discrete phenomena, such as individuals' and this is what biometrics is all about. Understanding some of the ways in which we use biometrics and for what specific purposes is what this book is all about

Goal-directed cross-system interactions in brain and deep learning networks

Deep neural networks (DNN) have recently emerged as promising models for the mammalian ventral visual stream. However, how ventral stream adapts to various goal-directed influences and coordinates with higher-level brain regions during learning remain poorly understood. By incorporating top-down influences involving attentional cues, linguistic labels and novel category learning into DNN models, the thesis offers an explanation for how the tasks we do shape representations across levels in models and related brain regions including ventral visual stream, HPC and ventromedial prefrontal cortex (vmPFC) via a theoretical modelling approach. The thesis include three main contributions. In the first contribution, I developed a goal-directed attention mechanism which extends general-purpose DNN with the ability to reconfigure itself to better suit the current task goal, much like PFC modulates activity along the ventral stream. In the second contribution, I uncovered how linguistic labelling shapes semantic representation by amending existing DNN to both predict the meaning and the categorical label of an object. Supported by simulation results involving fine-grained and coarse-grained labels, I concluded that differences in label use, whether across languages or levels of expertise, manifest in differences in the semantic representations that support label discrimination. In the third contribution, I aimed to better understand cross-brain mechanisms in a novel learning task by combining insights on labelling and attention obtained from preceding efforts. Integrating DNN with a novel clustering model built off from SUSTAIN, the proposed account captures human category learning behaviour and the underlying neural mechanisms across multiple interacting brain areas involving HPC, vmPFC and the ventral visual stream. By extending models of the ventral stream to incorporate goal-directed cross-system coordination, I hope the thesis can inform understanding of the neurobiology supporting object recognition and category learning which in turn help us advance designs of deep learning models

Distributed Control of Networked Nonlinear Euler-Lagrange Systems

Motivated by recent developments in formation and cooperative control of networked multi-agent systems, the main goal of this thesis is development of efficient synchronization and formation control algorithms for distributed control of networked nonlinear systems whose dynamics can be described by Euler-Lagrange (EL) equations. One of the main challenges in the design of the formation control algorithm is its optimality and robustness to parametric uncertainties, external disturbances and ability to reconfigure in presence of component, actuator, or sensor faults. Furthermore, the controller should be capable of handling switchings in the communication network topology. In this work, nonlinear optimal control techniques are studied for developing distributed controllers for networked EL systems. An individual cost function is introduced to design a controller that relies on only local information exchanges among the agents. In the development of the controller, it is assumed that the communication graph is not fixed (in other words the topology is switching). Additionally, parametric uncertainties and faults in the EL systems are considered and two approaches, namely adaptive and robust techniques are introduced to compensate for the effects of uncertainties and actuator faults. Next, a distributed H_infinity performance measure is considered to develop distributed robust controllers for uncertain networked EL systems. The developed distributed controller is obtained through rigorous analysis and by considering an individual cost function to enhance the robustness of the controllers in presence of parametric uncertainties and external bounded disturbances. Moreover, a rigorous analysis is conducted on the performance of the developed controllers in presence of actuator faults as well as fault diagnostic and identification (FDI) imperfections. Next, synchronization and set-point tracking control of networked EL systems are investigated in presence of three constraints, namely, (i) input saturation constraints, (ii) unavailability of velocity feedback, and (iii) lack of knowledge on the system parameters. It is shown that the developed distributed controllers can accomplish the desired requirements and specification under the above constraints. Finally, a quaternion-based approach is considered for the attitude synchronization and set-point tracking control problem of formation flying spacecraft. Employing the quaternion in the control law design enables handling large rotations in the spacecraft attitude and, therefore, any singularities in the control laws are avoided. Furthermore, using the quaternion also enables one to guarantee boundedness of the control signals both with and without velocity feedback