Bio-inspired Neuromorphic Computing Using Memristor Crossbar Networks

Bio-inspired neuromorphic computing systems built with emerging devices such as memristors have become an active research field. Experimental demonstrations at the network-level have suggested memristor-based neuromorphic systems as a promising candidate to overcome the von-Neumann bottleneck in future computing applications. As a hardware system that offers co-location of memory and data processing, memristor-based networks represent an efficient computing platform with minimal data transfer and high parallelism. Furthermore, active utilization of the dynamic processes during resistive switching in memristors can help realize more faithful emulation of biological device and network behaviors, with the potential to process dynamic temporal inputs efficiently. In this thesis, I present experimental demonstrations of neuromorphic systems using fabricated memristor arrays as well as network-level simulation results. Models of resistive switching behavior in two types of memristor devices, conventional first-order and recently proposed second-order memristor devices, will be first introduced. Secondly, experimental demonstration of K-means clustering through unsupervised learning in a memristor network will be presented. The memristor based hardware systems achieved high classification accuracy (93.3%) on the standard IRIS data set, suggesting practical networks can be built with optimized memristor devices. Thirdly, implementation of a partial differential equation (PDE) solver in memristor arrays will be discussed. This work expands the capability of memristor-based computing hardware from ‘soft’ to ‘hard’ computing tasks, which require very high precision and accurate solutions. In general first-order memristors are suitable to perform tasks that are based on vector-matrix multiplications, ranging from K-means clustering to PDE solvers. On the other hand, utilizing internal device dynamics in second-order memristors can allow natural emulation of biological behaviors and enable network functions such as temporal data processing. An effort to explore second-order memristor devices and their network behaviors will be discussed. Finally, we propose ideas to build large-size passive memristor crossbar arrays, including fabrication approaches, guidelines of device structure, and analysis of the parasitic effects in larger arrays.PHDElectrical EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/147610/1/yjjeong_1.pd

Executive Attention, Action Selection and Attention-Based Learning in Neurally Controlled Autonomous Agents

I describe the design and implementation of an integrated neural architecture, modelled on human executive attention, which is used to control both automatic (reactive) and willed action selection in a simulated robot. The model, based upon Norman and Shallice's supervisory attention system, incorporates important features of human attentional control: selection of an intended task over a more salient automatic task; priming of future tasks that are anticipated; and appropriate levels of persistence of focus of attention. Recognising that attention-based learning, mediated by the limbic system, and the hippocampus in particular, plays an important role in adaptive learning, I extend the Norman and Shallice model, introducing an intrinsic, attention-based learning mechanism that enhances the automaticity of willed actions and reduces future need for attentional effort required for dealing with distractions. These enhanced features support a new level of attentional autonomy in the operation of the simulated robot. Some properties of the model are explored using lesion studies, leading to the identification of a correspondence between the behavioural pathologies of the simulated robot and those seen in human patients suffering dysfunction of executive attention

Functions and mechanisms of intrinsic motivations: the knowledge versus competence distinction

Mammals, and humans in particular, are endowed with an exceptional capacity for cumulative learning. This capacity crucially de- pends on the presence of intrinsic motivations, i.e. motivations that are not directly related to an organism\u27s survival and reproduction but rather to its ability to learn. Recently, there have been a number of attempts to model and reproduce intrinsic motivations in artificial systems. Different kinds of intrinsic motivations have been proposed both in psychology and in machine learning and robotics: some are based on the knowl- edge of the learning system, while others are based on its competence. In this contribution we discuss the distinction between knowledge-based and competence-based intrinsic motivations with respect to both the functional roles that motivations play in learning and the mechanisms by which those functions are implemented. In particular, after arguing that the principal function of intrinsic motivations consists in allowing the development of a repertoire of skills (rather than of knowledge), we suggest that at least two different sub-functions can be identified: (a) discovering which skills might be acquired and (b) deciding which skill to train when. We propose that in biological organisms knowledge-based intrinsic motivation mechanisms might implement the former function, whereas competence-based mechanisms might underly the latter one

Layered control architectures in robots and vertebrates

We revieiv recent research in robotics, neuroscience, evolutionary neurobiology, and ethology with the aim of highlighting some points of agreement and convergence. Specifically, we com pare Brooks' (1986) subsumption architecture for robot control with research in neuroscience demonstrating layered control systems in vertebrate brains, and with research in ethology that emphasizes the decomposition of control into multiple, intertwined behavior systems. From this perspective we then describe interesting parallels between the subsumption architecture and the natural layered behavior system that determines defense reactions in the rat. We then consider the action selection problem for robots and vertebrates and argue that, in addition to subsumption- like conflict resolution mechanisms, the vertebrate nervous system employs specialized selection mechanisms located in a group of central brain structures termed the basal ganglia. We suggest that similar specialized switching mechanisms might be employed in layered robot control archi tectures to provide effective and flexible action selection

Implementation of neural plasticity mechanisms on reconfigurable hardware for robot learning

It is often assumed that insects are “primitive” animals, without the ability to exhibit complex learning behaviour. Fortunately, their tiny brains quite often surprise us with their performance. This thesis investigates the plasticity mechanisms of the insect brain through the research method of neurorobotics, i.e., the development of a physical agent, equipped with a silicon brain. In order to implement such a brain, we have chosen to model it directly onto hardware. Not only does this allow us to take advantage of the inherent hardware parallelism, but the robot can also behave in a completely autonomous mode, without having to communicate with the software simulator of a remote machine. FPGAs offer both the option for such a lowlevel design approach and the flexibility required in computational studies of biological neural networks. With the use of VHDL (a hardware description language), we develop a simulator for neural networks, designed as a series of computational modules, running in parallel and solving the differential equations which describe neural processes. It has the ability to simulate networks with spiking neurons that follow a phenomenological model, proposed by Izhikevich, which requires only 13 operations per 1 ms of simulation. The synaptic plasticity mechanism can be either that of spike timing-dependent plasticity (STDP) or a modified version of STDP which is also affected by neuromodulators. There are no constraints, as far as the connectivity pattern is concerned. The hardware simulator is then added as a peripheral to an embedded system so that it can be more easily controlled through software and connected to a robot. We show that this hardware system is able to model networks with hundreds of neurons and with a speed performance that is better than real-time. With some slight modifications, it could also scale up to thousands of neurons, starting to approach the size of the insect brain. Subsequently, we use the simulator in order to model a neural network with an architecture inspired by the insect brain, representing the connectivity of the antennal lobe, the mushroom body and the lateral horn, structures which are part of the insect’s olfactory pathway. Our silicon brain is then attached to a robot and its limits and capabilities are tested in a series of experiments. The experiments involve tasks of associative learning inside an arena which is based on a T-maze set-up usually employed in behavioural experiments with flies. The robot is trained to associate different stimuli (or combinations of stimuli) with a punishment, as indicated by the presence of a light source. We observe that the robot can solve most of the tasks, including elemental learning, discrimination learning, biconditional discrimination and negative patterning but fails to solve the problem of positive patterning. It is concluded that the architecture of the insect’s olfactory pathway has the computational efficiency to solve even non-elemental learning tasks. However, this pattern of results does not precisely match the fly, suggesting we have not fully understood the learning mechanisms involved. Moreover, embedding the learning circuit in robot behaviour reveals that the simple version of STDP is not the appropriate mechanism which can link neural plasticity to learning behaviour. Although the modified version of STDP is more suitable, it remains problematic as well as sensitive to timing issues. Therefore, we propose that STDP might function more as a “priming” process rather than as the basic learning mechanism

Neurobehavioural and molecular mechanisms of social learning in zebrafish

Dissertation presented to obtain the Ph.D. degree in Behavioural Biology, presented at ISPA – Instituto Universitário in the year of 2019Os animais utilizam informação social e não social para tomarem decisões adaptativas que tem impacto no seu fitness. O uso de informação social traz vantagens como escapar a um predador, encontrar fontes de comida ou evitar lutas com indivíduos mais fortes, apenas por observação dos seus conspecíficos ou produtos relacionados com eles. A aprendizagem social ocorre quando os indivíduos observam o comportamento de outros ou as suas consequências para modificar o seu próprio comportamento. Esta estratégia comportamental é conservada entre espécies: os grilos, Nemobius sylvestris, adaptam o seu comportamento para evitar um predador depois de observar o comportamento de outros e mantem essas mudanças comportamentais, duradouramente, mesmo apos os demonstradores não estarem presentes; as abelhas operárias, Apis Mellifera, apresentam uma série de comportamentos motores estereotipados que informam outras operárias da localização precisa de uma fonte de comida. Os mecanismos neuronais da aprendizagem social não estão claramente compreendidos, e são o centro de debate nesta área de investigação. Alguns autores hipotetizam que os mecanismos neurais da aprendizagem social são partilhados, e outros autores defendem que a aprendizagem social é um domínio geral presente até em espécies solitárias. O principal objetivo deste trabalho é clarificar os mecanismos subjacentes a aprendizagem social e não social. Este trabalho subdivide-se em dois capítulos experimentais: o capítulo II, onde procuramos os circuitos neurais do condicionamento observado com um estímulo social ou não social; e capitulo III, no qual a eficácia de estímulos sociais químicos e visuais é testada num paradigma de condicionamento aversivo. Em ambos os capítulos, um gene de ativação imediata são usados como marcadores de atividade neuronal: no capítulo II utilizando a expressão de c-fos, por hibridação insitu, para mapear as regiões do cérebro recrutadas em aprendizagem social e não social; e no capítulo III, a reação quantitativa em cadeia da polimerase foi utilizada numa abordagem com genes e regiões do cérebro candidatas para perceber o envolvimento do sistema olfativo em aprendizagem social olfativa. No capítulo II, nós demonstramos que a aprendizagem social (SL) recruta diferentes regiões do cérebro quando comparada com a aprendizagem não social (AL): SL aumenta a expressão de c-fos nos bulbos olfativos, na zona ventral da área telencefálica ventral, na habénula ventral, no tálamo ventromedial e a AL diminui a expressão de c-fos na habénula dorsal e no núcleo tubercular anterior. Alem disso, conjuntos diferenciais de regiões cerebrais aparecem associados a aprendizagem social e não social depois de uma análise funcional da conectividade entre as regiões do cérebro. No capítulo III, nós mostramos que pistas sociais visuais, como a observação de um conspecífico a exibir uma resposta de alarme, não é eficaz como um estimulo não condicionado (US), mas pistas sociais olfativas, como substância de alarme, foi altamente eficiente como US em aprendizagem aversiva. Além disso, identificamos os bulbos olfativos como uma área do cérebro essencial para condicionamento observado olfativo. Uma análise funcional da coesão e conectividade dos núcleos do cérebro envolvidos em processamento olfativo mostraram uma rede apurada para condicionamento observado olfativo. Em resumo, a presente tese elucida o debate nesta área de investigação sobre os mecanismos da aprendizagem social. Este trabalho clarifica que ao nível comportamental a aprendizagem social requer um domínio geral e ao nível neuronal é necessária uma rede modular que permite a computação em simultâneo de várias informações com diferentes níveis de complexidade.Animals use social and asocial information to take adaptive decisions that impact their fitness. The use of social information brings advantages as to escape a predator, to find a food source or to avoid fights with strongest individuals, only by the observation of conspecifics or their related products. Social learning occurs when individuals observe the behaviour of others, or its consequences, to modify their own behaviour. This behavioural strategy is highly conserved across taxa: the crickets, Nemobius sylvestris, adapt their predator-avoidance behaviour after having observed the behaviour of knowledgeable others, and they maintain these behavioural changes lastingly after demonstrators are gone; the foragers of honeybees, Apis mellifera, display a series of stereotypical motor behaviours which inform other foragers of the precise location of floral food. The neuronal mechanisms of social learning are not clearly understood, and they are in centre of debate in the field. Some authors hypothesized that the neural mechanisms of social learning are shared and others that social learning is a general domain present even in solitary species. The main goal of the present work is to clarify the mechanisms underlying social and asocial learning. This work subdivide in two experimental chapters: the chapter II, where we search for the neuronal circuits of reward observational conditioning with social or asocial stimuli; and the chapter III, in which the effectiveness of a chemical and a visual social stimulus are tested as unconditioned stimulus (US) in an aversive learning paradigm. In both chapters, an immediate early gene is used as a marker of neuronal activity: in chapter II using the expression of c-fos, by in-situ hybridization, to map the brain regions recruited in social and asocial learning; and in chapter III, the quantitative polymerase chain reaction (pPCR) was used in a candidate genes and brain regions approach. In chapter II, we demonstrated that social learning (SL) recruit different brain regions than asocial learning (AL): SL increased the expression of c-fos in olfactory bulbs, in ventral zone of ventral telencephalic area, in ventral habenula, in ventromedial thalamus and AL decreased the expression of c-fos in dorsal habenula and in anterior tubercular nucleus. Moreover, differential sets of brain regions appear associated to social and asocial learning after a functional connectivity analysis. In chapter III, we showed that the social visual cue, the sight of alarmed conspecifics, was not effective as an US; but social olfactory cue, the alarm substance, was highly efficient in aversive learning paradigm. Also, we identified the olfactory bulbs as an essential brain region to olfactory observational conditioning. A functional analysis of the cohesion and connectivity of the brain nuclei involved in olfactory processing were tuned to chemical observational conditioning. In sum, the present thesis elucidated the debate in the field on the mechanisms of social learning. This work clarified that at the behavioural level social learning proved to be a general domain, and at the neuronal level a modular network is needed to allow the computation, at the same time, of high amount information with different levels of complexity

World model learning and inference

Understanding information processing in the brain-and creating general-purpose artificial intelligence-are long-standing aspirations of scientists and engineers worldwide. The distinctive features of human intelligence are high-level cognition and control in various interactions with the world including the self, which are not defined in advance and are vary over time. The challenge of building human-like intelligent machines, as well as progress in brain science and behavioural analyses, robotics, and their associated theoretical formalisations, speaks to the importance of the world-model learning and inference. In this article, after briefly surveying the history and challenges of internal model learning and probabilistic learning, we introduce the free energy principle, which provides a useful framework within which to consider neuronal computation and probabilistic world models. Next, we showcase examples of human behaviour and cognition explained under that principle. We then describe symbol emergence in the context of probabilistic modelling, as a topic at the frontiers of cognitive robotics. Lastly, we review recent progress in creating human-like intelligence by using novel probabilistic programming languages. The striking consensus that emerges from these studies is that probabilistic descriptions of learning and inference are powerful and effective ways to create human-like artificial intelligent machines and to understand intelligence in the context of how humans interact with their world