Storing cycles in Hopfield-type networks with pseudoinverse learning rule: admissibility and network topology

Cyclic patterns of neuronal activity are ubiquitous in animal nervous systems, and partially responsible for generating and controlling rhythmic movements such as locomotion, respiration, swallowing and so on. Clarifying the role of the network connectivities for generating cyclic patterns is fundamental for understanding the generation of rhythmic movements. In this paper, the storage of binary cycles in neural networks is investigated. We call a cycle $\Sigma$ admissible if a connectivity matrix satisfying the cycle's transition conditions exists, and construct it using the pseudoinverse learning rule. Our main focus is on the structural features of admissible cycles and corresponding network topology. We show that $\Sigma$ is admissible if and only if its discrete Fourier transform contains exactly $r={rank}(\Sigma)$ nonzero columns. Based on the decomposition of the rows of $\Sigma$ into loops, where a loop is the set of all cyclic permutations of a row, cycles are classified as simple cycles, separable or inseparable composite cycles. Simple cycles contain rows from one loop only, and the network topology is a feedforward chain with feedback to one neuron if the loop-vectors in $\Sigma$ are cyclic permutations of each other. Composite cycles contain rows from at least two disjoint loops, and the neurons corresponding to the rows in $\Sigma$ from the same loop are identified with a cluster. Networks constructed from separable composite cycles decompose into completely isolated clusters. For inseparable composite cycles at least two clusters are connected, and the cluster-connectivity is related to the intersections of the spaces spanned by the loop-vectors of the clusters. Simulations showing successfully retrieved cycles in continuous-time Hopfield-type networks and in networks of spiking neurons are presented.Comment: 48 pages, 3 figure

Neural node network and model, and method of teaching same

The present invention is a fully connected feed forward network that includes at least one hidden layer 16. The hidden layer 16 includes nodes 20 in which the output of the node is fed back to that node as an input with a unit delay produced by a delay device 24 occurring in the feedback path 22 (local feedback). Each node within each layer also receives a delayed output (crosstalk) produced by a delay unit 36 from all the other nodes within the same layer 16. The node performs a transfer function operation based on the inputs from the previous layer and the delayed outputs. The network can be implemented as analog or digital or within a general purpose processor. Two teaching methods can be used: (1) back propagation of weight calculation that includes the local feedback and the crosstalk or (2) more preferably a feed forward gradient decent which immediately follows the output computations and which also includes the local feedback and the crosstalk. Subsequent to the gradient propagation, the weights can be normalized, thereby preventing convergence to a local optimum. Education of the network can be incremental both on and off-line. An educated network is suitable for modeling and controlling dynamic nonlinear systems and time series systems and predicting the outputs as well as hidden states and parameters. The educated network can also be further educated during on-line processing

Mechanism, dynamics, and biological existence of multistability in a large class of bursting neurons

Multistability, the coexistence of multiple attractors in a dynamical system, is explored in bursting nerve cells. A modeling study is performed to show that a large class of bursting systems, as defined by a shared topology when represented as dynamical systems, is inherently suited to support multistability. We derive the bifurcation structure and parametric trends leading to multistability in these systems. Evidence for the existence of multirhythmic behavior in neurons of the aquatic mollusc Aplysia californica that is consistent with our proposed mechanism is presented. Although these experimental results are preliminary, they indicate that single neurons may be capable of dynamically storing information for longer time scales than typically attributed to nonsynaptic mechanisms.Comment: 24 pages, 8 figure

Investigation on Delayed Feedback Neural Networks and Two-layer Feedback Neural Networks

反馈神经网络是神经网络中最重要的类型之一，这种网络的突出特点就是它具有联想记忆的功能。反馈神经网络最重要的范例就是Hopfield网络，它已经被人们广泛地研究过，然而Hopfield网络和它的许多变种都存在着一些严重的缺陷，比如伪吸引子问题和低存储率问题。这些缺陷严重地限制了它们的实际应用能力。另外，Hopfield网络是典型的非线性动力系统，在近十几年来非线性动力学的发展中，人们认识到到了非线性动力系统不仅可能存在不动点解，而且也可能存在大量的极限环或周期运动解。目前，绝大部分反馈神经网络都是利用系统的不动点作为信息存储的载体，但这种方式没有利用非线性动力系统大量存在的周期解，从某种意义上说...Feedback neural network is one of the most important neural network, the most remarkable feature of this kind of network is the associative memory function. The most important paradigm of feedback neural network is the Hopfield network which has been studied widely, but the Hopfield network and its modifications have some serious limitations, such as the spurious attractor problem and low storage ...学位：理学博士院系专业：物理与机电工程学院物理学系_凝聚态物理学号：B20042400

Dynamical principles in neuroscience

Dynamical modeling of neural systems and brain functions has a history of success over the last half century. This includes, for example, the explanation and prediction of some features of neural rhythmic behaviors. Many interesting dynamical models of learning and memory based on physiological experiments have been suggested over the last two decades. Dynamical models even of consciousness now exist. Usually these models and results are based on traditional approaches and paradigms of nonlinear dynamics including dynamical chaos. Neural systems are, however, an unusual subject for nonlinear dynamics for several reasons: (i) Even the simplest neural network, with only a few neurons and synaptic connections, has an enormous number of variables and control parameters. These make neural systems adaptive and flexible, and are critical to their biological function. (ii) In contrast to traditional physical systems described by well-known basic principles, first principles governing the dynamics of neural systems are unknown. (iii) Many different neural systems exhibit similar dynamics despite having different architectures and different levels of complexity. (iv) The network architecture and connection strengths are usually not known in detail and therefore the dynamical analysis must, in some sense, be probabilistic. (v) Since nervous systems are able to organize behavior based on sensory inputs, the dynamical modeling of these systems has to explain the transformation of temporal information into combinatorial or combinatorial-temporal codes, and vice versa, for memory and recognition. In this review these problems are discussed in the context of addressing the stimulating questions: What can neuroscience learn from nonlinear dynamics, and what can nonlinear dynamics learn from neuroscience?This work was supported by NSF Grant No. NSF/EIA-0130708, and Grant No. PHY 0414174; NIH Grant No. 1 R01 NS50945 and Grant No. NS40110; MEC BFI2003-07276, and Fundación BBVA

Optimal modularity and memory capacity of neural reservoirs

The neural network is a powerful computing framework that has been exploited by biological evolution and by humans for solving diverse problems. Although the computational capabilities of neural networks are determined by their structure, the current understanding of the relationships between a neural network's architecture and function is still primitive. Here we reveal that neural network's modular architecture plays a vital role in determining the neural dynamics and memory performance of the network of threshold neurons. In particular, we demonstrate that there exists an optimal modularity for memory performance, where a balance between local cohesion and global connectivity is established, allowing optimally modular networks to remember longer. Our results suggest that insights from dynamical analysis of neural networks and information spreading processes can be leveraged to better design neural networks and may shed light on the brain's modular organization

Design and test of a neural microprocessor

En aquest projecte, es dissenya un microprocessador neuronal per ser implementat en FPGAs. Aquesta tecnologia consisteix en un processador softcore basat en RISC-V descrit amb SystemVerilog que s'utilitza per controlar un coprocessador encarregat d'executar una xarxa neuronal spiking amb propagació directa descrita amb VHDL. El control es fa amb senyals que es generen a partir d'instruccions SIMD personalitzades definides en una extensió del conjunt d’instruccions RSIC-V. Per fer-ho, es modifica el processador de manera que pugui detectar i descodificar les noves instruccions emmagatzemades a la seva memòria de programa. Per facilitar la tasca de definir el contingut de la memòria del programa, s'utilitza un codi escrit en C i es desenvolupa un conjunt d'instruccions C personalitzades. Aquestes instruccions es basen en l'ús de macros i inline assembly, i la seva finalitat és facilitar i permetre l'ús de les instruccions personalitzades RISC-V en el codi d'alt nivell. Per demostrar el correcte funcionament del projecte, se simula el microprocessador neuronal i després es prova a l'FPGA d'una placa de desenvolupament Nexys 4, amb el coprocessador implementat per resoldre el problema XOR. La implementació del coprocessador es replica amb C i s'executa a l'FPGA utilitzant només el processador predeterminat sense modificar. Finalment, els resultats s'analitzen i es comparen per determinar les compensacions entre els dos enfocaments en termes de temps d'execució, consum d'energia i espai utilitzat.En este proyecto, se diseña un microprocesador neuronal para su implementación en FPGAs. Esta tecnología consiste en un procesador softcore basado en RISC-V descrito con SystemVerilog que se utiliza para controlar a un coprocesador encargado de ejecutar una red neuronal spiking con propagación directa descrita con VHDL. El control se realiza con señales que se generan a partir de instrucciones SIMD personalizadas definidas en una extensión del conjunto de instrucciones RSIC-V. Para ello, se modifica el procesador de forma que pueda detectar y descodificar las nuevas instrucciones almacenadas en su memoria de programa. Para facilitar la tarea de definir el contenido de la memoria del programa, se utiliza un código escrito en C y se desarrolla un conjunto de instrucciones C personalizadas. Estas instrucciones se basan en el uso de macros e inline assembly, y su finalidad es facilitar y permitir el uso de las instrucciones personalizadas RISC-V en el código de alto nivel. Para demostrar el correcto funcionamiento del proyecto, se simula el microprocesador neuronal y después se prueba en la FPGA de una placa de desarrollo Nexys 4, con el coprocesador implementado para resolver el problema XOR. La implementación del coprocesador se replica con C y se ejecuta en la FPGA utilizando sólo el procesador predeterminado sin modifcar. Por último, los resultados se analizan y se comparan para determinar las compensaciones entre ambos enfoques en términos de tiempo de ejecución, consumo de energía y espacio utilizado.In this project, a neural microprocessor is designed to be implemented in FPGAs. This technology consists of a RISC-V-based soft processor described in SystemVerilog that is used to control a coprocessor in charge of executing a feedforward spiking neural network described in VHDL. The control is done with signals that are generated from custom-designed SIMD instructions defined in a RISC-V ISA extension. To do it, the processor is modified such that it can detect and decode the new instructions stored in its program memory. To facilitate the task of defining the program memory contents, a code written in C is used and a set of custom C instructions is developed. These instructions are based on the use of macros and inline assembly, and their purpose is to facilitate and allow the use of the RISC-V custom instructions in the high-level code. To demonstrate the correct operation of the project, the neural microprocessor is simulated and then tested on the FPGA of a Nexys 4 development board, with the coprocessor implemented for solving the XOR problem. The coprocessor implementation is replicated with C and executed in the FPGA using only the default processor without being modified. Finally, the results are analyzed and compared to determine the trade-offs between the two approaches in terms of execution time, power consumption, and utilized space

FPGA Accelerator Architecture for Q-learning and its Applications in Space Exploration Rovers

abstract: Achieving human level intelligence is a long-term goal for many Artificial Intelligence (AI) researchers. Recent developments in combining deep learning and reinforcement learning helped us to move a step forward in achieving this goal. Reinforcement learning using a delayed reward mechanism is an approach to machine intelligence which studies decision making with control and how a decision making agent can learn to act optimally in an environment-unaware conditions. Q-learning is one of the model-free reinforcement directed learning strategies which uses temporal differences to estimate the performances of state-action pairs called Q values. A simple implementation of Q-learning algorithm can be done using a Q table memory to store and update the Q values. However, with an increase in state space data due to a complex environment, and with an increase in possible number of actions an agent can perform, Q table reaches its space limit and would be difficult to scale well. Q-learning with neural networks eliminates the use of Q table by approximating the Q function using neural networks. Autonomous agents need to develop cognitive properties and become self-adaptive to be deployable in any environment. Reinforcement learning with Q-learning have been very efficient in solving such problems. However, embedded systems like space rovers and autonomous robots rarely implement such techniques due to the constraints faced like processing power, chip area, convergence rate and cost of the chip. These problems present a need for a portable, low power, area efficient hardware accelerator to accelerate the process of such learning. This problem is targeted by implementing a hardware schematic architecture for Q-learning using Artificial Neural networks. This architecture exploits the massive parallelism provided by neural network with a dedicated fine grain parallelism provided by a Field Programmable Gate Array (FPGA) thereby processing the Q values at a high throughput. Mars exploration rovers currently use Xilinx-Space-grade FPGA devices for image processing, pyrotechnic operation control and obstacle avoidance. The hardware resource consumption for the architecture has been synthesized considering Xilinx Virtex7 FPGA as the target device.Dissertation/ThesisMasters Thesis Engineering 201