High-capacity exponential associative memories

A generalized associative memory model with potentially high capacity is presented. A memory of this kind with M stored vectors of length N, can be implemented with M nonlinear neurons, N ordinary thresholding neurons, and 2MN binary synapses. It is shown that special cases of this model include the Hopfield and high-order correlation memories. A special case of the model, based on a neuron which can implement the subthreshold region, is presented. The authors analyze the capacity of this exponentially associative memory and show that it scales exponentially with N. In any practical realization, however, the dynamic range of the exponentiators is constrained. They show that the capacity for networks with fixed dynamic range exponential circuits is proportional to the dynamic range

Pavlov's dog associative learning demonstrated on synaptic-like organic transistors

In this letter, we present an original demonstration of an associative learning neural network inspired by the famous Pavlov's dogs experiment. A single nanoparticle organic memory field effect transistor (NOMFET) is used to implement each synapse. We show how the physical properties of this dynamic memristive device can be used to perform low power write operations for the learning and implement short-term association using temporal coding and spike timing dependent plasticity based learning. An electronic circuit was built to validate the proposed learning scheme with packaged devices, with good reproducibility despite the complex synaptic-like dynamic of the NOMFET in pulse regime

Storage Capacity Diverges with Synaptic Efficiency in an Associative Memory Model with Synaptic Delay and Pruning

It is known that storage capacity per synapse increases by synaptic pruning in the case of a correlation-type associative memory model. However, the storage capacity of the entire network then decreases. To overcome this difficulty, we propose decreasing the connecting rate while keeping the total number of synapses constant by introducing delayed synapses. In this paper, a discrete synchronous-type model with both delayed synapses and their prunings is discussed as a concrete example of the proposal. First, we explain the Yanai-Kim theory by employing the statistical neurodynamics. This theory involves macrodynamical equations for the dynamics of a network with serial delay elements. Next, considering the translational symmetry of the explained equations, we re-derive macroscopic steady state equations of the model by using the discrete Fourier transformation. The storage capacities are analyzed quantitatively. Furthermore, two types of synaptic prunings are treated analytically: random pruning and systematic pruning. As a result, it becomes clear that in both prunings, the storage capacity increases as the length of delay increases and the connecting rate of the synapses decreases when the total number of synapses is constant. Moreover, an interesting fact becomes clear: the storage capacity asymptotically approaches $2/\pi$ due to random pruning. In contrast, the storage capacity diverges in proportion to the logarithm of the length of delay by systematic pruning and the proportion constant is $4/\pi$. These results theoretically support the significance of pruning following an overgrowth of synapses in the brain and strongly suggest that the brain prefers to store dynamic attractors such as sequences and limit cycles rather than equilibrium states.Comment: 27 pages, 14 figure

Real time unsupervised learning of visual stimuli in neuromorphic VLSI systems

Neuromorphic chips embody computational principles operating in the nervous system, into microelectronic devices. In this domain it is important to identify computational primitives that theory and experiments suggest as generic and reusable cognitive elements. One such element is provided by attractor dynamics in recurrent networks. Point attractors are equilibrium states of the dynamics (up to fluctuations), determined by the synaptic structure of the network; a `basin' of attraction comprises all initial states leading to a given attractor upon relaxation, hence making attractor dynamics suitable to implement robust associative memory. The initial network state is dictated by the stimulus, and relaxation to the attractor state implements the retrieval of the corresponding memorized prototypical pattern. In a previous work we demonstrated that a neuromorphic recurrent network of spiking neurons and suitably chosen, fixed synapses supports attractor dynamics. Here we focus on learning: activating on-chip synaptic plasticity and using a theory-driven strategy for choosing network parameters, we show that autonomous learning, following repeated presentation of simple visual stimuli, shapes a synaptic connectivity supporting stimulus-selective attractors. Associative memory develops on chip as the result of the coupled stimulus-driven neural activity and ensuing synaptic dynamics, with no artificial separation between learning and retrieval phases.Comment: submitted to Scientific Repor

The malleable brain: plasticity of neural circuits and behavior: A review from students to students

One of the most intriguing features of the brain is its ability to be malleable, allowing it to adapt continually to changes in the environment. Specific neuronal activity patterns drive long-lasting increases or decreases in the strength of synaptic connections, referred to as long-term potentiation (LTP) and long-term depression (LTD) respectively. Such phenomena have been described in a variety of model organisms, which are used to study molecular, structural, and functional aspects of synaptic plasticity. This review originated from the first International Society for Neurochemistry (ISN) and Journal of Neurochemistry (JNC) Flagship School held in Alpbach, Austria (Sep 2016), and will use its curriculum and discussions as a framework to review some of the current knowledge in the field of synaptic plasticity. First, we describe the role of plasticity during development and the persistent changes of neural circuitry occurring when sensory input is altered during critical developmental stages. We then outline the signaling cascades resulting in the synthesis of new plasticity-related proteins, which ultimately enable sustained changes in synaptic strength. Going beyond the traditional understanding of synaptic plasticity conceptualized by LTP and LTD, we discuss system-wide modifications and recently unveiled homeostatic mechanisms, such as synaptic scaling. Finally, we describe the neural circuits and synaptic plasticity mechanisms driving associative memory and motor learning. Evidence summarized in this review provides a current view of synaptic plasticity in its various forms, offers new insights into the underlying mechanisms and behavioral relevance, and provides directions for future research in the field of synaptic plasticity.Fil: Schaefer, Natascha. University of Wuerzburg; AlemaniaFil: Rotermund, Carola. University of Tuebingen; AlemaniaFil: Blumrich, Eva Maria. Universitat Bremen; AlemaniaFil: Lourenco, Mychael V.. Universidade Federal do Rio de Janeiro; BrasilFil: Joshi, Pooja. Robert Debre Hospital; FranciaFil: Hegemann, Regina U.. University of Otago; Nueva ZelandaFil: Jamwal, Sumit. ISF College of Pharmacy; IndiaFil: Ali, Nilufar. Augusta University; Estados UnidosFil: García Romero, Ezra Michelet. Universidad Veracruzana; MéxicoFil: Sharma, Sorabh. Birla Institute of Technology and Science; IndiaFil: Ghosh, Shampa. Indian Council of Medical Research; IndiaFil: Sinha, Jitendra K.. Indian Council of Medical Research; IndiaFil: Loke, Hannah. Hudson Institute of Medical Research; AustraliaFil: Jain, Vishal. Defence Institute of Physiology and Allied Sciences; IndiaFil: Lepeta, Katarzyna. Polish Academy of Sciences; ArgentinaFil: Salamian, Ahmad. Polish Academy of Sciences; ArgentinaFil: Sharma, Mahima. Polish Academy of Sciences; ArgentinaFil: Golpich, Mojtaba. University Kebangsaan Malaysia Medical Centre; MalasiaFil: Nawrotek, Katarzyna. University Of Lodz; ArgentinaFil: Paid, Ramesh K.. Indian Institute of Chemical Biology; IndiaFil: Shahidzadeh, Sheila M.. Syracuse University; Estados UnidosFil: Piermartiri, Tetsade. Universidade Federal de Santa Catarina; BrasilFil: Amini, Elham. University Kebangsaan Malaysia Medical Centre; MalasiaFil: Pastor, Verónica. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Houssay. Instituto de Biología Celular y Neurociencia ; ArgentinaFil: Wilson, Yvette. University of Melbourne; AustraliaFil: Adeniyi, Philip A.. Afe Babalola University; NigeriaFil: Datusalia, Ashok K.. National Brain Research Centre; IndiaFil: Vafadari, Benham. Polish Academy of Sciences; ArgentinaFil: Saini, Vedangana. University of Nebraska; Estados UnidosFil: Suárez Pozos, Edna. Instituto Politécnico Nacional; MéxicoFil: Kushwah, Neetu. Defence Institute of Physiology and Allied Sciences; IndiaFil: Fontanet, Paula. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Houssay. Instituto de Biología Celular y Neurociencia ; ArgentinaFil: Turner, Anthony J.. University of Leeds; Reino Unid