1,844 research outputs found
Recommended from our members
Synaptic plasticity and memory addressing in biological and artificial neural networks
Biological brains are composed of neurons, interconnected by synapses to create large complex networks. Learning and memory occur, in large part, due to synaptic plasticity -- modifications in the efficacy of information transmission through these synaptic connections. Artificial neural networks model these with neural "units" which communicate through synaptic weights. Models of learning and memory propose synaptic plasticity rules that describe and predict the weight modifications. An equally important but under-evaluated question is the selection of \textit{which} synapses should be updated in response to a memory event. In this work, we attempt to separate the questions of synaptic plasticity from that of memory addressing.
Chapter 1 provides an overview of the problem of memory addressing and a summary of the solutions that have been considered in computational neuroscience and artificial intelligence, as well as those that may exist in biology. Chapter 2 presents in detail a solution to memory addressing and synaptic plasticity in the context of familiarity detection, suggesting strong feedforward weights and anti-Hebbian plasticity as the respective mechanisms. Chapter 3 proposes a model of recall, with storage performed by addressing through local third factors and neo-Hebbian plasticity, and retrieval by content-based addressing. In Chapter 4, we consider the problem of concurrent memory consolidation and memorization. Both storage and retrieval are performed by content-based addressing, but the plasticity rule itself is implemented by gradient descent, modulated according to whether an item should be stored in a distributed manner or memorized verbatim. However, the classical method for computing gradients in recurrent neural networks, backpropagation through time, is generally considered unbiological. In Chapter 5 we suggest a more realistic implementation through an approximation of recurrent backpropagation.
Taken together, these results propose a number of potential mechanisms for memory storage and retrieval, each of which separates the mechanism of synaptic updating -- plasticity -- from that of synapse selection -- addressing. Explicit studies of memory addressing may find applications not only in artificial intelligence but also in biology. In artificial networks, for example, selectively updating memories in large language models can help improve user privacy and security. In biological ones, understanding memory addressing can help with health outcomes and treating memory-based illnesses such as Alzheimers or PTSD
Recommended from our members
Pattern recognition in the nucleation kinetics of non-equilibrium self-assembly
Inspired by biology’s most sophisticated computer, the brain, neural networks constitute a profound reformulation of computational principles. Analogous high-dimensional, highly interconnected computational architectures also arise within information-processing molecular systems inside living cells, such as signal transduction cascades and genetic regulatory networks. Might collective modes analogous to neural computation be found more broadly in other physical and chemical processes, even those that ostensibly play non-information-processing roles? Here we examine nucleation during self-assembly of multicomponent structures, showing that high-dimensional patterns of concentrations can be discriminated and classified in a manner similar to neural network computation. Specifically, we design a set of 917 DNA tiles that can self-assemble in three alternative ways such that competitive nucleation depends sensitively on the extent of colocalization of high-concentration tiles within the three structures. The system was trained in silico to classify a set of 18 grayscale 30 × 30 pixel images into three categories. Experimentally, fluorescence and atomic force microscopy measurements during and after a 150 hour anneal established that all trained images were correctly classified, whereas a test set of image variations probed the robustness of the results. Although slow compared to previous biochemical neural networks, our approach is compact, robust and scalable. Our findings suggest that ubiquitous physical phenomena, such as nucleation, may hold powerful information-processing capabilities when they occur within high-dimensional multicomponent systems
Robustness analysis of Cohen-Grossberg neural network with piecewise constant argument and stochastic disturbances
Robustness of neural networks has been a hot topic in recent years. This paper mainly studies the robustness of the global exponential stability of Cohen-Grossberg neural networks with a piecewise constant argument and stochastic disturbances, and discusses the problem of whether the Cohen-Grossberg neural networks can still maintain global exponential stability under the perturbation of the piecewise constant argument and stochastic disturbances. By using stochastic analysis theory and inequality techniques, the interval length of the piecewise constant argument and the upper bound of the noise intensity are derived by solving transcendental equations. In the end, we offer several examples to illustrate the efficacy of the findings
Non-perturbative renormalization group analysis of nonlinear spiking networks
The critical brain hypothesis posits that neural circuits may operate close
to critical points of a phase transition, which has been argued to have
functional benefits for neural computation. Theoretical and computational
studies arguing for or against criticality in neural dynamics largely rely on
establishing power laws or scaling functions of statistical quantities, while a
proper understanding of critical phenomena requires a renormalization group
(RG) analysis. However, neural activity is typically non-Gaussian, nonlinear,
and non-local, rendering models that capture all of these features difficult to
study using standard statistical physics techniques. Here, we overcome these
issues by adapting the non-perturbative renormalization group (NPRG) to work on
(symmetric) network models of stochastic spiking neurons. By deriving a pair of
Ward-Takahashi identities and making a ``local potential approximation,'' we
are able to calculate non-universal quantities such as the effective firing
rate nonlinearity of the network, allowing improved quantitative estimates of
network statistics. We also derive the dimensionless flow equation that admits
universal critical points in the renormalization group flow of the model, and
identify two important types of critical points: in networks with an absorbing
state there is Directed Percolation (DP) fixed point corresponding to a
non-equilibrium phase transition between sustained activity and extinction of
activity, and in spontaneously active networks there is a \emph{complex valued}
critical point, corresponding to a spinodal transition observed, e.g., in the
Lee-Yang model of Ising magnets with explicitly broken symmetry. Our
Ward-Takahashi identities imply trivial dynamical exponents in
both cases, rendering it unclear whether these critical points fall into the
known DP or Ising universality classes
An investigation of entorhinal spatial representations in self-localisation behaviours
Spatial-modulated cells of the medial entorhinal cortex (MEC) and neighbouring cortices are thought to provide the neural substrate for self-localisation behaviours. These cells include grid cells of the MEC which are thought to compute path integration operations to update self-location estimates. In order to read this grid code, downstream cells are thought to reconstruct a positional estimate as a simple rate-coded representation of space.
Here, I show the coding scheme of grid cell and putative readout cells recorded from mice performing a virtual reality (VR) linear location task which engaged mice in both beaconing and path integration behaviours. I found grid cells can encode two unique coding schemes on the linear track, namely a position code which reflects periodic grid fields anchored to salient features of the track and a distance code which reflects periodic grid fields without this anchoring. Grid cells were found to switch between these coding schemes within sessions. When grid cells were encoding position, mice performed better at trials that required path integration but not on trials that required beaconing. This result provides the first mechanistic evidence linking grid cell activity to path integration-dependent behaviour.
Putative readout cells were found in the form of ramp cells which fire proportionally as a function of location in defined regions of the linear track. This ramping activity was found to be primarily explained by track position rather than other kinematic variables like speed and acceleration. These representations were found to be maintained across both trial types and outcomes indicating they likely result from recall of the track structure.
Together, these results support the functional importance of grid and ramp cells for self-localisation behaviours. Future investigations will look into the coherence between these two neural populations, which may together form a complete neural system for coding and decoding self-location in the brain
Beam scanning by liquid-crystal biasing in a modified SIW structure
A fixed-frequency beam-scanning 1D antenna based on Liquid Crystals (LCs) is designed for application in 2D scanning with lateral alignment. The 2D array environment imposes full decoupling of adjacent 1D antennas, which often conflicts with the LC requirement of DC biasing: the proposed design accommodates both. The LC medium is placed inside a Substrate Integrated Waveguide (SIW) modified to work as a Groove Gap Waveguide, with radiating slots etched on the upper broad wall, that radiates as a Leaky-Wave Antenna (LWA). This allows effective application of the DC bias voltage needed for tuning the LCs. At the same time, the RF field remains laterally confined, enabling the possibility to lay several antennas in parallel and achieve 2D beam scanning. The design is validated by simulation employing the actual properties of a commercial LC medium
Analog Photonics Computing for Information Processing, Inference and Optimisation
This review presents an overview of the current state-of-the-art in photonics
computing, which leverages photons, photons coupled with matter, and
optics-related technologies for effective and efficient computational purposes.
It covers the history and development of photonics computing and modern
analogue computing platforms and architectures, focusing on optimization tasks
and neural network implementations. The authors examine special-purpose
optimizers, mathematical descriptions of photonics optimizers, and their
various interconnections. Disparate applications are discussed, including
direct encoding, logistics, finance, phase retrieval, machine learning, neural
networks, probabilistic graphical models, and image processing, among many
others. The main directions of technological advancement and associated
challenges in photonics computing are explored, along with an assessment of its
efficiency. Finally, the paper discusses prospects and the field of optical
quantum computing, providing insights into the potential applications of this
technology.Comment: Invited submission by Journal of Advanced Quantum Technologies;
accepted version 5/06/202
Global exponential periodicity of nonlinear neural networks with multiple time-varying delays
Global exponential periodicity of nonlinear neural networks with multiple time-varying delays is investigated. Such neural networks cannot be written in the vector-matrix form because of the existence of the multiple delays. It is noted that although the neural network with multiple time-varying delays has been investigated by Lyapunov-Krasovskii functional method in the literature, the sufficient conditions in the linear matrix inequality form have not been obtained. Two sets of sufficient conditions in the linear matrix inequality form are established by Lyapunov-Krasovskii functional and linear matrix inequality to ensure that two arbitrary solutions of the neural network with multiple delays attract each other exponentially. This is a key prerequisite to prove the existence, uniqueness, and global exponential stability of periodic solutions. Some examples are provided to demonstrate the effectiveness of the established results. We compare the established theoretical results with the previous results and show that the previous results are not applicable to the systems in these examples
Recommended from our members
Exploring the socioeconomic and environmental factors influencing smallholder macadamia production and productivity in Malawi.
Macadamia (Macadamia integrifolia Maiden & Betche) is a highly valued crop in Malawi. The crop is a vital source of food security and ecosystem services, and its high-export cash value makes it a key contributor to the country's economy. Malawi ranks seventh in global macadamia production, comprising two subsectors: smallholders and commercial estates. However, significant yield gaps have been reported between smallholder and commercial estate producers. While commercial estates achieve higher average annual tree yields (30 kg), smallholder yields remain consistently low, averaging at or below 10 kg tree-1 year-1. Improving macadamia productivity among smallholders can help reduce poverty, improve household food security, and promote economic growth in Malawi.
Despite the significant contributions of smallholders in the Malawian macadamia subsector, research on the factors influencing the crop's productivity has primarily focused on commercial estate production. To address this knowledge gap, this Ph.D thesis focuses on smallholder macadamia production in Malawi. The thesis examines the socioeconomic characteristics of smallholder macadamia farmers, including demographics, cultivar preferences, and production constraints. Secondly, it evaluates the climatic factors influencing smallholder macadamia production and predicts the current and future suitable geographical areas for the crop. Lastly, it assesses the soil fertility status of smallholder macadamia farms in relation to macadamia production requirements.
Results of this study reveal that the majority (62%) of macadamia smallholders are over 50 years of age and consider farming their main occupation. However, this poses significant risks to the macadamia subsector, as older farmers are risk-averse and less innovative, hindering their willingness to adopt new agricultural technologies and ability to learn. Regarding cultivar preferences, the study finds that smallholder macadamia farmers prefer high-yielding cultivars with superior nut qualities, such as large and heavy nuts, and extended flowering periods. The most preferred macadamia cultivars in descending order are Hawaiian Agricultural Experimental Station (HAES) 660, 800, 816, and 246, which are the "core" of established cultivars in Malawi. The study identifies insect pests, diseases, market availability, strong winds, and a lack of agricultural extension services as the most significant challenges affecting smallholder macadamia farmers.
The study's suitability analysis reveals that the ensemble model has an excellent fit and high performance in predicting the current agro-climatically suitable areas for macadamia production (AUC = 0.90). The findings show that precipitation related variables (60.2%) are more important in determining the suitable areas for growing macadamia than temperature related variables (39.8%). The model results show that 57% (53,925 km2) of Malawi is currently suitable for macadamia cultivation, with the central region having the highest suitability (25.8%, 24,327 km2) and the southern region the lowest (10.7%, 10,257 km2). Optimal suitability (26%, 24,565 km2) is observed in the highland areas with elevations ranging from 1000–1400 metres above sea level (m.a.s.l.). Under the intermediate emission scenario (RCP 4.5) and the pessimistic scenario (RCP 8.5), the impact models predict net losses of 18% (17,015 km2) and 21.6% (20,414 km2), respectively, in the extent of suitable areas for macadamia in the 2050s.
The results of the soil fertility analysis indicate suboptimal fertility among the sampled macadamia farms. The majority of the soils are strongly acidic and deficient in essential nutrients required for the healthy growth of macadamia trees. Moreover, the average cation exchange capacity (1.67 cmol (+) kg-1) and the soil organic matter content (≤ 1%) are below the minimum optimal levels required for macadamia trees. These findings indicate that soil fertility is one of the primary limiting factors to the crop's productivity, even in areas with suitable climatic conditions. Therefore, addressing the soil fertility issues is crucial to improving the land suitability of the smallholder farms for macadamia, which can lead to optimal yields.
This study extends the frontiers of knowledge concerning the macadamia subsector in Malawi by providing insights into the smallholder macadamia farming systems, including demographics, cultivar preferences, and production constraints. It also provides novel empirical evidence on the climate factors that influence the suitability of rainfed macadamia cultivation and identifies current and future suitable growing areas in the country. Additionally, the study addresses the research gap on the soil fertility status of Malawian smallholder macadamia farms. Therefore, the findings of this research have practical implications for various areas such as macadamia cultivar introductions and breeding, land use planning, soil fertility management, and policy formulation for agricultural extension services, inputs, and marketing of the crop
Breaking Implicit Assumptions of Physical Delay-Feedback Reservoir Computing
The Reservoir Computing (RC) paradigm is a supervised machine learning scheme using the natural computational ability of dynamical systems. Such dynamical systems incorporate time delays showcasing intricate dynamics. This richness in dynamics, particularly the system's transient response to external stimuli makes them suitable for RC. A subset of RCs, Delay-Feedback Reservoir Computing (DFRC), is distinguished by its unique features: a system that consists of a single nonlinear node and a delay-line, with `virtual' nodes defined along the delay-line by time-multiplexing procedure of the input. These characteristics render DFRC particularly useful for hardware integration. In this thesis, the aim is to break the implicit assumptions made in the design of physical DFRC based on Mackey-Glass dynamical system.
The first assumption we address is the performance of DFRC is not affected by the attenuation in physcial delay-line as the nodes defined along it are 'virtual'. However, our experimental results contradict this. To mitigate the impact of losses along the delay line, we propose a methodology `Devirtualisation', which describes the procedure of directly tapping into the delay lines at the position of a `virtual' node, rather than at the delay line's end. It trade-offs the DFRC system's read-out frequency and the quantity of output lines. Masking plays a crucial role in DFRC, as it defines `virtual' nodes along the delay-line. The second assumption is that the mask used should randomly generated numbers uniformly distributed between [-u,u]. We experimentally compare Binary Weight Mask (BWM) vs. Random Weight Mask (RWM) under different scenarios; and investigate the randomness of BWM signal distribution's impact. The third implicit assumption is that, DFRC is designed to solve time series prediction tasks involving a single input and output with no external feedback. To break this assumption, we propose two approaches to mix multi-input signals into DFRC; to validate these approaches, a novel task for DFRC that inherently necessitates multiple inputs: the control of a forced Van der Pol oscillator system, is proposed
- …