2,669 research outputs found
Recommended from our members
Synaptic plasticity and memory addressing in biological and artificial neural networks
Biological brains are composed of neurons, interconnected by synapses to create large complex networks. Learning and memory occur, in large part, due to synaptic plasticity -- modifications in the efficacy of information transmission through these synaptic connections. Artificial neural networks model these with neural "units" which communicate through synaptic weights. Models of learning and memory propose synaptic plasticity rules that describe and predict the weight modifications. An equally important but under-evaluated question is the selection of \textit{which} synapses should be updated in response to a memory event. In this work, we attempt to separate the questions of synaptic plasticity from that of memory addressing.
Chapter 1 provides an overview of the problem of memory addressing and a summary of the solutions that have been considered in computational neuroscience and artificial intelligence, as well as those that may exist in biology. Chapter 2 presents in detail a solution to memory addressing and synaptic plasticity in the context of familiarity detection, suggesting strong feedforward weights and anti-Hebbian plasticity as the respective mechanisms. Chapter 3 proposes a model of recall, with storage performed by addressing through local third factors and neo-Hebbian plasticity, and retrieval by content-based addressing. In Chapter 4, we consider the problem of concurrent memory consolidation and memorization. Both storage and retrieval are performed by content-based addressing, but the plasticity rule itself is implemented by gradient descent, modulated according to whether an item should be stored in a distributed manner or memorized verbatim. However, the classical method for computing gradients in recurrent neural networks, backpropagation through time, is generally considered unbiological. In Chapter 5 we suggest a more realistic implementation through an approximation of recurrent backpropagation.
Taken together, these results propose a number of potential mechanisms for memory storage and retrieval, each of which separates the mechanism of synaptic updating -- plasticity -- from that of synapse selection -- addressing. Explicit studies of memory addressing may find applications not only in artificial intelligence but also in biology. In artificial networks, for example, selectively updating memories in large language models can help improve user privacy and security. In biological ones, understanding memory addressing can help with health outcomes and treating memory-based illnesses such as Alzheimers or PTSD
Recommended from our members
Pattern recognition in the nucleation kinetics of non-equilibrium self-assembly
Inspired by biology’s most sophisticated computer, the brain, neural networks constitute a profound reformulation of computational principles. Analogous high-dimensional, highly interconnected computational architectures also arise within information-processing molecular systems inside living cells, such as signal transduction cascades and genetic regulatory networks. Might collective modes analogous to neural computation be found more broadly in other physical and chemical processes, even those that ostensibly play non-information-processing roles? Here we examine nucleation during self-assembly of multicomponent structures, showing that high-dimensional patterns of concentrations can be discriminated and classified in a manner similar to neural network computation. Specifically, we design a set of 917 DNA tiles that can self-assemble in three alternative ways such that competitive nucleation depends sensitively on the extent of colocalization of high-concentration tiles within the three structures. The system was trained in silico to classify a set of 18 grayscale 30 × 30 pixel images into three categories. Experimentally, fluorescence and atomic force microscopy measurements during and after a 150 hour anneal established that all trained images were correctly classified, whereas a test set of image variations probed the robustness of the results. Although slow compared to previous biochemical neural networks, our approach is compact, robust and scalable. Our findings suggest that ubiquitous physical phenomena, such as nucleation, may hold powerful information-processing capabilities when they occur within high-dimensional multicomponent systems
Non-perturbative renormalization group analysis of nonlinear spiking networks
The critical brain hypothesis posits that neural circuits may operate close
to critical points of a phase transition, which has been argued to have
functional benefits for neural computation. Theoretical and computational
studies arguing for or against criticality in neural dynamics largely rely on
establishing power laws or scaling functions of statistical quantities, while a
proper understanding of critical phenomena requires a renormalization group
(RG) analysis. However, neural activity is typically non-Gaussian, nonlinear,
and non-local, rendering models that capture all of these features difficult to
study using standard statistical physics techniques. Here, we overcome these
issues by adapting the non-perturbative renormalization group (NPRG) to work on
(symmetric) network models of stochastic spiking neurons. By deriving a pair of
Ward-Takahashi identities and making a ``local potential approximation,'' we
are able to calculate non-universal quantities such as the effective firing
rate nonlinearity of the network, allowing improved quantitative estimates of
network statistics. We also derive the dimensionless flow equation that admits
universal critical points in the renormalization group flow of the model, and
identify two important types of critical points: in networks with an absorbing
state there is Directed Percolation (DP) fixed point corresponding to a
non-equilibrium phase transition between sustained activity and extinction of
activity, and in spontaneously active networks there is a \emph{complex valued}
critical point, corresponding to a spinodal transition observed, e.g., in the
Lee-Yang model of Ising magnets with explicitly broken symmetry. Our
Ward-Takahashi identities imply trivial dynamical exponents in
both cases, rendering it unclear whether these critical points fall into the
known DP or Ising universality classes
Novel Neural Network Applications to Mode Choice in Transportation: Estimating Value of Travel Time and Modelling Psycho-Attitudinal Factors
Whenever researchers wish to study the behaviour of individuals choosing among a set of alternatives, they usually rely on models based on the random utility theory, which postulates that the single individuals modify their behaviour so that they can maximise of their utility. These models, often identified as discrete choice models (DCMs), usually require the definition of the utilities for each alternative, by first identifying the variables influencing the decisions. Traditionally, DCMs focused on observable variables and treated users as optimizing tools with predetermined needs. However, such an approach is in contrast with the results from studies in social sciences which show that choice behaviour can be influenced by psychological factors such as attitudes and preferences. Recently there have been formulations of DCMs which include latent constructs for capturing the impact of subjective factors. These are called hybrid choice models or integrated choice and latent variable models (ICLV). However, DCMs are not exempt from issues, like, the fact that researchers have to choose the variables to include and their relations to define the utilities. This is probably one of the reasons which has recently lead to an influx of numerous studies using machine learning (ML) methods to study mode choice, in which researchers tried to find alternative methods to analyse travellers’ choice behaviour. A ML algorithm is any generic method that uses the data itself to understand and build a model, improving its performance the more it is allowed to learn. This means they do not require any a priori input or hypotheses on the structure and nature of the relationships between the several variables used as its inputs. ML models are usually considered black-box methods, but whenever researchers felt the need for interpretability of ML results, they tried to find alternative ways to use ML methods, like building them by using some a priori knowledge to induce specific constrains. Some researchers also transformed the outputs of ML algorithms so that they could be interpreted from an economic point of view, or built hybrid ML-DCM models. The object of this thesis is that of investigating the benefits and the disadvantages deriving from adopting either DCMs or ML methods to study the phenomenon of mode choice in transportation. The strongest feature of DCMs is the fact that they produce very precise and descriptive results, allowing for a thorough interpretation of their outputs. On the other hand, ML models offer a substantial benefit by being truly data-driven methods and thus learning most relations from the data itself. As a first contribution, we tested an alternative method for calculating the value of travel time (VTT) through the results of ML algorithms. VTT is a very informative parameter to consider, since the time consumed by individuals whenever they need to travel normally represents an undesirable factor, thus they are usually willing to exchange their money to reduce travel times. The method proposed is independent from the mode-choice functions, so it can be applied to econometric models and ML methods equally, if they allow the estimation of individual level probabilities. Another contribution of this thesis is a neural network (NN) for the estimation of choice models with latent variables as an alternative to DCMs. This issue arose from wanting to include in ML models not only level of service variables of the alternatives, and socio-economic attributes of the individuals, but also psycho-attitudinal indicators, to better describe the influence of psychological factors on choice behaviour. The results were estimated by using two different datasets. Since NN results are dependent on the values of their hyper-parameters and on their initialization, several NNs were estimated by using different hyper-parameters to find the optimal values, which were used to verify the stability of the results with different initializations
AI: Limits and Prospects of Artificial Intelligence
The emergence of artificial intelligence has triggered enthusiasm and promise of boundless opportunities as much as uncertainty about its limits. The contributions to this volume explore the limits of AI, describe the necessary conditions for its functionality, reveal its attendant technical and social problems, and present some existing and potential solutions. At the same time, the contributors highlight the societal and attending economic hopes and fears, utopias and dystopias that are associated with the current and future development of artificial intelligence
An Intelligent Time and Performance Efficient Algorithm for Aircraft Design Optimization
Die Optimierung des Flugzeugentwurfs erfordert die Beherrschung der komplexen Zusammenhänge mehrerer Disziplinen. Trotz seiner Abhängigkeit von einer Vielzahl unabhängiger Variablen zeichnet sich dieses komplexe Entwurfsproblem durch starke indirekte Verbindungen und eine daraus resultierende geringe Anzahl lokaler Minima aus. Kürzlich entwickelte intelligente Methoden, die auf selbstlernenden Algorithmen basieren, ermutigten die Suche nach einer diesem Bereich zugeordneten neuen Methode. Tatsächlich wird der in dieser Arbeit entwickelte Hybrid-Algorithmus (Cavus) auf zwei Hauptdesignfälle im Luft- und Raumfahrtbereich angewendet: Flugzeugentwurf- und Flugbahnoptimierung. Der implementierte neue Ansatz ist in der Lage, die Anzahl der Versuchspunkte ohne große Kompromisse zu reduzieren. Die Trendanalyse zeigt, dass der Cavus-Algorithmus für die komplexen Designprobleme, mit einer proportionalen Anzahl von Prüfpunkten konservativer ist, um die erfolgreichen Muster zu finden.
Aircraft Design Optimization requires mastering of the complex interrelationships of multiple disciplines. Despite its dependency on a diverse number of independent variables, this complex design problem has favourable nature as having strong indirect links and as a result a low number of local minimums. Recently developed intelligent methods that are based on self-learning algorithms encouraged finding a new method dedicated to this area. Indeed, the hybrid (Cavus) algorithm developed in this thesis is applied two main design cases in aerospace area: aircraft design optimization and trajectory optimization. The implemented new approach is capable of reducing the number of trial points without much compromise. The trend analysis shows that, for the complex design problems the Cavus algorithm is more conservative with a proportional number of trial points in finding the successful patterns
Beam scanning by liquid-crystal biasing in a modified SIW structure
A fixed-frequency beam-scanning 1D antenna based on Liquid Crystals (LCs) is designed for application in 2D scanning with lateral alignment. The 2D array environment imposes full decoupling of adjacent 1D antennas, which often conflicts with the LC requirement of DC biasing: the proposed design accommodates both. The LC medium is placed inside a Substrate Integrated Waveguide (SIW) modified to work as a Groove Gap Waveguide, with radiating slots etched on the upper broad wall, that radiates as a Leaky-Wave Antenna (LWA). This allows effective application of the DC bias voltage needed for tuning the LCs. At the same time, the RF field remains laterally confined, enabling the possibility to lay several antennas in parallel and achieve 2D beam scanning. The design is validated by simulation employing the actual properties of a commercial LC medium
Outcome-Oriented Prescriptive Process Monitoring Based on Temporal Logic Patterns
Prescriptive Process Monitoring systems recommend, during the execution of a
business process, interventions that, if followed, prevent a negative outcome
of the process. Such interventions have to be reliable, that is, they have to
guarantee the achievement of the desired outcome or performance, and they have
to be flexible, that is, they have to avoid overturning the normal process
execution or forcing the execution of a given activity. Most of the existing
Prescriptive Process Monitoring solutions, however, while performing well in
terms of recommendation reliability, provide the users with very specific
(sequences of) activities that have to be executed without caring about the
feasibility of these recommendations. In order to face this issue, we propose a
new Outcome-Oriented Prescriptive Process Monitoring system recommending
temporal relations between activities that have to be guaranteed during the
process execution in order to achieve a desired outcome. This softens the
mandatory execution of an activity at a given point in time, thus leaving more
freedom to the user in deciding the interventions to put in place. Our approach
defines these temporal relations with Linear Temporal Logic over finite traces
patterns that are used as features to describe the historical process data
recorded in an event log by the information systems supporting the execution of
the process. Such encoded log is used to train a Machine Learning classifier to
learn a mapping between the temporal patterns and the outcome of a process
execution. The classifier is then queried at runtime to return as
recommendations the most salient temporal patterns to be satisfied to maximize
the likelihood of a certain outcome for an input ongoing process execution. The
proposed system is assessed using a pool of 22 real-life event logs that have
already been used as a benchmark in the Process Mining community.Comment: 38 pages, 6 figures, 8 table
- …