Diffusion-based neuromodulation can eliminate catastrophic forgetting in simple neural networks

A long-term goal of AI is to produce agents that can learn a diversity of skills throughout their lifetimes and continuously improve those skills via experience. A longstanding obstacle towards that goal is catastrophic forgetting, which is when learning new information erases previously learned information. Catastrophic forgetting occurs in artificial neural networks (ANNs), which have fueled most recent advances in AI. A recent paper proposed that catastrophic forgetting in ANNs can be reduced by promoting modularity, which can limit forgetting by isolating task information to specific clusters of nodes and connections (functional modules). While the prior work did show that modular ANNs suffered less from catastrophic forgetting, it was not able to produce ANNs that possessed task-specific functional modules, thereby leaving the main theory regarding modularity and forgetting untested. We introduce diffusion-based neuromodulation, which simulates the release of diffusing, neuromodulatory chemicals within an ANN that can modulate (i.e. up or down regulate) learning in a spatial region. On the simple diagnostic problem from the prior work, diffusion-based neuromodulation 1) induces task-specific learning in groups of nodes and connections (task-specific localized learning), which 2) produces functional modules for each subtask, and 3) yields higher performance by eliminating catastrophic forgetting. Overall, our results suggest that diffusion-based neuromodulation promotes task-specific localized learning and functional modularity, which can help solve the challenging, but important problem of catastrophic forgetting

Exploring the effects of robotic design on learning and neural control

The ongoing deep learning revolution has allowed computers to outclass humans in various games and perceive features imperceptible to humans during classification tasks. Current machine learning techniques have clearly distinguished themselves in specialized tasks. However, we have yet to see robots capable of performing multiple tasks at an expert level. Most work in this field is focused on the development of more sophisticated learning algorithms for a robot's controller given a largely static and presupposed robotic design. By focusing on the development of robotic bodies, rather than neural controllers, I have discovered that robots can be designed such that they overcome many of the current pitfalls encountered by neural controllers in multitask settings. Through this discovery, I also present novel metrics to explicitly measure the learning ability of a robotic design and its resistance to common problems such as catastrophic interference. Traditionally, the physical robot design requires human engineers to plan every aspect of the system, which is expensive and often relies on human intuition. In contrast, within the field of evolutionary robotics, evolutionary algorithms are used to automatically create optimized designs, however, such designs are often still limited in their ability to perform in a multitask setting. The metrics created and presented here give a novel path to automated design that allow evolved robots to synergize with their controller to improve the computational efficiency of their learning while overcoming catastrophic interference. Overall, this dissertation intimates the ability to automatically design robots that are more general purpose than current robots and that can perform various tasks while requiring less computation.Comment: arXiv admin note: text overlap with arXiv:2008.0639

Adaptive Reorganization of Neural Pathways for Continual Learning with Spiking Neural Networks

The human brain can self-organize rich and diverse sparse neural pathways to incrementally master hundreds of cognitive tasks. However, most existing continual learning algorithms for deep artificial and spiking neural networks are unable to adequately auto-regulate the limited resources in the network, which leads to performance drop along with energy consumption rise as the increase of tasks. In this paper, we propose a brain-inspired continual learning algorithm with adaptive reorganization of neural pathways, which employs Self-Organizing Regulation networks to reorganize the single and limited Spiking Neural Network (SOR-SNN) into rich sparse neural pathways to efficiently cope with incremental tasks. The proposed model demonstrates consistent superiority in performance, energy consumption, and memory capacity on diverse continual learning tasks ranging from child-like simple to complex tasks, as well as on generalized CIFAR100 and ImageNet datasets. In particular, the SOR-SNN model excels at learning more complex tasks as well as more tasks, and is able to integrate the past learned knowledge with the information from the current task, showing the backward transfer ability to facilitate the old tasks. Meanwhile, the proposed model exhibits self-repairing ability to irreversible damage and for pruned networks, could automatically allocate new pathway from the retained network to recover memory for forgotten knowledge

A computational model for continual learning and synaptic consolidation

How humans are able to learn and memorize is a long-standing question in science. Much progress has been achieved in recent decades to answer this question but the are still many open problems. One of these problems refers to the human ability to learn several tasks in sequence without forgetting. In neuronal networks learning can interfere with pre-existing memories when the network is engaged in continual learning. The interference is particularly pronounced if, for instance, similar sensory stimuli require different responses depending on the context. Unlike in humans, this can lead to a memory loss termed catastrophic forgetting. To avoid interference and its fatal consequences, only a subset of synaptic weights should be consolidated. In this work we propose as computational model which performs selective consolidation by incorporating the synaptic tagging and capture hypothesis. This hypothesis, well grounded by experimental evidences, claims that synaptic consolidation requires both a synaptic-specific tag and diffusible plasticity-related proteins. We show that synaptic tagging and capture can be modeled by two classes of synaptic processes acting on different time scales. The two classes, characterized whether protein synthesis is required, are represented in our model by two synaptic components interacting with each other. With our approach we demonstrate that synaptic consolidation can not only diminishes the problem of catastrophic forgetting during continual learning but also enables fast learning through strongly changing synaptic strengths during the early phase of long-term potentiation. The model reproduces various experimental observations on synaptic tagging and cross-tagging. It also explains why learning in psychophysical experiments is hampered when different types of stimuli are randomly intermixed

Practopoiesis: Or how life fosters a mind

The mind is a biological phenomenon. Thus, biological principles of organization should also be the principles underlying mental operations. Practopoiesis states that the key for achieving intelligence through adaptation is an arrangement in which mechanisms laying a lower level of organization, by their operations and interaction with the environment, enable creation of mechanisms lying at a higher level of organization. When such an organizational advance of a system occurs, it is called a traverse. A case of traverse is when plasticity mechanisms (at a lower level of organization), by their operations, create a neural network anatomy (at a higher level of organization). Another case is the actual production of behavior by that network, whereby the mechanisms of neuronal activity operate to create motor actions. Practopoietic theory explains why the adaptability of a system increases with each increase in the number of traverses. With a larger number of traverses, a system can be relatively small and yet, produce a higher degree of adaptive/intelligent behavior than a system with a lower number of traverses. The present analyses indicate that the two well-known traverses-neural plasticity and neural activity-are not sufficient to explain human mental capabilities. At least one additional traverse is needed, which is named anapoiesis for its contribution in reconstructing knowledge e.g., from long-term memory into working memory. The conclusions bear implications for brain theory, the mind-body explanatory gap, and developments of artificial intelligence technologies.Comment: Revised version in response to reviewer comment

Task Difficulty Aware Parameter Allocation & Regularization for Lifelong Learning

Parameter regularization or allocation methods are effective in overcoming catastrophic forgetting in lifelong learning. However, they solve all tasks in a sequence uniformly and ignore the differences in the learning difficulty of different tasks. So parameter regularization methods face significant forgetting when learning a new task very different from learned tasks, and parameter allocation methods face unnecessary parameter overhead when learning simple tasks. In this paper, we propose the Parameter Allocation & Regularization (PAR), which adaptively select an appropriate strategy for each task from parameter allocation and regularization based on its learning difficulty. A task is easy for a model that has learned tasks related to it and vice versa. We propose a divergence estimation method based on the Nearest-Prototype distance to measure the task relatedness using only features of the new task. Moreover, we propose a time-efficient relatedness-aware sampling-based architecture search strategy to reduce the parameter overhead for allocation. Experimental results on multiple benchmarks demonstrate that, compared with SOTAs, our method is scalable and significantly reduces the model's redundancy while improving the model's performance. Further qualitative analysis indicates that PAR obtains reasonable task-relatedness.Comment: Accepted by CVPR2023. Code is available at https://github.com/WenjinW/PA

Brain Learning, Attention, and Consciousness

The processes whereby our brains continue to learn about a changing world in a stable fashion throughout life are proposed to lead to conscious experiences. These processes include the learning of top-down expectations, the matching of these expectations against bottom-up data, the focusing of attention upon the expected clusters of information, and the development of resonant states between bottom-up and top-down processes as they reach an attentive consensus between what is expected and what is there in the outside world. It is suggested that all conscious states in the brain are resonant states, and that these resonant states trigger learning of sensory and cognitive representations. The model which summarize these concepts are therefore called Adaptive Resonance Theory, or ART, models. Psychophysical and neurobiological data in support of ART are presented from early vision, visual object recognition, auditory streaming, variable-rate speech perception, somatosensory perception, and cognitive-emotional interactions, among others. It is noted that ART mechanisms seem to be operative at all levels of the visual system, and it is proposed how these mechanisms are realized by known laminar circuits of visual cortex. It is predicted that the same circuit realization of ART mechanisms will be found in the laminar circuits of all sensory and cognitive neocortex. Concepts and data are summarized concerning how some visual percepts may be visibly, or modally, perceived, whereas amoral percepts may be consciously recognized even though they are perceptually invisible. It is also suggested that sensory and cognitive processing in the What processing stream of the brain obey top-down matching and learning laws that arc often complementary to those used for spatial and motor processing in the brain's Where processing stream. This enables our sensory and cognitive representations to maintain their stability a.s we learn more about the world, while allowing spatial and motor representations to forget learned maps and gains that are no longer appropriate as our bodies develop and grow from infanthood to adulthood. Procedural memories are proposed to be unconscious because the inhibitory matching process that supports these spatial and motor processes cannot lead to resonance.Defense Advance Research Projects Agency; Office of Naval Research (N00014-95-1-0409, N00014-95-1-0657); National Science Foundation (IRI-97-20333

Localist representation can improve efficiency for detection and counting

Almost all representations have both distributed and localist aspects, depending upon what properties of the data are being considered. With noisy data, features represented in a localist way can be detected very efficiently, and in binary representations they can be counted more efficiently than those represented in a distributed way. Brains operate in noisy environments, so the localist representation of behaviourally important events is advantageous, and fits what has been found experimentally. Distributed representations require more neurons to perform as efficiently, but they do have greater versatility