Minimizing Control for Credit Assignment with Strong Feedback

The success of deep learning ignited interest in whether the brain learns hierarchical representations using gradient-based learning. However, current biologically plausible methods for gradient-based credit assignment in deep neural networks need infinitesimally small feedback signals, which is problematic in biologically realistic noisy environments and at odds with experimental evidence in neuroscience showing that top-down feedback can significantly influence neural activity. Building upon deep feedback control (DFC), a recently proposed credit assignment method, we combine strong feedback influences on neural activity with gradient-based learning and show that this naturally leads to a novel view on neural network optimization. Instead of gradually changing the network weights towards configurations with low output loss, weight updates gradually minimize the amount of feedback required from a controller that drives the network to the supervised output label. Moreover, we show that the use of strong feedback in DFC allows learning forward and feedback connections simultaneously, using learning rules fully local in space and time. We complement our theoretical results with experiments on standard computer-vision benchmarks, showing competitive performance to backpropagation as well as robustness to noise. Overall, our work presents a fundamentally novel view of learning as control minimization, while sidestepping biologically unrealistic assumptions

Neural networks with late-phase weights

The largely successful method of training neural networks is to learn their weights using some variant of stochastic gradient descent (SGD). Here, we show that the solutions found by SGD can be further improved by ensembling a subset of the weights in late stages of learning. At the end of learning, we obtain back a single model by taking a spatial average in weight space. To avoid incurring increased computational costs, we investigate a family of low-dimensional late-phase weight models which interact multiplicatively with the remaining parameters. Our results show that augmenting standard models with late-phase weights improves generalization in established benchmarks such as CIFAR-10/100, ImageNet and enwik8. These findings are complemented with a theoretical analysis of a noisy quadratic problem which provides a simplified picture of the late phases of neural network learning.Comment: 25 pages, 6 figure

A Theoretical Framework for Target Propagation

The success of deep learning, a brain-inspired form of AI, has sparked interest in understanding how the brain could similarly learn across multiple layers of neurons. However, the majority of biologically-plausible learning algorithms have not yet reached the performance of backpropagation (BP), nor are they built on strong theoretical foundations. Here, we analyze target propagation (TP), a popular but not yet fully understood alternative to BP, from the standpoint of mathematical optimization. Our theory shows that TP is closely related to Gauss-Newton optimization and thus substantially differs from BP. Furthermore, our analysis reveals a fundamental limitation of difference target propagation (DTP), a well-known variant of TP, in the realistic scenario of non-invertible neural networks. We provide a first solution to this problem through a novel reconstruction loss that improves feedback weight training, while simultaneously introducing architectural flexibility by allowing for direct feedback connections from the output to each hidden layer. Our theory is corroborated by experimental results that show significant improvements in performance and in the alignment of forward weight updates with loss gradients, compared to DTP.Comment: 13 pages and 4 figures in main manuscript; 41 pages and 8 figures in supplementary materia

Neural networks with late-phase weights

The largely successful method of training neural networks is to learn their weights using some variant of stochastic gradient descent (SGD). Here, we show that the solutions found by SGD can be further improved by ensembling a subset of the weights in late stages of learning. At the end of learning, we obtain back a single model by taking a spatial average in weight space. To avoid incurring increased computational costs, we investigate a family of low-dimensional late-phase weight models which interact multiplicatively with the remaining parameters. Our results show that augmenting standard models with late-phase weights improves generalization in established benchmarks such as CIFAR-10/100, ImageNet and enwik8. These findings are complemented with a theoretical analysis of a noisy quadratic problem which provides a simplified picture of the late phases of neural network learning

Continual Learning in Recurrent Neural Networks with Hypernetworks

The last decade has seen a surge of interest in continual learning (CL), and a variety of methods have been developed to alleviate catastrophic forgetting. However, most prior work has focused on tasks with static data, while CL on sequential data has remained largely unexplored. Here we address this gap in two ways. First, we evaluate the performance of established CL methods when applied to recurrent neural networks (RNNs). We primarily focus on elastic weight consolidation, which is limited by a stability-plasticity trade-off, and explore the particularities of this trade-off when using sequential data. We show that high working memory requirements, but not necessarily sequence length, lead to an increased need for stability at the cost of decreased performance on subsequent tasks. Second, to overcome this limitation we employ a recent method based on hypernetworks and apply it to RNNs to address catastrophic forgetting on sequential data. By generating the weights of a main RNN in a task-dependent manner, our approach disentangles stability and plasticity, and outperforms alternative methods in a range of experiments. Overall, our work provides several key insights on the differences between CL in feedforward networks and in RNNs, while offering a novel solution to effectively tackle CL on sequential data.Comment: 13 pages and 4 figures in the main text; 20 pages and 2 figures in the supplementary material

Efficacy of live B1 or Ulster 2C Newcastle disease vaccines simultaneously vaccinated with inactivated oil adjuvant vaccine for protection of Newcastle disease virus in broiler chickens

Two hundred, one-day-old broiler chicks were divided into groups 1, 2 and 3 containing 60, 70 and 70 chicks, respectively. The groups were divided into subgroups of 10 chicks that were vaccinated according to the following scheme: group 1 unvaccinated control, group 2 vaccinated subcutaneously at 1 day old with inactivated oil adjuvant vaccine (IOAV) in combination with live B1 vaccine. Group 3 was vaccinated in the same mode as group 2 with IOAV and live Ulster 2C vaccine. All birds were challenged when they were 28 days old. Mortality rate, body weight gain and feed conversion ratio (FCR) were monitored before and after challenge. All the chickens in group 1 died, indicating that there was no disease resistance of this unvaccinated control group of chickens. Conversely, the monitored disease resistance of chickens in groups 2 and 3 was 68.57% ± 18.64 and 88.57% ± 9.00, respectively (P < 0.05). The morbidity of chickens in groups 2 and 3 was 37.89% ± 14.36 and 14.76% ± 12.40, respectively (P < 0.05). The body weight gain, feed intake and FCR of group 3 were significantly better than those of group 2 (P < 0.05) during 1–42 days old. The simultaneous vaccination with B1 or Ulster 2C and IOAV of 1-day-old chicks gave some protection of 28-day-old broilers without a booster vaccination

Newcastle Disease Virus in Madagascar: Identification of an Original Genotype Possibly Deriving from a Died Out Ancestor of Genotype IV

In Madagascar, Newcastle disease (ND) has become enzootic after the first documented epizootics in 1946, with recurrent annual outbreaks causing mortality up to 40%. Four ND viruses recently isolated in Madagascar were genotypically and pathotypically characterised. By phylogenetic inference based on the F and HN genes, and also full-genome sequence analyses, the NDV Malagasy isolates form a cluster distant enough to constitute a new genotype hereby proposed as genotype XI. This new genotype is presumably deriving from an ancestor close to genotype IV introduced in the island probably more than 50 years ago. Our data show also that all the previously described neutralising epitopes are conserved between Malagasy and vaccine strains. However, the potential implication in vaccination failures of specific amino acid substitutions predominantly found on surface-exposed epitopes of F and HN proteins is discussed

Challenges for Using Impact Regularizers to Avoid Negative Side Effects

Designing reward functions for reinforcement learning is difficult: besides specifying which behavior is rewarded for a task, the reward also has to discourage undesired outcomes. Misspecified reward functions can lead to unintended negative side effects, and overall unsafe behavior. To overcome this problem, recent work proposed to augment the specified reward function with an impact regularizer that discourages behavior that has a big impact on the environment. Although initial results with impact regularizers seem promising in mitigating some types of side effects, important challenges remain. In this paper, we examine the main current challenges of impact regularizers and relate them to fundamental design decisions. We discuss in detail which challenges recent approaches address and which remain unsolved. Finally, we explore promising directions to overcome the unsolved challenges in preventing negative side effects with impact regularizers