Class-Incremental Learning with Repetition

Real-world data streams naturally include the repetition of previous concepts. From a Continual Learning (CL) perspective, repetition is a property of the environment and, unlike replay, cannot be controlled by the agent. Nowadays, the Class-Incremental (CI) scenario represents the leading test-bed for assessing and comparing CL strategies. This scenario type is very easy to use, but it never allows revisiting previously seen classes, thus completely neglecting the role of repetition. We focus on the family of Class-Incremental with Repetition (CIR) scenario, where repetition is embedded in the definition of the stream. We propose two stochastic stream generators that produce a wide range of CIR streams starting from a single dataset and a few interpretable control parameters. We conduct the first comprehensive evaluation of repetition in CL by studying the behavior of existing CL strategies under different CIR streams. We then present a novel replay strategy that exploits repetition and counteracts the natural imbalance present in the stream. On both CIFAR100 and TinyImageNet, our strategy outperforms other replay approaches, which are not designed for environments with repetition.Comment: Accepted to the 2nd Conference on Lifelong Learning Agents (CoLLAs), 2023 19 page

Continuous Coordination As a Realistic Scenario for Lifelong Learning

Current deep reinforcement learning (RL) algorithms are still highly task-specific and lack the ability to generalize to new environments. Lifelong learning (LLL), however, aims at solving multiple tasks sequentially by efficiently transferring and using knowledge between tasks. Despite a surge of interest in lifelong RL in recent years, the lack of a realistic testbed makes robust evaluation of LLL algorithms difficult. Multi-agent RL (MARL), on the other hand, can be seen as a natural scenario for lifelong RL due to its inherent non-stationarity, since the agents' policies change over time. In this work, we introduce a multi-agent lifelong learning testbed that supports both zero-shot and few-shot settings. Our setup is based on Hanabi -- a partially-observable, fully cooperative multi-agent game that has been shown to be challenging for zero-shot coordination. Its large strategy space makes it a desirable environment for lifelong RL tasks. We evaluate several recent MARL methods, and benchmark state-of-the-art LLL algorithms in limited memory and computation regimes to shed light on their strengths and weaknesses. This continual learning paradigm also provides us with a pragmatic way of going beyond centralized training which is the most commonly used training protocol in MARL. We empirically show that the agents trained in our setup are able to coordinate well with unseen agents, without any additional assumptions made by previous works. The code and all pre-trained models are available at https://github.com/chandar-lab/Lifelong-Hanabi.Comment: 19 pages with supplementary materials. Added results for Lifelong RL methods and some future work. Accepted to ICML 202

Does Continual Learning = Catastrophic Forgetting?

Continual learning is known for suffering from catastrophic forgetting, a phenomenon where earlier learned concepts are forgotten at the expense of more recent samples. In this work, we challenge the assumption that continual learning is inevitably associated with catastrophic forgetting by presenting a set of tasks that surprisingly do not suffer from catastrophic forgetting when learned continually. We provide evidence that these reconstruction-type tasks exhibit positive forward transfer and that single-view 3D shape reconstruction improves the performance on learned and novel categories over time. We provide the novel analysis of knowledge transfer ability by looking at the output distribution shift across sequential learning tasks. Finally, we show that the robustness of these tasks leads to the potential of having a proxy representation learning task for continual classification. The codebase, dataset, and pre-trained models released with this article can be found at https://github.com/rehg-lab/CLRec

Development and research of a neural network alternate incremental learning algorithm

В работе показывается актуальность разработки инкрементных методов и алгоритмов обучения нейронной сети. Представлены семейства техник инкрементного обучения. Проведена оценка возможности применения машины экстремального обучения как инкрементного обучения. Эксперименты показывают возможность использования машины экстремального обучения как инкрементного обучения, однако при увеличении числа обучающих примеров нейронная сеть становится непригодна для дальнейшего обучения. Для решения данной проблемы предложен алгоритм инкрементного обучения нейронной сети, основанный на поочередном применении машины экстремального обучения для корректировки весов только выходного слоя сети (состояние функционирования) и метода обратного распространения ошибки (глубокого обучения) для корректировки всех весов сети (состояние сна). Полагается, что в ходе состояния функционирования нейронная сеть выдает результаты или учится на новых задачах, а в состоянии сна оптимизирует свои весовые коэффициенты. Особенностью предложенного алгоритма является его возможность адаптироваться в реальном масштабе времени под изменение внешних условий на этапе функционирования. На примере решения задачи аппроксимации показана эффективность предложенного алгоритма. Представлены результаты аппроксимации после выполнения каждого шага алгоритма. Приведено сравнение значений среднеквадратической ошибки при использовании машины экстремального обучения как инкрементного обучения и разработанного алгоритма посменного инкрементного обучения нейронной сети