Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence

Incremental learning (IL) has received a lot of attention recently, however, the literature lacks a precise problem definition, proper evaluation settings, and metrics tailored specifically for the IL problem. One of the main objectives of this work is to fill these gaps so as to provide a common ground for better understanding of IL. The main challenge for an IL algorithm is to update the classifier whilst preserving existing knowledge. We observe that, in addition to forgetting, a known issue while preserving knowledge, IL also suffers from a problem we call intransigence, inability of a model to update its knowledge. We introduce two metrics to quantify forgetting and intransigence that allow us to understand, analyse, and gain better insights into the behaviour of IL algorithms. We present RWalk, a generalization of EWC++ (our efficient version of EWC [Kirkpatrick2016EWC]) and Path Integral [Zenke2017Continual] with a theoretically grounded KL-divergence based perspective. We provide a thorough analysis of various IL algorithms on MNIST and CIFAR-100 datasets. In these experiments, RWalk obtains superior results in terms of accuracy, and also provides a better trade-off between forgetting and intransigence

Scalable Recollections for Continual Lifelong Learning

Given the recent success of Deep Learning applied to a variety of single tasks, it is natural to consider more human-realistic settings. Perhaps the most difficult of these settings is that of continual lifelong learning, where the model must learn online over a continuous stream of non-stationary data. A successful continual lifelong learning system must have three key capabilities: it must learn and adapt over time, it must not forget what it has learned, and it must be efficient in both training time and memory. Recent techniques have focused their efforts primarily on the first two capabilities while questions of efficiency remain largely unexplored. In this paper, we consider the problem of efficient and effective storage of experiences over very large time-frames. In particular we consider the case where typical experiences are O(n) bits and memories are limited to O(k) bits for k << n. We present a novel scalable architecture and training algorithm in this challenging domain and provide an extensive evaluation of its performance. Our results show that we can achieve considerable gains on top of state-of-the-art methods such as GEM.Comment: AAAI 201

The Performance of Dialysis Care: Routinization and Adaptation on the Floor

Previous studies of communication in dialysis centers primarily focused on communication between nurses and patients. In this study, ethnographic methods were used to explore the dominant communication performances enacted by dialysis staff members, including registered nurses, patient care technicians, technical aides, a social worker, and a dietitian. Findings suggest a dialectic between extreme routinization of care and continual adaptation. The dominant routine involved repeating the same preparation, treatment, and discharge process 3 shifts per day, thrice weekly for each patient. At the same time, near-constant adjustments to scheduling, coordination of tasks, and problem solving were needed to maintain the performance of repetition. The balancing of this dialectic has significant implications for new staff training and socialization, understanding the role of technology and routine in dialysis and in health care systems more generally, and in further theorizing the role of unbounded communication interactions in health care

Solicitors' CPD: time to change from regulatory stick to regulatory carrot?

Summary: The legal professions are agreed on the need for some form of continuing professional development (“CPD”) after qualification. What is less clear is the intention of such frameworks and in contrast to other forms of more diffuse learning in the workplace. I will explore two areas of tension in the current solicitors‟ CPD system which will bear attention before any of these three related objectives can be achieved: Between a didactic form of delivery focussing on technical updating of knowledge of law and procedure and more “difficult” participative CPD activity; Between accountability, regulation and personal development as drivers behind the CPD scheme dear to different stakeholders. The paper will conclude that, whilst the paradigm shift apparent in the regulators and the professional body is to be welcomed, a change of culture in the profession as a whole is required. This requires CPD, in partnership with other forms of learning, to be viewed in terms of outputs and benefits: the carrots of the title. It is not only a negligence-avoiding maintenance of a static level of competence but a mechanism to address the change which will inevitably result from the full implementation of the Legal Services Act 2007