Data and Computation Efficient Meta-Learning

In order to make predictions with high accuracy, conventional deep learning systems require large training datasets consisting of thousands or millions of examples and long training times measured in hours or days, consuming high levels of electricity with a negative impact on our environment. It is desirable to have have machine learning systems that can emulate human behavior such that they can quickly learn new concepts from only a few examples. This is especially true if we need to quickly customize or personalize machine learning models to specific scenarios where it would be impractical to acquire a large amount of training data and where a mobile device is the means for computation. We define a data efficient machine learning system to be one that can learn a new concept from only a few examples (or shots) and a computation efficient machine learning system to be one that can learn a new concept rapidly without retraining on an everyday computing device such as a smart phone. In this work, we design, develop, analyze, and extend the theory of machine learning systems that are both data efficient and computation efficient. We present systems that are trained using multiple tasks such that it "learns how to learn" to solve new tasks from only a few examples. These systems can efficiently solve new, unseen tasks drawn from a broad range of data distributions, in both the low and high data regimes, without the need for costly retraining. Adapting to a new task requires only a forward pass of the example task data through the trained network making the learning of new tasks possible on mobile devices. In particular, we focus on few-shot image classification systems, i.e. machine learning systems that can distinguish between numerous classes of objects depicted in digital images given only a few examples of each class of object to learn from. To accomplish this, we first develop ML-PIP, a general framework for Meta-Learning approximate Probabilistic Inference for Prediction. ML-PIP extends existing probabilistic interpretations of meta-learning to cover a broad class of methods. We then introduce Versa, an instance of the framework employing a fast, flexible and versatile amortization network that takes few-shot learning datasets as inputs, with arbitrary numbers of training examples, and outputs a distribution over task-specific parameters in a single forward pass of the network. We evaluate Versa on benchmark datasets, where at the time, the method achieved state-of-the-art results when compared to meta-learning approaches using similar training regimes and feature extractor capacity. Next, we build on Versa and add a second amortized network to adapt key parameters in the feature extractor to the current task. To accomplish this, we introduce CNAPs, a conditional neural process based approach to multi-task classification. We demonstrate that, at the time, CNAPs achieved state-of-the-art results on the challenging Meta-Dataset benchmark indicating high-quality transfer-learning. Timing experiments reveal that CNAPs is computationally efficient when adapting to an unseen task as it does not involve gradient back propagation computations. We show that trained models are immediately deployable to continual learning and active learning where they can outperform existing approaches that do not leverage transfer learning. Finally, we investigate the effects of different methods of batch normalization on meta-learning systems. Batch normalization has become an essential component of deep learning systems as it significantly accelerates the training of neural networks by allowing the use of higher learning rates and decreasing the sensitivity to network initialization. We show that the hierarchical nature of the meta-learning setting presents several challenges that can render conventional batch normalization ineffective. We evaluate a range of approaches to batch normalization for few-shot learning scenarios, and develop a novel approach that we call TaskNorm. Experiments demonstrate that the choice of batch normalization has a dramatic effect on both classification accuracy and training time for both gradient based- and gradient-free meta-learning approaches and that TaskNorm consistently improves performance

Meta-Learning Probabilistic Inference For Prediction

This paper introduces a new framework for data efficient and versatile learning. Specifically: 1) We develop ML-PIP, a general framework for Meta-Learning approximate Probabilistic Inference for Prediction. ML-PIP extends existing probabilistic interpretations of meta-learning to cover a broad class of methods. 2) We introduce VERSA, an instance of the framework employing a flexible and versatile amortization network that takes few-shot learning datasets as inputs, with arbitrary numbers of shots, and outputs a distribution over task-specific parameters in a single forward pass. VERSA substitutes optimization at test time with forward passes through inference networks, amortizing the cost of inference and relieving the need for second derivatives during training. 3) We evaluate VERSA on benchmark datasets where the method sets new state-of-the-art results, handles arbitrary numbers of shots, and for classification, arbitrary numbers of classes at train and test time. The power of the approach is then demonstrated through a challenging few-shot ShapeNet view reconstruction task

Fast and Flexible Multi-Task Classification Using Conditional Neural Adaptive Processes

The goal of this paper is to design image classification systems that, after an initial multi-task training phase, can automatically adapt to new tasks encountered at test time. We introduce a conditional neural process based approach to the multi-task classification setting for this purpose, and establish connections to the meta-learning and few-shot learning literature. The resulting approach, called CNAPs, comprises a classifier whose parameters are modulated by an adaptation network that takes the current task's dataset as input. We demonstrate that CNAPs achieves state-of-the-art results on the challenging Meta-Dataset benchmark indicating high-quality transfer-learning. We show that the approach is robust, avoiding both over-fitting in low-shot regimes and under-fitting in high-shot regimes. Timing experiments reveal that CNAPs is computationally efficient at test-time as it does not involve gradient based adaptation. Finally, we show that trained models are immediately deployable to continual learning and active learning where they can outperform existing approaches that do not leverage transfer learning

Gender differences in home care clients and admission to long-term care in Ontario, Canada: a population-based retrospective cohort study

BACKGROUND: Home care is integral to enabling older adults to delay or avoid long-term care (LTC) admission. To date, there is little population-based data about gender differences in home care users and their subsequent outcomes. Our objectives were to quantify differences between women and men who used home care in Ontario, Canada and to determine if there were subsequent differences in LTC admission. METHODS: This is a population-based retrospective cohort study. We identified all adults aged 76+ years living in Ontario and receiving home care on April 1, 2007 (baseline). Using the Resident Assessment Instrument – Home Care (RAI-HC) linked to other databases, we characterized the cohort by living condition, health and functioning, and identified all acute care and LTC use in the year following baseline. RESULTS: The cohort consisted of 51,201 women and 20,102 men. Women were older, more likely to live alone, and more likely to rely on a child or child-in-law for caregiver support. Men most frequently identified a spouse as caregiver and their caregivers reported distress twice as often as women’s caregivers. Men had higher rates of most chronic conditions and were more likely to experience impairment. Men were more likely to be admitted to hospital, to have longer stays in hospital, and to be admitted to LTC. CONCLUSIONS: Understanding who uses home care and why is critical to ensuring that these programs effectively reduce LTC use. We found that women outnumbered men but that men presented with higher levels of need. This detailed gender analysis highlights how needs differ between older women, men, and their respective caregivers

Perfusion

The value of perfusion is reasonably well established yet the field is still developing and the ultimate applications and potentialities of perfusion techniques arc as yet undefined. It is a subject on which, as I am sharply aware, my knowledge is very incomplete, and I will discuss, in the main, work being carried out in my department in Glasgow Royal Infirmary and the University of Glasgow

Linkage of whole genome sequencing with administrative health, and electronic medical record data for the study of autism spectrum disorder: Feasibility, Opportunities and Challenges

Introduction Autism Spectrum Disorder (ASD) is a neurodevelopmental disorder (NDD) that presents with a high degree of heterogeneity (e.g., co-occurrence of other NDDs and other co-morbid conditions), contributing to differential health system needs. Genetics are known to play an important role in ASD and may be associated with different disease trajectories. Objectives and Approach In this proof of principle project, our objective is to link >2,200 children with a confirmed diagnosis of a NDD from the Province of Ontario Neurodevelopmental (POND) Study to administrative health data and electronic medical record (EMR) data in order to identify subgroups of ASD with unique health system trajectories. POND includes detailed phenotype and whole genome sequencing (WGS) data. Identified subgroups will be characterized based on clinical phenotype and genetics. To meet this goal, consideration of WGS-specific privacy and data issues is needed to implement processes which are above and beyond traditional requirements for analyzing individual-level administrative health data. Results Linkage of WGS data with administrative health data is an emerging area of research. As such it has presented a number of initial challenges for our study of ASD. Privacy concerns surrounding the use of WGS data and rare-variant analysis are of particular importance. Practical issues required the need for analysts with expertise in administrative data, EMR data and genetic analyses, and specialized software and sufficient processing power to analyze WGS data. Transdisciplinary discussions of the scope and significance of research questions addressed through this linkage were crucial. The identification of genetic determinants of phenotypes and trajectories in ASD could support targeted early interventions; EMR linkage may inform algorithms to identify ASD in broader populations. These approaches could improve both patient outcome and family experience. Conclusion/Implications As the cost of genetic sequencing decreases, WGS data will become part of the routine clinical management of patients. Linkage of WGS, EMR and administrative data has tremendous potential that has largely not been realized; including population-level ASD research to improve our ability to predict long-term outcomes associated with ASD

A comprehensive profile of the sociodemographic, psychosocial and health characteristics of Ontario home care clients with dementia

Abstract Introduction: This study provides a comprehensive summary of the sociodemographic, psychosocial and health characteristics of a large population-based cohort of Ontario home care clients (aged 50 years and over) with dementia and examines the variation in these characteristics in those with co-existing neurological conditions