4 research outputs found
Deep supervised learning using local errors
Error backpropagation is a highly effective mechanism for learning
high-quality hierarchical features in deep networks. Updating the features or
weights in one layer, however, requires waiting for the propagation of error
signals from higher layers. Learning using delayed and non-local errors makes
it hard to reconcile backpropagation with the learning mechanisms observed in
biological neural networks as it requires the neurons to maintain a memory of
the input long enough until the higher-layer errors arrive. In this paper, we
propose an alternative learning mechanism where errors are generated locally in
each layer using fixed, random auxiliary classifiers. Lower layers could thus
be trained independently of higher layers and training could either proceed
layer by layer, or simultaneously in all layers using local error information.
We address biological plausibility concerns such as weight symmetry
requirements and show that the proposed learning mechanism based on fixed,
broad, and random tuning of each neuron to the classification categories
outperforms the biologically-motivated feedback alignment learning technique on
the MNIST, CIFAR10, and SVHN datasets, approaching the performance of standard
backpropagation. Our approach highlights a potential biological mechanism for
the supervised, or task-dependent, learning of feature hierarchies. In
addition, we show that it is well suited for learning deep networks in custom
hardware where it can drastically reduce memory traffic and data communication
overheads
Recommended from our members
Human-Centered Machine Learning for Healthcare: Examples in Neurology and Pulmonology
Machine learning (ML) in healthcare has enabled the automatic detection of diseases from medical images or sensors with high accuracy, often outperforming domain experts. Unfortunately, there is a large variance between how such diagnostic aids perform in research settings and in the real-world. This is due to challenges unique to healthcare that, if unaddressed, limit the usefulness of ML-based software when deployed in hospitals. For example, in human subjects research and clinical trials, subject recruitment and data acquisition are involved processes for both patients and healthcare providers; there are several regulatory and cybersecurity requirements to satisfy to ensure that patient care is not compromised in the pursuit of big data. Without abundant data to train ML models, it can be difficult to elicit good performance that also generalizes well on unseen data in clinical practice. Moreover, ML tools in hospitals cannot function independently but must integrate with existing workflows. There are ethical considerations with respect to how these tools influence the decision making of clinicians and whether they encourage an over-reliance on predictions.In this dissertation, we discuss these and other concerns in the context of three focus areas: stroke, respiratory disease, and Parkinson’s disease. We present machine and deep learning pipelines for weakness detection in stroke patients from video, respiratory disease classification from audio of coughs, and gait assessment in Parkinson’s disease with body sensors. In our efforts, we were cognizant of the technical and human challenges of healthcare. We developed models that not only performed well but also could be trained and rigorously evaluated in a data-conscious way. Our ML solutions ranged from simple leave-one-out approaches to data augmentation with generative adversarial nets. Lastly, we show how ML can more effectively aid medical diagnosis when paired with human-centered design. We describe a clinical decision support system for acute stroke, focusing on the development of an intuitive user interface that balances neurologist assessments with the symptom predictions of our models. This dissertation details novel, human-centered ML techniques for disease diagnosis in neurology and pulmonology, highlighting several lessons learned to benefit the field of machine learning in healthcare at large
Recommended from our members
Deep Supervised Learning Using Local Errors.
Error backpropagation is a highly effective mechanism for learning high-quality hierarchical features in deep networks. Updating the features or weights in one layer, however, requires waiting for the propagation of error signals from higher layers. Learning using delayed and non-local errors makes it hard to reconcile backpropagation with the learning mechanisms observed in biological neural networks as it requires the neurons to maintain a memory of the input long enough until the higher-layer errors arrive. In this paper, we propose an alternative learning mechanism where errors are generated locally in each layer using fixed, random auxiliary classifiers. Lower layers could thus be trained independently of higher layers and training could either proceed layer by layer, or simultaneously in all layers using local error information. We address biological plausibility concerns such as weight symmetry requirements and show that the proposed learning mechanism based on fixed, broad, and random tuning of each neuron to the classification categories outperforms the biologically-motivated feedback alignment learning technique on the CIFAR10 dataset, approaching the performance of standard backpropagation. Our approach highlights a potential biological mechanism for the supervised, or task-dependent, learning of feature hierarchies. In addition, we show that it is well suited for learning deep networks in custom hardware where it can drastically reduce memory traffic and data communication overheads. Code used to run all learning experiments is available under https://gitlab.com/hesham-mostafa/learning-using-local-erros.git