40 research outputs found
Energy-efficient Amortized Inference with Cascaded Deep Classifiers
Deep neural networks have been remarkable successful in various AI tasks but
often cast high computation and energy cost for energy-constrained applications
such as mobile sensing. We address this problem by proposing a novel framework
that optimizes the prediction accuracy and energy cost simultaneously, thus
enabling effective cost-accuracy trade-off at test time. In our framework, each
data instance is pushed into a cascade of deep neural networks with increasing
sizes, and a selection module is used to sequentially determine when a
sufficiently accurate classifier can be used for this data instance. The
cascade of neural networks and the selection module are jointly trained in an
end-to-end fashion by the REINFORCE algorithm to optimize a trade-off between
the computational cost and the predictive accuracy. Our method is able to
simultaneously improve the accuracy and efficiency by learning to assign easy
instances to fast yet sufficiently accurate classifiers to save computation and
energy cost, while assigning harder instances to deeper and more powerful
classifiers to ensure satisfiable accuracy. With extensive experiments on
several image classification datasets using cascaded ResNet classifiers, we
demonstrate that our method outperforms the standard well-trained ResNets in
accuracy but only requires less than 20% and 50% FLOPs cost on the CIFAR-10/100
datasets and 66% on the ImageNet dataset, respectively
Learning Anytime Predictions in Neural Networks via Adaptive Loss Balancing
This work considers the trade-off between accuracy and test-time
computational cost of deep neural networks (DNNs) via \emph{anytime}
predictions from auxiliary predictions. Specifically, we optimize auxiliary
losses jointly in an \emph{adaptive} weighted sum, where the weights are
inversely proportional to average of each loss. Intuitively, this balances the
losses to have the same scale. We demonstrate theoretical considerations that
motivate this approach from multiple viewpoints, including connecting it to
optimizing the geometric mean of the expectation of each loss, an objective
that ignores the scale of losses. Experimentally, the adaptive weights induce
more competitive anytime predictions on multiple recognition data-sets and
models than non-adaptive approaches including weighing all losses equally. In
particular, anytime neural networks (ANNs) can achieve the same accuracy faster
using adaptive weights on a small network than using static constant weights on
a large one. For problems with high performance saturation, we also show a
sequence of exponentially deepening ANNscan achieve near-optimal anytime
results at any budget, at the cost of a const fraction of extra computation
Batching for Green AI -- An Exploratory Study on Inference
The batch size is an essential parameter to tune during the development of
new neural networks. Amongst other quality indicators, it has a large degree of
influence on the model's accuracy, generalisability, training times and
parallelisability. This fact is generally known and commonly studied. However,
during the application phase of a deep learning model, when the model is
utilised by an end-user for inference, we find that there is a disregard for
the potential benefits of introducing a batch size. In this study, we examine
the effect of input batching on the energy consumption and response times of
five fully-trained neural networks for computer vision that were considered
state-of-the-art at the time of their publication. The results suggest that
batching has a significant effect on both of these metrics. Furthermore, we
present a timeline of the energy efficiency and accuracy of neural networks
over the past decade. We find that in general, energy consumption rises at a
much steeper pace than accuracy and question the necessity of this evolution.
Additionally, we highlight one particular network, ShuffleNetV2(2018), that
achieved a competitive performance for its time while maintaining a much lower
energy consumption. Nevertheless, we highlight that the results are model
dependent.Comment: 8 pages, 4 figures, 1 table. Accepted at Euromicro Conference Series
on Software Engineering and Advanced Applications (SEAA) 202
Edge Video Analytics: A Survey on Applications, Systems and Enabling Techniques
Video, as a key driver in the global explosion of digital information, can
create tremendous benefits for human society. Governments and enterprises are
deploying innumerable cameras for a variety of applications, e.g., law
enforcement, emergency management, traffic control, and security surveillance,
all facilitated by video analytics (VA). This trend is spurred by the rapid
advancement of deep learning (DL), which enables more precise models for object
classification, detection, and tracking. Meanwhile, with the proliferation of
Internet-connected devices, massive amounts of data are generated daily,
overwhelming the cloud. Edge computing, an emerging paradigm that moves
workloads and services from the network core to the network edge, has been
widely recognized as a promising solution. The resulting new intersection, edge
video analytics (EVA), begins to attract widespread attention. Nevertheless,
only a few loosely-related surveys exist on this topic. The basic concepts of
EVA (e.g., definition, architectures) were not fully elucidated due to the
rapid development of this domain. To fill these gaps, we provide a
comprehensive survey of the recent efforts on EVA. In this paper, we first
review the fundamentals of edge computing, followed by an overview of VA. The
EVA system and its enabling techniques are discussed next. In addition, we
introduce prevalent frameworks and datasets to aid future researchers in the
development of EVA systems. Finally, we discuss existing challenges and foresee
future research directions. We believe this survey will help readers comprehend
the relationship between VA and edge computing, and spark new ideas on EVA.Comment: 31 pages, 13 figure
Recommended from our members
Automatic channel detection using deep learning
Picking 3D channel geobodies in seismic volumes is an important objective in seismic interpretation for hydrocarbon exploration. Manual detection of channel geobodies is a time-consuming and subjective process. The interpreter can calculate different seismic attributes such as coherence to aid for manual detection of channel geobodies in seismic volumes. However, these attributes still do not directly identify 3D channel geobodies.
Machine learning and deep learning are data-driven techniques that have been getting more attention recently in different fields, such as medical imaging and computer vision. With large volumes of available data in different types and a development of powerful computational resources, geophysics is a promising field for applying machine learning and deep learning. Many seismic interpretation steps are analogous to different problems in computer vision that have been solved successfully using deep learning. Channel detection in seismic volumes is analogous to segmentation problems for images. Applying deep learning to seismic interpretations, specifically to automatic channel detection in 3D seismic volumes, can make the process faster and the workflow less subjective. Decision-making based on interpretations is uncertain; so uncertainties in interpretation results are very important. Deep learning with different algorithms can also help interpreters quantify this uncertainty.Geological Science
Reasoning with Uncertainty in Deep Learning for Safer Medical Image Computing
Deep learning is now ubiquitous in the research field of medical image computing. As such technologies progress towards clinical translation, the question of safety becomes critical. Once deployed, machine learning systems unavoidably face situations where the correct decision or prediction is ambiguous. However, the current methods disproportionately rely on deterministic algorithms, lacking a mechanism to represent and manipulate uncertainty. In safety-critical applications such as medical imaging, reasoning under uncertainty is crucial for developing a reliable decision making system. Probabilistic machine learning provides a natural framework to quantify the degree of uncertainty over different variables of interest, be it the prediction, the model parameters and structures, or the underlying data (images and labels). Probability distributions are used to represent all the uncertain unobserved quantities in a model and how they relate to the data, and probability theory is used as a language to compute and manipulate these distributions. In this thesis, we explore probabilistic modelling as a framework to integrate uncertainty information into deep learning models, and demonstrate its utility in various high-dimensional medical imaging applications. In the process, we make several fundamental enhancements to current methods. We categorise our contributions into three groups according to the types of uncertainties being modelled: (i) predictive; (ii) structural and (iii) human uncertainty. Firstly, we discuss the importance of quantifying predictive uncertainty and understanding its sources for developing a risk-averse and transparent medical image enhancement application. We demonstrate how a measure of predictive uncertainty can be used as a proxy for the predictive accuracy in the absence of ground-truths. Furthermore, assuming the structure of the model is flexible enough for the task, we introduce a way to decompose the predictive uncertainty into its orthogonal sources i.e. aleatoric and parameter uncertainty. We show the potential utility of such decoupling in providing a quantitative “explanations” into the model performance. Secondly, we introduce our recent attempts at learning model structures directly from data. One work proposes a method based on variational inference to learn a posterior distribution over connectivity structures within a neural network architecture for multi-task learning, and share some preliminary results in the MR-only radiotherapy planning application. Another work explores how the training algorithm of decision trees could be extended to grow the architecture of a neural network to adapt to the given availability of data and the complexity of the task. Lastly, we develop methods to model the “measurement noise” (e.g., biases and skill levels) of human annotators, and integrate this information into the learning process of the neural network classifier. In particular, we show that explicitly modelling the uncertainty involved in the annotation process not only leads to an improvement in robustness to label noise, but also yields useful insights into the patterns of errors that characterise individual experts