Impatient DNNs - Deep Neural Networks with Dynamic Time Budgets

We propose Impatient Deep Neural Networks (DNNs) which deal with dynamic time budgets during application. They allow for individual budgets given a priori for each test example and for anytime prediction, i.e., a possible interruption at multiple stages during inference while still providing output estimates. Our approach can therefore tackle the computational costs and energy demands of DNNs in an adaptive manner, a property essential for real-time applications. Our Impatient DNNs are based on a new general framework of learning dynamic budget predictors using risk minimization, which can be applied to current DNN architectures by adding early prediction and additional loss layers. A key aspect of our method is that all of the intermediate predictors are learned jointly. In experiments, we evaluate our approach for different budget distributions, architectures, and datasets. Our results show a significant gain in expected accuracy compared to common baselines.Comment: British Machine Vision Conference (BMVC) 201

SHADHO: Massively Scalable Hardware-Aware Distributed Hyperparameter Optimization

Computer vision is experiencing an AI renaissance, in which machine learning models are expediting important breakthroughs in academic research and commercial applications. Effectively training these models, however, is not trivial due in part to hyperparameters: user-configured values that control a model's ability to learn from data. Existing hyperparameter optimization methods are highly parallel but make no effort to balance the search across heterogeneous hardware or to prioritize searching high-impact spaces. In this paper, we introduce a framework for massively Scalable Hardware-Aware Distributed Hyperparameter Optimization (SHADHO). Our framework calculates the relative complexity of each search space and monitors performance on the learning task over all trials. These metrics are then used as heuristics to assign hyperparameters to distributed workers based on their hardware. We first demonstrate that our framework achieves double the throughput of a standard distributed hyperparameter optimization framework by optimizing SVM for MNIST using 150 distributed workers. We then conduct model search with SHADHO over the course of one week using 74 GPUs across two compute clusters to optimize U-Net for a cell segmentation task, discovering 515 models that achieve a lower validation loss than standard U-Net.Comment: 10 pages, 6 figure

A Hybrid Approach Support Vector Machine (SVM) – Neuro Fuzzy for Fast Data Classification

In recent decade, support vector machine (SVM) was a machine learning method that widely used in several application domains. It was due to SVM has a good performance for solving data classification problems, particularly in non-linear case. Nevertheless, several studies indicated that SVM still has some inadequacies, especially the high time complexity in testing phase that is caused by increasing the number of support vector for high dimensional data. To address this problem, we propose a hybrid approach SVM – Neuro Fuzzy (SVMNF), which neuro fuzzy here is used to avoid influence of support vector in testing phase of SVM. Moreover, our approach is also equipped with a feature selection that can reduce data attributes in testing phase, so that it can improve the effectiveness of time computation. Based on our evaluation in real benchmark datasets, our approach outperformed SVM in testing phase for solving data classification problems without significantly affecting the accuracy of SVM

Autoencoding the Retrieval Relevance of Medical Images

Content-based image retrieval (CBIR) of medical images is a crucial task that can contribute to a more reliable diagnosis if applied to big data. Recent advances in feature extraction and classification have enormously improved CBIR results for digital images. However, considering the increasing accessibility of big data in medical imaging, we are still in need of reducing both memory requirements and computational expenses of image retrieval systems. This work proposes to exclude the features of image blocks that exhibit a low encoding error when learned by a $n/p/n$ autoencoder ($p\!<\!n$). We examine the histogram of autoendcoding errors of image blocks for each image class to facilitate the decision which image regions, or roughly what percentage of an image perhaps, shall be declared relevant for the retrieval task. This leads to reduction of feature dimensionality and speeds up the retrieval process. To validate the proposed scheme, we employ local binary patterns (LBP) and support vector machines (SVM) which are both well-established approaches in CBIR research community. As well, we use IRMA dataset with 14,410 x-ray images as test data. The results show that the dimensionality of annotated feature vectors can be reduced by up to 50% resulting in speedups greater than 27% at expense of less than 1% decrease in the accuracy of retrieval when validating the precision and recall of the top 20 hits.Comment: To appear in proceedings of The 5th International Conference on Image Processing Theory, Tools and Applications (IPTA'15), Nov 10-13, 2015, Orleans, Franc

A Hybrid Approach Support Vector Machine (SVM) – Neuro Fuzzy For Fast Data Classification

In recent decade, support vector machine (SVM) was a machine learning method that widely used in several application domains. It was due to SVM has a good performance for solving data classification problems, particularly in non-linear case. Nevertheless, several studies indicated that SVM still has some inadequacies, especially the high time complexity in testing phase that is caused by increasing the number of support vector for high dimensional data. To address this problem, we propose a hybrid approach SVM – Neuro Fuzzy (SVMNF), which neuro fuzzy here is used to avoid influence of support vector in testing phase of SVM. Moreover, our approach is also equipped with a feature selection that can reduce data attributes in testing phase, so that it can improve the effectiveness of time computation. Based on our evaluation in real benchmark datasets, our approach outperformed SVM in testing phase for solving data classification problems without significantly affecting the accuracy of SVM

Predicting the Law Area and Decisions of French Supreme Court Cases

In this paper, we investigate the application of text classification methods to predict the law area and the decision of cases judged by the French Supreme Court. We also investigate the influence of the time period in which a ruling was made over the textual form of the case description and the extent to which it is necessary to mask the judge's motivation for a ruling to emulate a real-world test scenario. We report results of 96% f1 score in predicting a case ruling, 90% f1 score in predicting the law area of a case, and 75.9% f1 score in estimating the time span when a ruling has been issued using a linear Support Vector Machine (SVM) classifier trained on lexical features.Comment: RANLP 201