8 research outputs found
Scalable Reinforcement-Learning-Based Neural Architecture Search for Cancer Deep Learning Research
Cancer is a complex disease, the understanding and treatment of which are
being aided through increases in the volume of collected data and in the scale
of deployed computing power. Consequently, there is a growing need for the
development of data-driven and, in particular, deep learning methods for
various tasks such as cancer diagnosis, detection, prognosis, and prediction.
Despite recent successes, however, designing high-performing deep learning
models for nonimage and nontext cancer data is a time-consuming,
trial-and-error, manual task that requires both cancer domain and deep learning
expertise. To that end, we develop a reinforcement-learning-based neural
architecture search to automate deep-learning-based predictive model
development for a class of representative cancer data. We develop custom
building blocks that allow domain experts to incorporate the
cancer-data-specific characteristics. We show that our approach discovers deep
neural network architectures that have significantly fewer trainable
parameters, shorter training time, and accuracy similar to or higher than those
of manually designed architectures. We study and demonstrate the scalability of
our approach on up to 1,024 Intel Knights Landing nodes of the Theta
supercomputer at the Argonne Leadership Computing Facility.Comment: SC '19: IEEE/ACM International Conference on High Performance
Computing, Networking, Storage and Analysis, November 17--22, 2019, Denver,
C
Recommended from our members
Informative Hyper-parameter Optimization and Selection
Hyper-parameter optimization methods allow efficient and robust hyperparameter search-ing without the need to hand-select each value and combination. Although hyper-parameter tuners, such as BOHB, Hyperopt, and SMAC have been investigated by researchers in terms of performance, there has yet to be an in-depth analysis of the values each tuner selected over alliterations. We propose a thorough aggregation of data in terms of the efficiency of the search values selected by each tuner over 59 datasets and ten popular ML algorithms from Scikit-learn. From this extensive data accumulated, we observe and advise which tuners show better results for particular datasets, through its meta-data, and algorithms. Through this research, we have also developed a simple plug-in for BOHB, Hyperopt, and SMAC into DARPA’s Data-driven discovery(D3M) Auto-ML systems for smooth implementation of various tuners. This is advantageous as the desired hyper-parameter tuner may change depending on the pipeline search method in anAuto-ML system, particularly when compared with Auto-ML systems that only utilize one search method. Our results show that for Auto-ML systems, the Hyperopt tuner will give more desirable results in a fewer amount of iterations due to the significant exploration component, and BOHB performs the best generally over a large number of datasets and algorithms owing to strategic budgeting