Hyperparameter Importance Across Datasets

With the advent of automated machine learning, automated hyperparameter optimization methods are by now routinely used in data mining. However, this progress is not yet matched by equal progress on automatic analyses that yield information beyond performance-optimizing hyperparameter settings. In this work, we aim to answer the following two questions: Given an algorithm, what are generally its most important hyperparameters, and what are typically good values for these? We present methodology and a framework to answer these questions based on meta-learning across many datasets. We apply this methodology using the experimental meta-data available on OpenML to determine the most important hyperparameters of support vector machines, random forests and Adaboost, and to infer priors for all their hyperparameters. The results, obtained fully automatically, provide a quantitative basis to focus efforts in both manual algorithm design and in automated hyperparameter optimization. The conducted experiments confirm that the hyperparameters selected by the proposed method are indeed the most important ones and that the obtained priors also lead to statistically significant improvements in hyperparameter optimization.Comment: \c{opyright} 2018. Copyright is held by the owner/author(s). Publication rights licensed to ACM. This is the author's version of the work. It is posted here for your personal use, not for redistribution. The definitive Version of Record was published in Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Minin

Prediction of Radiation-Induced Parotid Gland-Related Xerostomia in Patients With Head and Neck Cancer:Regeneration-Weighted Dose

Purpose: Despite improvements to treatment, patients with head and neck cancer (HNC) still experience radiation-induced xerostomia due to salivary gland damage. The stem cells of the parotid gland (PG), concentrated in the gland's main ducts (stem cell rich [SCR] region), play a critical role in the PG's response to radiation. Treatment optimization requires a dose metric that properly accounts for the relative contributions of dose to this SCR region and the PG's remainder (non-SCR region) to the risk of xerostomia in normal tissue complication probability (NTCP) models for xerostomia.Materials and methods: Treatment and toxicity data of 1013 prospectively followed patients with HNC treated with definitive radiation therapy (RT) were used. The regeneration-weighted dose, enabling accounting for the hypothesized different effects of dose to the SCR and non-SCR region on the risk of xerostomia, was defined as Dreg PG = Dmean SCR region + r × Dmean non-SCR region, where Dreg is the regeneration-weighted dose, Dmean is the mean dose, and r is the weighting factor. Considering the different volumes of these regions, r > 3.6 in Dreg PG demonstrates an enhanced effect of the SCR region. The most predictive value of r was estimated in 102 patients of a previously published trial testing stem cell sparing RT. For each endpoint, Dreg PG, dose to other organs, and clinical factors were used to develop NTCP models using multivariable logistic regression analysis in 663 patients. The models were validated in 350 patients.Results: Dose to the contralateral PG was associated with daytime, eating-related, and physician-rated grade ≥2 xerostomia. Consequently, r was estimated and found to be smaller than 3.6 for most PG function-related endpoints. Therefore, the contribution of Dmean SCR region to the risk of xerostomia was larger than predicted by Dmean PG. Other frequently selected predictors were pretreatment xerostomia and Dmean oral cavity. The validation showed good discrimination and calibration.Conclusions: Tools for clinical implementation of stem cell sparing RT were developed: regeneration-weighted dose to the parotid gland that accounted for regional differences in radiosensitivity within the gland and NTCP models that included this new dose metric and other prognostic factors