Python Wrapper for Simulating Multi-Fidelity Optimization on HPO Benchmarks without Any Wait

Hyperparameter (HP) optimization of deep learning (DL) is essential for high performance. As DL often requires several hours to days for its training, HP optimization (HPO) of DL is often prohibitively expensive. This boosted the emergence of tabular or surrogate benchmarks, which enable querying the (predictive) performance of DL with a specific HP configuration in a fraction. However, since actual runtimes of a DL training are significantly different from query response times, in a naive implementation, simulators of an asynchronous HPO, e.g. multi-fidelity optimization, must wait for the actual runtimes at each iteration; otherwise, the evaluation order in the simulator does not match with the real experiment. To ease this issue, we develop a Python wrapper to force each worker to wait in order to match the evaluation order with the real experiment and describe the usage. Our implementation reduces the waiting time to 0.01 seconds and it is available at https://github.com/nabenabe0928/mfhpo-simulator/

c-TPE: Tree-structured Parzen Estimator with Inequality Constraints for Expensive Hyperparameter Optimization

Hyperparameter optimization (HPO) is crucial for strong performance of deep learning algorithms and real-world applications often impose some constraints, such as memory usage, or latency on top of the performance requirement. In this work, we propose constrained TPE (c-TPE), an extension of the widely-used versatile Bayesian optimization method, tree-structured Parzen estimator (TPE), to handle these constraints. Our proposed extension goes beyond a simple combination of an existing acquisition function and the original TPE, and instead includes modifications that address issues that cause poor performance. We thoroughly analyze these modifications both empirically and theoretically, providing insights into how they effectively overcome these challenges. In the experiments, we demonstrate that c-TPE exhibits the best average rank performance among existing methods with statistical significance on 81 expensive HPO settings.Comment: Accepted to IJCAI 202

Features of ice sheet flow in East Dronning Maud Land, East Antarctica

The Japanese Antarctic Research Expeditions(JAREs) have done glaciological studies on ice sheet dynamics and surface mass balance in East Dronning Maud Land, mainly around the Shirase Glacier drainage basin, during more than 30 years. The surface mass balance, obtained mainly by the snow stake method, was more than 250mm/a in the coastal region, less than 50mm/a in the inland region higher than 3500m in altitude, and about 100mm/a on average in the five drainage basins in East Dronning Maud Land. The ice flow velocity was observed around East Dronning Maud Land in three observation periods: on a route transversal to the Shirase Glacier flow in 1969 to 1974, along a route longitudinal to Shirase Glacier and a transversal route from Mizuho Station(70°42\u27S , 44°17\u27E , 2250m a.s.l.) to the Sr Rondane Mountains area in 1982 to 1987, and along a route from S16(69°02\u27S , 40°03\u27E , 554m a.s.l.) near the coast to Dome Fuji Station(77°19\u27S , 39°42\u27E , 3810m a.s.l.) in 1992 to 1995. Assuming steady ice flow, the balance velocity is calculated by integrating the surface mass balance in the upstream area from a specific point to the flow origin between adjacent stream lines. From the relation between balance velocity and basal shear stress, the basal sliding area was specified

Experimental results on the formation of hard compacted snow in Rikubetsu in northern Japan: A first step toward the construction of a compacted-snow runway on the Antarctic ice sheet

This paper describes the experimental methods and results on the formation of hard compacted snow in Rikubetsu in northern Japan during the winter of 1999. This basic research was the first step towards the construction of a compacted-snow runway on the Antarctic ice sheet. In Rikubetsu, we constructed three test fields(20m in length, 7m in width, and 0.4-1.0m in thickness) on compacted basal snow(approximately 0.05m in thickness). First, 0.1-0.35-m-thick layers of snow were deposited on the basal snow of the fields using a rotary snowplow. Next, the surface snow was smoothed using an excavator. Finally, the snow layers were compacted four times using a bulldozer. This entire process was repeated three to four times in order to construct 0.4-1.0-m-thick test fields. The ram hardness, snow density, and snow structure of these fields were investigated. A comparison with the criteria established by a U.S. scientist for a large aircraft-such as the C-130(Abele, 1990)-revealed that if snow in the form of three 0.2-0.25-m-thick layers is compacted four times by a bulldozer, it is sufficiently hard to serve as a runway at H68(69°11\u279″S, 41°03\u2734″E, 1204m a.s.l.) for a wheeled C-130. The Japanese Antarctic Research Expedition plans to conduct a feasibility study on the construction of the hard com-pacted-snow runway at this location