Efficient Benchmarking of Algorithm Configuration Procedures via Model-Based Surrogates

The optimization of algorithm (hyper-)parameters is crucial for achieving peak performance across a wide range of domains, ranging from deep neural networks to solvers for hard combinatorial problems. The resulting algorithm configuration (AC) problem has attracted much attention from the machine learning community. However, the proper evaluation of new AC procedures is hindered by two key hurdles. First, AC benchmarks are hard to set up. Second and even more significantly, they are computationally expensive: a single run of an AC procedure involves many costly runs of the target algorithm whose performance is to be optimized in a given AC benchmark scenario. One common workaround is to optimize cheap-to-evaluate artificial benchmark functions (e.g., Branin) instead of actual algorithms; however, these have different properties than realistic AC problems. Here, we propose an alternative benchmarking approach that is similarly cheap to evaluate but much closer to the original AC problem: replacing expensive benchmarks by surrogate benchmarks constructed from AC benchmarks. These surrogate benchmarks approximate the response surface corresponding to true target algorithm performance using a regression model, and the original and surrogate benchmark share the same (hyper-)parameter space. In our experiments, we construct and evaluate surrogate benchmarks for hyperparameter optimization as well as for AC problems that involve performance optimization of solvers for hard combinatorial problems, drawing training data from the runs of existing AC procedures. We show that our surrogate benchmarks capture overall important characteristics of the AC scenarios, such as high- and low-performing regions, from which they were derived, while being much easier to use and orders of magnitude cheaper to evaluate

A novel satellite mission concept for upper air water vapour, aerosol and cloud observations using integrated path differential absorption LiDAR limb sounding

We propose a new satellite mission to deliver high quality measurements of upper air water vapour. The concept centres around a LiDAR in limb sounding by occultation geometry, designed to operate as a very long path system for differential absorption measurements. We present a preliminary performance analysis with a system sized to send 75 mJ pulses at 25 Hz at four wavelengths close to 935 nm, to up to 5 microsatellites in a counter-rotating orbit, carrying retroreflectors characterized by a reflected beam divergence of roughly twice the emitted laser beam divergence of 15 µrad. This provides water vapour profiles with a vertical sampling of 110 m; preliminary calculations suggest that the system could detect concentrations of less than 5 ppm. A secondary payload of a fairly conventional medium resolution multispectral radiometer allows wide-swath cloud and aerosol imaging. The total weight and power of the system are estimated at 3 tons and 2,700 W respectively. This novel concept presents significant challenges, including the performance of the lasers in space, the tracking between the main spacecraft and the retroreflectors, the refractive effects of turbulence, and the design of the telescopes to achieve a high signal-to-noise ratio for the high precision measurements. The mission concept was conceived at the Alpbach Summer School 2010

CUSTARD (Cranfield University Space Technology Advanced Research Demonstrator) - A Micro-System Technology Demonstrator Nanosatellite. Summary of the Group Design Project MSc in Astronautics and Space Engineering. 1999-2000, Cranfield University

CUSTARD (Cranfield University Space Technology And Research Demonstrator) was the group design project for students of the MSc in Astronautics and Space Engineering for the Academic Year 1999/2000 at Cranfield University. The project involved the initial design of a nanosatellite to be used as a technology demonstrator for microsystem technology (MST) in space. The students worked together as one group (organised into several subgroups, e.g. system, mechanical), with each student responsible for a set of work packages. The nanosatellite designed had a mass of 4 kg, lifetime of 3 months in low Earth orbit, coarse 3-axis attitude control (no orbit control), and was capable of carrying up to 1 kg of payload. The electrical power available was 18 W (peak). Assuming a single X-band ground station at RAL (UK), a data rate of up to 1 M bit s-1 for about 3000 s per day is possible. The payloads proposed are a microgravity laboratory and a formation flying experiment. The report summarises the results of the project and includes executive summaries from all team members. Further information and summaries of the full reports are available from the College of Aeronautics, Cranfield University