Online Learning of a Memory for Learning Rates

The promise of learning to learn for robotics rests on the hope that by extracting some information about the learning process itself we can speed up subsequent similar learning tasks. Here, we introduce a computationally efficient online meta-learning algorithm that builds and optimizes a memory model of the optimal learning rate landscape from previously observed gradient behaviors. While performing task specific optimization, this memory of learning rates predicts how to scale currently observed gradients. After applying the gradient scaling our meta-learner updates its internal memory based on the observed effect its prediction had. Our meta-learner can be combined with any gradient-based optimizer, learns on the fly and can be transferred to new optimization tasks. In our evaluations we show that our meta-learning algorithm speeds up learning of MNIST classification and a variety of learning control tasks, either in batch or online learning settings.Comment: accepted to ICRA 2018, code available: https://github.com/fmeier/online-meta-learning ; video pitch available: https://youtu.be/9PzQ25FPPO

An Efficient Algorithm for Automatic Structure Optimization in X-ray Standing-Wave Experiments

X-ray standing-wave photoemission experiments involving multilayered samples are emerging as unique probes of the buried interfaces that are ubiquitous in current device and materials research. Such data require for their analysis a structure optimization process comparing experiment to theory that is not straightforward. In this work, we present a new computer program for optimizing the analysis of standing-wave data, called SWOPT, that automates this trial-and-error optimization process. The program includes an algorithm that has been developed for computationally expensive problems: so-called black-box simulation optimizations. It also includes a more efficient version of the Yang X-ray Optics Program (YXRO) [Yang, S.-H., Gray, A.X., Kaiser, A.M., Mun, B.S., Sell, B.C., Kortright, J.B., Fadley, C.S., J. Appl. Phys. 113, 1 (2013)] which is about an order of magnitude faster than the original version. Human interaction is not required during optimization. We tested our optimization algorithm on real and hypothetical problems and show that it finds better solutions significantly faster than a random search approach. The total optimization time ranges, depending on the sample structure, from minutes to a few hours on a modern laptop computer, and can be up to 100x faster than a corresponding manual optimization. These speeds make the SWOPT program a valuable tool for realtime analyses of data during synchrotron experiments

To Index or Not to Index: Optimizing Exact Maximum Inner Product Search

Exact Maximum Inner Product Search (MIPS) is an important task that is widely pertinent to recommender systems and high-dimensional similarity search. The brute-force approach to solving exact MIPS is computationally expensive, thus spurring recent development of novel indexes and pruning techniques for this task. In this paper, we show that a hardware-efficient brute-force approach, blocked matrix multiply (BMM), can outperform the state-of-the-art MIPS solvers by over an order of magnitude, for some -- but not all -- inputs. In this paper, we also present a novel MIPS solution, MAXIMUS, that takes advantage of hardware efficiency and pruning of the search space. Like BMM, MAXIMUS is faster than other solvers by up to an order of magnitude, but again only for some inputs. Since no single solution offers the best runtime performance for all inputs, we introduce a new data-dependent optimizer, OPTIMUS, that selects online with minimal overhead the best MIPS solver for a given input. Together, OPTIMUS and MAXIMUS outperform state-of-the-art MIPS solvers by 3.2$\times$ on average, and up to 10.9$\times$, on widely studied MIPS datasets.Comment: 12 pages, 8 figures, 2 table

Distributed multilevel optimization for complex structures

Optimization problems concerning complex structures with many design variables may entail an unacceptable computational cost. This problem can be reduced considerably with a multilevel approach: A structure consisting of several components is optimized as a whole (global) as well as on the component level. In this paper, an optimization method is discussed with applications in the assessment of the impact of new design considerations in the development of a structure. A strategy based on fully stressed design is applied for optimization problems in linear statics. A global model is used to calculate the interactions (e.g., loads) for each of the components. These components are then optimized using the prescribed interactions, followed by a new global calculation to update the interactions. Mixed discrete and continuous design variables as well as different design configurations are possible. An application of this strategy is presented in the form of the full optimization of a vertical tail plane center box of a generic large passenger aircraft. In linear dynamics, the parametrization of the component interactions is problematic due to the frequency dependence. Hence, a modified method is presented in which the speed of component mode synthesis is used to avoid this parametrization. This method is applied to a simple test case that originates from noise control. \u