Modeling Scalability of Distributed Machine Learning

Present day machine learning is computationally intensive and processes large amounts of data. It is implemented in a distributed fashion in order to address these scalability issues. The work is parallelized across a number of computing nodes. It is usually hard to estimate in advance how many nodes to use for a particular workload. We propose a simple framework for estimating the scalability of distributed machine learning algorithms. We measure the scalability by means of the speedup an algorithm achieves with more nodes. We propose time complexity models for gradient descent and graphical model inference. We validate our models with experiments on deep learning training and belief propagation. This framework was used to study the scalability of machine learning algorithms in Apache Spark.Comment: 6 pages, 4 figures, appears at ICDE 201

Competitive Policy Optimization

A core challenge in policy optimization in competitive Markov decision processes is the design of efficient optimization methods with desirable convergence and stability properties. To tackle this, we propose competitive policy optimization (CoPO), a novel policy gradient approach that exploits the game-theoretic nature of competitive games to derive policy updates. Motivated by the competitive gradient optimization method, we derive a bilinear approximation of the game objective. In contrast, off-the-shelf policy gradient methods utilize only linear approximations, and hence do not capture interactions among the players. We instantiate CoPO in two ways:(i) competitive policy gradient, and (ii) trust-region competitive policy optimization. We theoretically study these methods, and empirically investigate their behavior on a set of comprehensive, yet challenging, competitive games. We observe that they provide stable optimization, convergence to sophisticated strategies, and higher scores when played against baseline policy gradient methods.Comment: 11 pages main paper, 6 pages references, and 31 pages appendix. 14 figure

Competitive Policy Optimization

A core challenge in policy optimization in competitive Markov decision processes is the design of efficient optimization methods with desirable convergence and stability properties. To tackle this, we propose competitive policy optimization (CoPO), a novel policy gradient approach that exploits the game-theoretic nature of competitive games to derive policy updates. Motivated by the competitive gradient optimization method, we derive a bilinear approximation of the game objective. In contrast, off-the-shelf policy gradient methods utilize only linear approximations, and hence do not capture interactions among the players. We instantiate CoPO in two ways:(i) competitive policy gradient, and (ii) trust-region competitive policy optimization. We theoretically study these methods, and empirically investigate their behavior on a set of comprehensive, yet challenging, competitive games. We observe that they provide stable optimization, convergence to sophisticated strategies, and higher scores when played against baseline policy gradient methods

From microlattices to 3d microprinting of multiphase micro-components: Resolution limits and mechanical properties under extreme conditions

Two-photon lithography (TPL) enables the fabrication of metamaterials such as polymer micro-lattices. They are designed to achieve their envisioned mechanical properties through stretching and bending of individual trusses. Several novel approaches are developed here to a) directly print metal microlattices, b) fabricate multiphase composite microlattices and c) shrink the truss diameter below the diffraction limit of light, all with the ultimate goal to enable fabrication of a full dense material with microprinted 3D architecture of different phases. Copper microlattices and micropillars with truss diameters in the few micron range were printed directly via fluid AFM based local electroplating [1]. It was identified that microcrystalline copper micropillars deform in a singleshear like manner exhibiting a weak strain rate dependence at all strain rates. Ultrafine grained (UFG) copper micropillars, however, deform homogenously via barreling and show strong rate-dependence and small activation volumes at strain rates up to ∼ 0.1 s−1, suggesting dislocation nucleation as the deformation mechanism. At higher strain rates, yield stress saturates remarkably, resulting in a decrease of strain rate sensitivity implying a transition in deformation mechanism to collective dislocation nucleation. Finally, the copper microlattices are shown to increase in strength if conformally coated with Nickel with thicknesses in the several 100nm range. Please click Download on the upper right corner to see the full abstract

State-of-the-art in string similarity search and join

String similarity search and its variants are fundamental problems with many applications in areas such as data integration, data quality, computational linguistics, or bioinformatics. A plethora of methods have been developed over the last decades. Obtaining an overview of the state-of-the-art in this field is difficult, as results are published in various domains without much cross-talk, papers use different data sets and often study subtle variations of the core problems, and the sheer number of proposed methods exceeds the capacity of a single research group. In this paper, we report on the results of the probably largest benchmark ever performed in this field. To overcome the resource bottleneck, we organized the benchmark as an international competition, a workshop at EDBT/ICDT 2013. Various teams from different fields and from all over the world developed or tuned programs for two crisply defined problems. All algorithms were evaluated by an external group on two machines. Altogether, we compared 14 different programs on two string matching problems (k-approximate search and k-approximate join) using data sets of increasing sizes and with different characteristics from two different domains. We compare programs primarily by wall clock time, but also provide results on memory usage, indexing time, batch query effects and scalability in terms of CPU cores. Results were averaged over several runs and confirmed on a second, different hardware platform. A particularly interesting observation is that disciplines can and should learn more from each other, with the three best teams rooting in computational linguistics, databases, and bioinformatics, respectively

Temperature-dependent dynamic plasticity of micro-scale fused silica

Please click Additional Files below to see the full abstract