SHADHO: Massively Scalable Hardware-Aware Distributed Hyperparameter Optimization

Computer vision is experiencing an AI renaissance, in which machine learning models are expediting important breakthroughs in academic research and commercial applications. Effectively training these models, however, is not trivial due in part to hyperparameters: user-configured values that control a model's ability to learn from data. Existing hyperparameter optimization methods are highly parallel but make no effort to balance the search across heterogeneous hardware or to prioritize searching high-impact spaces. In this paper, we introduce a framework for massively Scalable Hardware-Aware Distributed Hyperparameter Optimization (SHADHO). Our framework calculates the relative complexity of each search space and monitors performance on the learning task over all trials. These metrics are then used as heuristics to assign hyperparameters to distributed workers based on their hardware. We first demonstrate that our framework achieves double the throughput of a standard distributed hyperparameter optimization framework by optimizing SVM for MNIST using 150 distributed workers. We then conduct model search with SHADHO over the course of one week using 74 GPUs across two compute clusters to optimize U-Net for a cell segmentation task, discovering 515 models that achieve a lower validation loss than standard U-Net.Comment: 10 pages, 6 figure

Industrial Policy in the Field of Informatics in Brazil

This article first presents a brief overview of Brazilian industrial development. This overview provides a basis for understanding how the Brazilian Government\u27s informatics policy differs from past Brazilian industrial models. The article then describes the Brazilian Government\u27s policy in the field of informatics. It concludes that a policy which is less protectionist than the government\u27s current program would, through allowing greater foreign participation in the market, better encourage the development of Brazilian informatic companies

Industrial Policy in the Field of Informatics in Brazil

This article first presents a brief overview of Brazilian industrial development. This overview provides a basis for understanding how the Brazilian Government\u27s informatics policy differs from past Brazilian industrial models. The article then describes the Brazilian Government\u27s policy in the field of informatics. It concludes that a policy which is less protectionist than the government\u27s current program would, through allowing greater foreign participation in the market, better encourage the development of Brazilian informatic companies

The Use Of Probabilistic Multiphase Flow Equations In The Study Of The Hydrodynamics And Heat Transfer In Gas-solids Suspensions

This thesis represents the initial step in a more rigorous approach to the study of the hydrodynamic and heat transfer behaviour of gas-solids suspension flows. General Probabilistic Multiphase flow equa- tions were developed and used as the basis for analyzing pressure drops and pipe wall to suspension heat transfer coefficients meas- ured in dilute, vertical, fully developed suspension flows. Hydrody- namic experiments were carried out with sand ((\u27)d(,p) = 172, 249 (mu)m) and glass bead ((\u27)d(,p) = 65 (mu)m) suspensions in a 12 m high, 20 mm ID transport line installation designed to allow pressure drops, solids and gas flowrates and mean solids concentrations to be measured. Heat transfer experiments were carried out with 249 (mu)m sand suspen- sions in the same installation with a modified test section. A thinned-walled stainless steel pipe was heated electrically in order to provide a constant heat flux from the test section wall to the suspension flowing inside.;The general equation analysis showed pressure drops in dilute vertical suspension flows to be dependent on particle-wall coeffi- cients of restitution and the local solids concentration and velocity in the vicinity of the pipe wall. The lack of information available on solids concentration and velocity profiles prevented the develop- ment of a final pressure drop expression in terms of mean values, averaged across the pipe cross-section. The hydrodynamic results did nevertheless suggest that reduced concentration and velocity profiles were independent of the mean solids concentration in dilute flows. This conclusion and an asymptotic analysis of the energy equations allowed the development of a general expression for the variation of fully developed heat transfer coefficients as a function of loading ratio. Heat transfer coefficient ratios were predicted to either increase, decrease, or pass through a minimum and then increase depending on the relative magnitudes of three coefficients which are fully determined by the hydrodynamics of the system. The general predicted variation was confirmed experimentally and numerical values of the hydrodynamic coefficients were determined