Workload-aware Automatic Parallelization for Multi-GPU DNN Training

Deep neural networks (DNNs) have emerged as successful solutions for variety of artificial intelligence applications, but their very large and deep models impose high computational requirements during training. Multi-GPU parallelization is a popular option to accelerate demanding computations in DNN training, but most state-of-the-art multi-GPU deep learning frameworks not only require users to have an in-depth understanding of the implementation of the frameworks themselves, but also apply parallelization in a straight-forward way without optimizing GPU utilization. In this work, we propose a workload-aware auto-parallelization framework (WAP) for DNN training, where the work is automatically distributed to multiple GPUs based on the workload characteristics. We evaluate WAP using TensorFlow with popular DNN benchmarks (AlexNet and VGG-16), and show competitive training throughput compared with the state-of-the-art frameworks, and also demonstrate that WAP automatically optimizes GPU assignment based on the workload's compute requirements, thereby improving energy efficiency.Comment: This paper is accepted in ICASSP201

Solving for Micro- and Macro- Scale Electrostatic Configurations Using the Robin Hood Algorithm

We present a novel technique by which highly-segmented electrostatic configurations can be solved. The Robin Hood method is a matrix-inversion algorithm optimized for solving high density boundary element method (BEM) problems. We illustrate the capabilities of this solver by studying two distinct geometry scales: (a) the electrostatic potential of a large volume beta-detector and (b) the field enhancement present at surface of electrode nano-structures. Geometries with elements numbering in the O(10^5) are easily modeled and solved without loss of accuracy. The technique has recently been expanded so as to include dielectrics and magnetic materials.Comment: 40 pages, 20 figure

The measurement of air supply volumes and velocities in cleanrooms

Air supply volumes and velocities in cleanrooms are monitored by airflow measuring hoods and anemometers but these measuring methods can be inaccurate if used incorrectly. It is demonstrated in this article that measuring hoods are accurate if the air supply passes evenly out of the hood, as occurs when the air volume is measured from a four-way diffuser or no air supply diffuser. However, when a swirl diffuser was investigated, the measuring hood gave readings more than 50% greater than the true volume. The reasons for the inaccuracy, and methods to correct it were established. Vane anemometers give inaccurate readings at the face of high-efficiency air supply filters, and it was found that the most accurate reading was found about 15 cm from the filter face. The number of readings required across the filter face to obtain an accurate average velocity was investigated, as was a scanning method using overlapping passes

Continuum variational and diffusion quantum Monte Carlo calculations

This topical review describes the methodology of continuum variational and diffusion quantum Monte Carlo calculations. These stochastic methods are based on many-body wave functions and are capable of achieving very high accuracy. The algorithms are intrinsically parallel and well-suited to petascale computers, and the computational cost scales as a polynomial of the number of particles. A guide to the systems and topics which have been investigated using these methods is given. The bulk of the article is devoted to an overview of the basic quantum Monte Carlo methods, the forms and optimisation of wave functions, performing calculations within periodic boundary conditions, using pseudopotentials, excited-state calculations, sources of calculational inaccuracy, and calculating energy differences and forces

Flux creep in type-II superconductors: self-organized criticality approach

We consider the current density distribution function of a flux creep regime in type-II superconductors by mapping the flux creep process to the dynamics of a model with a self-organized criticality. We use an extremal Robin Hood type model which evolves to Been's type critical state to treat magnetic flux penetration into a superconductor and derive an analog of the current-voltage characteristics in the flux creep region.Comment: 7 pages, 7 figure