Accelerating Neural Network Training with Distributed Asynchronous and Selective Optimization (DASO)

With increasing data and model complexities, the time required to train neural networks has become prohibitively large. To address the exponential rise in training time, users are turning to data parallel neural networks (DPNN) and large-scale distributed resources on computer clusters. Current DPNN approaches implement the network parameter updates by synchronizing and averaging gradients across all processes with blocking communication operations after each forward-backward pass. This synchronization is the central algorithmic bottleneck. We introduce the Distributed Asynchronous and Selective Optimization (DASO) method, which leverages multi-GPU compute node architectures to accelerate network training while maintaining accuracy. DASO uses a hierarchical and asynchronous communication scheme comprised of node-local and global networks while adjusting the global synchronization rate during the learning process. We show that DASO yields a reduction in training time of up to 34% on classical and state-of-the-art networks, as compared to current optimized data parallel training methods

Deep Learning as a Parton Shower

We make the connection between certain deep learning architectures and the renormalisation group explicit in the context of QCD by using a deep learning network to construct a toy parton shower model. The model aims to describe proton-proton collisions at the Large Hadron Collider. A convolutional autoencoder learns a set of kernels that efficiently encode the behaviour of fully showered QCD collision events. The network is structured recursively so as to ensure self-similarity, and the number of trained network parameters is low. Randomness is introduced via a novel custom masking layer, which also preserves existing parton splittings by using layer-skipping connections. By applying a shower merging procedure, the network can be evaluated on unshowered events produced by a matrix element calculation. The trained network behaves as a parton shower that qualitatively reproduces jet-based observables.Comment: 26 pages, 13 figure

Moving Arts Leadership Forward: A Changing Landscape

Since 2009, the William and Flora Hewlett Foundation's Performing Arts Program has been making grants to help emerging arts leaders develop satisfying and successful careers through the Next Generation Arts Leadership Initiative. The first phase of that work, which ended in 2015, was funded in partnership with the James Irvine Foundation. It focused on training and retaining emerging arts leaders -- defined as eighteen to thirty-five-year-olds with ten years or less of arts experience -- in anticipation of a widely predicted wave of retirements. The Initiative made grants totaling $1.9 million to five leadership networks across California, and to statewide regranting programs, managed by the Center for Cultural Innovation to support professional development for individuals and innovative organizational practices. While an assessment conducted in 2011 showed that the Initiative was successful in achieving its early goals of building infrastructure and opportunities for younger arts leaders, the Performing Arts Program and our partners continued to grapple with a few persistent questions: what were we preparing up-and-coming leaders to do? To what degree did we aim to sustain the field as it exists or spur its transformation? Were we adequately preparing leaders for the challenges to come? To help answer these questions, in late 2014 we commissioned Michael Courville of Open Mind Consulting to reassess the arts leadership landscape in California and explore opportunities for future investments in arts leadership.The research was conducted in collaboration with a cross-section of local, regional, and national arts leaders, and with the Initiative's partners. It reveals that the arts landscape is in a state of flux and that there is a timely opportunity to reimagine how the nonprofit arts field defines and practices leadership

Strengthening Integrated Primary Health Care in Sofala, Mozambique

Background: Large increases in health sector investment and policies favoring upgrading and expanding the public sector health network have prioritized maternal and child health in Mozambique and, over the past decade, Mozambique has achieved substantial improvements in maternal and child health indicators. Over this same period, the government of Mozambique has continued to decentralize the management of public sector resources to the district level, including in the health sector, with the aim of bringing decision-making and resources closer to service beneficiaries. Weak district level management capacity has hindered the decentralization process, and building this capacity is an important link to ensure that resources translate to improved service delivery and further improvements in population health. A consortium of the Ministry of Health, Health Alliance International, Eduardo Mondlane University, and the University of Washington are implementing a health systems strengthening model in Sofala Province, central Mozambique.Description of implementation: The Mozambique Population Health Implementation and Training (PHIT) Partnership focuses on improving the quality of routine data and its use through appropriate tools to facilitate decision making by health system managers; strengthening management and planning capacity and funding district health plans; and building capacity for operations research to guide system-strengthening efforts. This seven-year effort covers all 13 districts and 146 health facilities in Sofala Province.Evaluation design: A quasi-experimental controlled time-series design will be used to assess the overall impact of the partnership strategy on under-5 mortality by examining changes in mortality pre- and post-implementation in Sofala Province compared with neighboring Manica Province. The evaluation will compare a broad range of input, process, output, and outcome variables to strengthen the plausibility that the partnership strategy led to healthsystem improvements and subsequent population health impact.Discussion: The Mozambique PHIT Partnership expects to provide evidence on the effect of efforts to improvedata quality coupled with the introduction of tools, training, and supervision to improve evidence-based decision making. This contribution to the knowledge base on what works to enhance health systems is highly replicable for rapid scale-up to other provinces in Mozambique, as well as other sub-Saharan African countries with limitedresources and a commitment to comprehensive primary health care