DNNShifter: An Efficient DNN Pruning System for Edge Computing

Deep neural networks (DNNs) underpin many machine learning applications. Production quality DNN models achieve high inference accuracy by training millions of DNN parameters which has a significant resource footprint. This presents a challenge for resources operating at the extreme edge of the network, such as mobile and embedded devices that have limited computational and memory resources. To address this, models are pruned to create lightweight, more suitable variants for these devices. Existing pruning methods are unable to provide similar quality models compared to their unpruned counterparts without significant time costs and overheads or are limited to offline use cases. Our work rapidly derives suitable model variants while maintaining the accuracy of the original model. The model variants can be swapped quickly when system and network conditions change to match workload demand. This paper presents DNNShifter, an end-to-end DNN training, spatial pruning, and model switching system that addresses the challenges mentioned above. At the heart of DNNShifter is a novel methodology that prunes sparse models using structured pruning. The pruned model variants generated by DNNShifter are smaller in size and thus faster than dense and sparse model predecessors, making them suitable for inference at the edge while retaining near similar accuracy as of the original dense model. DNNShifter generates a portfolio of model variants that can be swiftly interchanged depending on operational conditions. DNNShifter produces pruned model variants up to 93x faster than conventional training methods. Compared to sparse models, the pruned model variants are up to 5.14x smaller and have a 1.67x inference latency speedup, with no compromise to sparse model accuracy. In addition, DNNShifter has up to 11.9x lower overhead for switching models and up to 3.8x lower memory utilisation than existing approaches.Comment: 14 pages, 7 figures, 5 table

DNNShifter : an efficient DNN pruning system for edge computing

Funding: This research is funded by Rakuten Mobile, Japan .Deep neural networks (DNNs) underpin many machine learning applications. Production quality DNN models achieve high inference accuracy by training millions of DNN parameters which has a significant resource footprint. This presents a challenge for resources operating at the extreme edge of the network, such as mobile and embedded devices that have limited computational and memory resources. To address this, models are pruned to create lightweight, more suitable variants for these devices. Existing pruning methods are unable to provide similar quality models compared to their unpruned counterparts without significant time costs and overheads or are limited to offline use cases. Our work rapidly derives suitable model variants while maintaining the accuracy of the original model. The model variants can be swapped quickly when system and network conditions change to match workload demand. This paper presents DNNShifter  , an end-to-end DNN training, spatial pruning, and model switching system that addresses the challenges mentioned above. At the heart of DNNShifter  is a novel methodology that prunes sparse models using structured pruning - combining the accuracy-preserving benefits of unstructured pruning with runtime performance improvements of structured pruning. The pruned model variants generated by DNNShifter  are smaller in size and thus faster than dense and sparse model predecessors, making them suitable for inference at the edge while retaining near similar accuracy as of the original dense model. DNNShifter  generates a portfolio of model variants that can be swiftly interchanged depending on operational conditions. DNNShifter  produces pruned model variants up to 93x faster than conventional training methods. Compared to sparse models, the pruned model variants are up to 5.14x smaller and have a 1.67x inference latency speedup, with no compromise to sparse model accuracy. In addition, DNNShifter  has up to 11.9x lower overhead for switching models and up to 3.8x lower memory utilisation than existing approaches. DNNShifter  is available for public use from https://github.com/blessonvar/DNNShifter.Publisher PDFPeer reviewe

Maternal serum cytokine concentrations in healthy pregnancy and preeclampsia

The maternal immune response is essential for successful pregnancy, promoting immune tolerance to the fetus while maintaining innate and adaptive immunity. Uncontrolled, increased proinflammatory responses are a contributing factor to the pathogenesis of preeclampsia. The Th1/Th2 cytokine shift theory, characterised by bias production of Th2 anti-inflammatory cytokine midgestation, was frequently used to reflect the maternal immune response in pregnancy. This theory is simplistic as it is based on limited information and does not consider the role of other T cell subsets, Th17 and Tregs. A range of maternal peripheral cytokines have been measured in pregnancy cohorts, albeit the changes in individual cytokine concentrations across gestation is not well summarised. Using available data, this review was aimed at summarising changes in individual maternal serum cytokine concentrations throughout healthy pregnancy and evaluating their association with preeclampsia. We report that TNF-α increases as pregnancy progresses, IL-8 decreases in the second trimester, and IL-4 concentrations remain consistent throughout gestation. Lower second trimester IL-10 concentrations may be an early predictor for developing preeclampsia. Proinflammatory cytokines (TNF-α, IFN-γ, IL-2, IL-8, and IL-6) are significantly elevated in preeclampsia. More research is required to determine the usefulness of using cytokines, particularly IL-10, as early biomarkers of pregnancy health

FAR out? An examination of converging, diverging and intersecting smart grid futures in the United Kingdom

We describe a novel application of the field anomaly relaxation (FAR) method of scenario construction to the complex problem of smart grid development. We augment the FAR methodology with extensive expert input through all four steps to incorporate detailed knowledge of the technical, economic and policy issues relevant to informing scenarios for smart grid development in the United Kingdom. These steps inform scenarios useful to policymakers, regulators and the energy industry. We found this extended method to be flexible and reliable. Analysis of smart grid development yielded seven dimensions, allowing for portrayal of a complex and informed set of scenarios. The expert input and feedback identified branching points allowing switching between scenarios – a powerful dynamic feature to assist policy development for a fast-changing technological and regulatory landscape

Synthesis of 3-D coronal-solar wind energetic particle acceleration modules

1. Introduction Acute space radiation hazards pose one of the most serious risks to future human and robotic exploration. Large solar energetic particle (SEP) events are dangerous to astronauts and equipment. The ability to predict when and where large SEPs will occur is necessary in order to mitigate their hazards. The Coronal-Solar Wind Energetic Particle Acceleration (C-SWEPA) modeling effort in the NASA/NSF Space Weather Modeling Collaborative [Schunk, 2014] combines two successful Living With a Star (LWS) (http://lws. gsfc.nasa.gov/) strategic capabilities: the Earth-Moon-Mars Radiation Environment Modules (EMMREM) [Schwadron et al., 2010] that describe energetic particles and their effects, with the Next Generation Model for the Corona and Solar Wind developed by the Predictive Science, Inc. (PSI) group. The goal of the C-SWEPA effort is to develop a coupled model that describes the conditions of the corona, solar wind, coronal mass ejections (CMEs) and associated shocks, particle acceleration, and propagation via physics-based modules. Assessing the threat of SEPs is a difficult problem. The largest SEPs typically arise in conjunction with X class flares and very fast (\u3e1000 km/s) CMEs. These events are usually associated with complex sunspot groups (also known as active regions) that harbor strong, stressed magnetic fields. Highly energetic protons generated in these events travel near the speed of light and can arrive at Earth minutes after the eruptive event. The generation of these particles is, in turn, believed to be primarily associated with the shock wave formed very low in the corona by the passage of the CME (injection of particles from the flare site may also play a role). Whether these particles actually reach Earth (or any other point) depends on their transport in the interplanetary magnetic field and their magnetic connection to the shock

Effects of strength training on postpubertal adolescent distance runners

Purpose Strength training activities have consistently been shown to improve running economy (RE) and neuromuscular characteristics, such as force-producing ability and maximal speed, in adult distance runners. However, the effects on adolescent (<18 yr) runners remains elusive. This randomized control trial aimed to examine the effect of strength training on several important physiological and neuromuscular qualities associated with distance running performance. Methods Participants (n = 25, 13 female, 17.2 ± 1.2 yr) were paired according to their sex and RE and randomly assigned to a 10-wk strength training group (STG) or a control group who continued their regular training. The STG performed twice weekly sessions of plyometric, sprint, and resistance training in addition to their normal running. Outcome measures included body mass, maximal oxygen uptake (VO 2max ), speed at VO 2max , RE (quantified as energy cost), speed at fixed blood lactate concentrations, 20-m sprint, and maximal voluntary contraction during an isometric quarter-squat. Results Eighteen participants (STG: n = 9, 16.1 ± 1.1 yr; control group: n = 9, 17.6 ± 1.2 yr) completed the study. The STG displayed small improvements (3.2%-3.7%; effect size (ES), 0.31-0.51) in RE that were inferred as "possibly beneficial" for an average of three submaximal speeds. Trivial or small changes were observed for body composition variables, VO 2max and speed at VO 2max ; however, the training period provided likely benefits to speed at fixed blood lactate concentrations in both groups. Strength training elicited a very likely benefit and a possible benefit to sprint time (ES, 0.32) and maximal voluntary contraction (ES, 0.86), respectively. Conclusions Ten weeks of strength training added to the program of a postpubertal distance runner was highly likely to improve maximal speed and enhances RE by a small extent, without deleterious effects on body composition or other aerobic parameters

Trajectories of Change in an Open-access Internet-Based Cognitive Behavior Program for Childhood and Adolescent Anxiety: Open Trial

Background: Although evidence bolstering the efficacy of internet-based cognitive behavioral therapy (iCBT) for treating childhood anxiety has been growing continuously, there is scant empirical research investigating the timing of benefits made in iCBT programs (eg, early or delayed). Objective: This study aims to examine the patterns of symptom trajectories (changes in anxiety) across an iCBT program for anxiety (BRAVE Self-Help). Methods: This study’s participants included 10,366 Australian youth aged 7 to 17 years (4140 children aged 7-12 years; 6226 adolescents aged 12-17 years) with elevated anxiety who registered for the BRAVE Self-Help program. Participants self-reported their anxiety symptoms at baseline or session 1 and then at the commencement of each subsequent session. Results: The results show that young people completing the BRAVE Self-Help program tend to fall into two trajectory classes that can be reliably identified in terms of high versus moderate baseline levels of anxiety and subsequent reduction in symptoms. Both high and moderate anxiety severity trajectory classes showed significant reductions in anxiety, with the greatest level of change being achieved within the first six sessions for both classes. However, those in the moderate anxiety severity class tended to show reductions in anxiety symptoms to levels below the elevated range, whereas those in the high symptom group tended to remain in the elevated range despite improvements. Conclusions: These findings suggest that those in the high severity group who do not respond well to iCBT on a self-help basis may benefit from the additional support provided alongside the program or a stepped-care approach where progress is monitored and support can be provided as necessary