Bard, Debbie

English

Digital Commons @ Montana Tech

Montana Tech Library Digital Commons @ Montana Tech National Lab Day Lectures 10-9-2019 High Performance Computing in the DOE Debbie Bard Follow this and additional works at: https://digitalcommons.mtech.edu/national-lab-day High Performance Computing in the DOEDebbie BardGroup Lead for Data Science EngagementDOE’s Office of Science Advanced Scientific Computing Research (ASCR) has 3 Computation User FacilitiesNERSC Cori  is 30 PF• Oak Ridge Leadership Computing Facility (OLCF):DOE Leadership Computing Facility• Argonne Leadership Computing Facility (ALCF): DOE Leadership Computing Facility National Energy Research Scientific Computing Center (NERSC): Mission computing facility for DOE Office of ScienceOLCFSummit is 200 PFALCFTheta is 11 PF‐ 2 ‐The LCFs provide world‐leading computing facilitiesWhat’s unique about the LCFs? • Highly competitive open user allocation programs (INCITE, ALCC).• Small number of projects accepted, each receives huge amounts of computing resources• LCF centers partner with users to enable science & engineering breakthroughs.• Collaborative DOE Office of Science user‐facility program at ORNL and ANL• Mission: Provide the computational and data resources required to solve the most challenging problems.• 2‐centers/2‐architectures to address diverse and growing computational needs of the scientific community‐ 3 ‐DOE roadmap to Exascale systems‐ 4 ‐‐ 4 ‐Biology, Energy,  Environment Computing Materials, Chemistry, GeophysicsParticle Physics, AstrophysicsLargest funder of physical science research in U.S. Nuclear Physics Fusion Energy, Plasma Physics- 5 -NERSC is the mission HPC and data facility for the U.S  Department of Energy Office of Science‐ 5 ‐NERSC is the mission HPC and data facility for the U.S  Department of Energy Office of ScienceSimulations at scaleData analysis support for DOE’s experimental and observational facilitiesLargest funder of physical science research in U.S. 7,000 Users800 Projects700 Codes~2000 publications per year ‐ 6 ‐NERSC Systems: present and futureNERSC-7: EdisonMulticore CPUNERSC-8: Cori Manycore CPUNESAP Launched: transition applications to advanced architectures201320162024NERSC-9: CPU and GPU nodes Continued transition of applications and support for complex workflows2020NERSC-10:Exa system2028NERSC-11:BeyondMoore‐ 7 ‐Perlmutter: A System Optimized for Science‐ 8 ‐• Cray Shasta System providing 3‐4x capability of Cori system• First NERSC system designed to meet needs of both large scale simulation and data analysis from experimental facilities– Includes both NVIDIA GPU‐accelerated and AMD CPU‐only nodes– Cray Slingshot network for Terabit‐rate connections to system– Optimised data software stack enabling analytics and Machine Learning at scale– All‐flash file system for accelerated IOMontana’s use of NERSC‐ 9 ‐Montanans used NERSC for 2 projects this year: • Joint Genome Institute: provides the environmental genomics community resources for sequencing data and analysis• Fusion: computational studies of the equilibrium and stability properties of stellarators, and numerical studies of transport dynamics in toroidal confinement devices. (how many hours on our supercomputers)How NERSC time is allocated• SBIR: small business innovation and research• HPC4Manufacturing/HPC4 Energy Innovation‐ 10 ‐• Eg Agenda2020 partnered with LLNL/LBNL/NERSC to optimize most energy-intensive steps in the  papermaking process: drying the pulp• Used HPC simulations of water movement through pulp• What would the universe look like under different theoretical models? • Do telescope observations match the theoretical prediction?HACC Outer Rim simulations up to ~1% of observable universeSupercomputing for modeling the universe‐ 12 ‐• How does photosynthesis happen?• How do drugs dock with proteins in our cells?• Why do jet engines fail?Supercomputing for real‐time experiments Super-intense femtosecond xray pulses, >10PB data, up to 100 PF required for analysisSupercomputing for data analysis‐ 13 ‐A billion proton-proton collisions per second and multi-GB of data per second. • What is the relationship between fundamental particles?• What is the mechanism that gives matter mass? Supercomputing for genome sequencing• How does the soil microbiome impact crop success? • How did viruses evolve?• Can we engineer enzymes for more effective carbon fixation?>170 trillion bases sequenced per year, >7PB of archived data, >100,000 usersWhy do we need supercomputers for AI? • Scientific data is big– Takes days/weeks to train algorithm on scientific datasets– Scientific thought moves faster than this!• Scientific data is complex– Noisy, large dynamic range, multi‐dimensional– Need to try many models to find one that works for your data• Scientific data is detailed– Cannot use low‐fidelity approximations– Cannot use hardware designed for FP8 format‐ 15 ‐AI for science needs to perform the same tasks as AI for commercial applicationsCharacterising Extreme Weather in a Changing Climate• High quality segmentation results obtained for climate data.• Network scaled out to 4560 nodes on Summit @ OLCF (27,360 Volta GPUs).• Largest application of TensorFlow on GPU‐based system, first Exascale Deep Learning app.• Won Gordon Bell Prize 2018‐ 16 ‐ https://arxiv.org/abs/1810.01993• The DOE Computing Facilities enable scientists to push the edge of what is possible for US science and innovation—enabling scientists and engineers to explore systems too large, too complex, too dangerous, too small, or too fleeting to do in the real world.• We lead the nation and the world in supercomputing, high‐end computational science, and advanced networking for science.  . Thanks! NERSC is hiring! nersc.gov/careersNERSC’s current supercomputer: Cori• Cray XC System– >9600 68‐core Intel KNL compute nodes, >2800 32‐core Intel Haswell nodes• Cray Aries Interconnect• NVRAM Burst Buffer, 1.6PB of SSDs, 1.7TB/sec I/O• Lustre file system 28 PB of disk, >700 GB/sec I/O• Cori is capable of 28 petaflops– 28 thousand million million floating point operations per second– Equal to 7 billion people on 4 million earths doing one calculation per second‐ 19 ‐#5 most powerful computer on the planet in Nov 2016.#12 today. What makes a supercomputer special? ‐ 20 ‐The high speed, low latency network  is what allows the nodes to “talk” to each other and work together to solve problems you could never solve on your laptop, or even 150,000 laptops.• Typical point-to-point bandwidth:• A supercomputer: 10 GB/sec• Your home/4G network: 0.02 GB/secEg to simulation the behavior of the Earth’s atmosphere, divide the Earth into zones1. Each processor computes what’s happening in its zone.1. Share results of calculations with the other nearby  processors, as each atmospheric cell is connected. 2. Calculate what’s happening now in each zoneWithout the custom high-speed network on a supercomputer, the communication would take far more time than the calculation! Supercomputers have super‐fast interconnect between nodes• Too Big• To  Small• Too Dangerous• Too Fast• Too Slow• Too ExpensiveToo slowSimulating climate events under differing conditions• E.g. count how many hurricanes/tropical storms occurToo expensiveMaterial design• Simulate characteristics of different materials that could be used for more efficient photovoltaic  cells or more efficient batteries• Reduce need to fabricate and test useless materials in the lab Some science problems cannot be solved without Exascalecomputing• 1 Exaflop is a lot of compute power! • 1,000,000,000,000,000,000 calculations per second• This is the power of 50 million laptops• All 7.5 billion people on 130 million earths, doing one calculation per second(slide from Intel)

High Performance Computing in the DOE

https://digitalcommons.mtech.edu/cgi/viewcontent.cgi?article=1018&amp;context=national-lab-day

High Performance Computing in the DOE

Abstract

Similar works

Full text

Available Versions

Digital Commons @ Montana Tech