Baer, Troy

Johnson, Douglas

Oottikkal, Shameema

KnowledgeBank at OSU

Translational Data Analytics @ Ohio StateApache Spark on HPC clustersHPC service at OSCShameema Oottikkal, Troy Baer, and Douglas JohnsonOhio Supercomputer Center, 1224 Kinnear Road, Columbus, OH 43212Data Analytics Tools at OSC Case studyEmpowering Clients: Organizational Impact CY2015OSC Service CatalogCluster Computing• High Performance Computing• High Throughput Computing• Data-intensive ComputingResearch Data Storage• Project Storage• Archival StorageClient Services• 24x7 Call Center• Level 2 Engineering SupportClient Facilitation• Consultation (in-person and online)• Training and Education• Classroom accountsScientific Software Development• Software Development• Software ParallelizationWeb Software Development• Software Development• Software ConsultingPartner on Proposals• Cyberinfrastructure solutions • Modeling & simulation for industryVisualization & Virtual Environments• Visualization Services• Virtual environments (DSL)Supercomputers at OSC Owens(2016)Ruby(2014)Oakley (2012)Theoretical Performance ~750 TF ~144 TF ~154 TF# Nodes ~820 240 692# CPU Cores ~23,500 4800 8304Total Memory ~120 TB ~15.3 TB ~33.4 TBMemory per Core >5 GB 3.2 GB 4 GBInterconnect EDR IB FDR/EN IB QDR IBAccess OSC HPC clusters via ondemand.osc.eduStorageHome Directory Space 900 TB usable (Disk) (Allocated to each user, 500 GB quota limit)Scratch – DDN GPFS1 PB with 40-50 GB/s peak performanceProject – DDN GPFS 3.4 PBPython: A popular general-purpose, high-level programming language with numerous mathematical and scientific packages available for data analytics.R: A programming language for statistical and machine learning applications with very strong graphical capabilities.MATLAB:A full featured data analysis toolkit with many advanced algorithms readily available.Spark: A framework for running map reduce algorithms in memory.Intel Compilers: Compilers for generating optimized code for Intel CPUs.Intel MKL: The Math Kernel Library provides optimized subroutines for common computation tasks such as matrix-matrix calculations.Statistical software: Octave, Stata, FFTW, ScaLAPACK, MINPACK, sprng2Apache SparkApache Spark is an open source cluster computing framework originally developed in the AMPLab at University of California, Berkeley but was later donated to the Apache SoftwareRun Spark Today on OSC clusters#PBS -N spark-statistics#PBS -l nodes=18:ppn=28#PBS -l walltime=00:20:00module load sparkpbs-spark-submit stati.py > stati.logExample Job Submission ScriptImport to SparkSave as parquet fileAnalysisReload to SparkNewerDataAppend to parquet fileImport to SparkDATA on MYSQL DBMaster nodeWorker nodeWorker nodeWorker nodeWorker nodeWorker nodeWorker nodeWorker nodeTaskResultTask ResultTaskResultTaskResultTaskResultTaskResultTask ResultStatistics MYSQL SPARKJob vs CPU 1 hour 5 secCPU vs Account 1.25 hour 5 secWalltime vs user 1.40 hour 5 secOhio Supercomputer Center provides high performance computing, software, storage and support services for Ohio's scientists, faculty, students, businesses and their research partners.Data mining of historical jobs records of OSC’s clustersAim: To understand client utilizations of OSC recourses. Data: Historical records of every Job that ran on any OSC clusters that includes information's such as number of nodes, software, CPU time  and timestamp.Running complex queries on MYSQL DB takes hours of CPU time. We implemented a workflow using Spark that reduces CPU time both for loading and analyzing data from hours to seconds. Bottle neck in the workflow is the process of loading data from MYSQL DB. But once loaded, data is saved as a parquet file to a faster file system which can be accessed in seconds by Spark.  Newer data will be loaded periodically to append to the parquet file, such that our internal users can run complex queries faster.ReferencesVisit Ohio Super Computer website: https://www.osc.edu/OSC OnDemand: https://ondemand.osc.edu/Available software at OSC: https://www.osc.edu/supercomputing/softwareApache Spark: http://spark.apache.org/High Performance big data: http://hibd.cse.ohio-state.edu/ResultsWorkflowData till 2016Foundation where it remains today. In contrast to Hadoop's disk-based analytics paradigm, Spark has multi-stage in-memory analytics.Pyspark code#importing datadf=sqlContext.read.parquet("/fs/scratch/pbsacct/Jobs.parquet")#Which types of queue is mostly useddf_mysql.select("jobid",queue").groupBy("queue").count().show()#Which software is used most?from pyspark.sql.functions import coldf_mysql.select("jobid","sw_app").groupBy("sw_app").count().sort(col("count").desc()) .show()’ #who uses gaussian software most?df_mysql.registerTempTable("mysql”) sqlContext.sql(" SELECT username FROM mysql WHERE sw_app='gaussian’ " ).show()

Apache Spark on HPC clusters

https://kb.osu.edu/bitstream/1811/79467/1/TDA_FallForum_2016_OottikkalShameema.pdf

Apache Spark on HPC clusters

Abstract

Similar works

Full text

Available Versions

KnowledgeBank at OSU