Bonn, M.

Chilingaryan, S.

Kopmann, A.

Mauch, V.

Mexner, W.

Ressmann, D.

KITopen

V. Mauch, M. Bonn, S. Chilingaryan, A. Kopmann, W. Mexner, D. RessmannKarlsruhe Institute of Technology (KIT), Karlsruhe, GermanyAbstractModern data analysis applications for 2D/3D data sam-ples require complex visual output features which are oftenbased on OpenGL, a multi-platform API for rendering vec-tor graphics. They demand special computing workstationswith a corresponding CPU and GPU power, enough mainmemory and fast network interconnects for a performantremote data access. For this reason, users depend heavilyon available free workstations, both temporally and locally.The provision of virtual machines (VMs) accessible via aremote connection could avoid this inflexibility. However,the automatic deployment, operation and remote access ofOpenGL-capable VMs with professional visualization ap-plications is a non-trivial task.In this paper, we discuss a concept for a flexible analysisinfrastructure that will be part in the project ASTOR, whichis the abbreviation for “Arthropod Structure revealed by ultra-fast Tomography and Online Reconstruction”. We presentan Analysis-as-a-Service (AaaS) approach based on the on-demand allocation of VMs with dedicated GPU cores and acorresponding analysis environment to provide a cloud-likeanalysis service for scientific users.INTRODUCTIONParticle accelerators like ANKA [1] provide synchrotronradiation for the investigation of solid material andbiological samples. X-ray imaging techniques producelarge data sets of up to several 100 Gigabytes. Thesubsequent processing of the data requires special analysisapplications whose features and performance dependheavily on available CPU, GPU and memory resources.Typical analysis application like Amira [2] or VG StudioMAX [3] provide visual output based on OpenGL [4], amulti-platform API for rendering vector graphics. Up tonow, several conditions concerning the user analysis processhave complicated a flexible analysis workflow:• transfer, access and storage of huge data sets• required amount of computing and memory resources• OpenGL/DirectX capable GPUs for visual output, even-tually also CUDA [5] or OpenCL [6] are necessary• expensive workstation licenses for commercial analysissoftwareAll aspects mentioned above prevent the use of standardend-user devices for analysis and result in the operationof dedicated workstations for scientists. The required pro-fessional workstations need to provide a high-throughputnetwork connection to the data set storage, licensed analysissoftware and hardware setup. This scenario implies furtherdisadvantages for users. They have to rent free time rangesfor the workstation usage. Therefore they have to be presentin special computing rooms within their institution sharedwith other users. Due to the operation of powerful hard-ware combined with an increased heat generation, insidethe computing room one can often sense a high backgroundnoise. Furthermore, the workstations are typically config-ured with limited guest account privileges, which is whyusers have to consult an administrator to install additionalsoftware packages for a special purpose.To improve this situation, one part of the project ASTORdeals with the automatic deployment of virtual machines(VMs) with remote connections supporting the renderingand display of OpenGL features. Scientists should be able touse a web portal with an overview of their available data setsand allocate virtual resources for their analysis on-demand.In this paper we discuss an Analysis-as-a-Service (AaaS)concept and present a first prototype implementation for thesynchrotron community.CONCEPTThe replacement of static localized stand-alone worksta-tion towards a flexible analysis infrastructure concept forscientists is achievable with an Infrastructure-as-a-Service(IaaS) approach. Besides the provision of virtualizedanalysis environments, a suitable analysis workflow has to bedefined which also considers the previous data detection andrecording, data set access and the final result archiving. Thecomplete analysis workflow could be considered as a novelAnalysis-as-a-Service (AaaS) concept, defining a cloud-likeservice for individually customized analysis processes.The main part of this concept is the intelligent integra-tion of virtualized analysis environments. However, until afew years ago, the provision of remote connections to VMssupporting OpenGL or other complex visual APIs was im-possible, as most virtualization solutions only came up withsimplified graphic interfaces within a guest system. Fur-thermore the remote access protocol is a key aspect withregard to lossless data compression and smooth transfer ofvisual information. It must ensure a low-latency user inter-action without any disturbing lag effects, as most analysisprocesses require several hours of work. Currently, thereare just a few complete solution suites available offeringthe provision of virtualized workstations for professionalvisualization applications:• Citrix XenDesktop [7]• Microsoft RemoteFX [8]• VMware Horizon View [9]OpenGL-BASED DATA ANALYSIS IN VIRTUALIZED SELF-SERVICEENVIRONMENTSFCO202 Proceedings of PCaPAC2014, Karlsruhe, GermanyISBN 978-3-95450-146-5234Copyright©2014CC-BY-3.0andbytherespectiveauthorsUser Interfaces and Data DisplaysAll of them are based on special GPU resources like theNVidia GRID Technology [10]. The allocation of the GPUresources is possible via a special software layer which al-lows GPU sharing between several VMs. To take advan-tage of all possible GPU-based features within a VM, it isusual to grant direct access to a dedicated GPU core via PCIpassthrough.A fast network interconnect between VMs and the datastorage systems is another critical factor. The remote pro-cessing of large data sets in the order of up to 100 GB vianetwork mount points requires a network bandwidth startingat ~10 Gbit/s. In this regard software virtualized networkdevices are reaching their limits. Providing near-native per-formance of high-speed network interconnects like 10 Gbit/sEthernet or even 56 Gbit/s InfiniBand within VMs requireshardware virtualized approaches like Single Root - I/O Vir-tualization (SR-IOV) [11]. This PCI specification extensionallows a single PCI Express (PCIe) I/O device to appear asmultiple, separate devices — called Virtual Functions (VF)—a kind of a "light weighted" virtual PCIe device. Each VFcan be allocated to one VM via PCI passthrough.To ensure the best possible user experience, the provi-sion of the virtualized environments has to be integratedwithin the IT systems of the corresponding synchrotron in-stitution. Users should be able to authenticate with theirexisting federated accounts towards a web service providingtheir data sets and analysis infrastructure. New created VMsshould contain the user-specific network mount points for thedata set access. Pre-installed proprietary analysis softwarehas to be activated automatically by contacting existing pre-defined license servers. Non-used allocated resources haveto be reassigned to other users. In this case a mechanismis needed which "motivates" inactive user to release theirunused resources, e.g. shut down and/or delete the VM.In the next section, we will describe our prototypic imple-mentation based on the basic conditions mentioned above.IMPLEMENTATIONOne goal of the ASTOR project foresees the provision ofvirtualized workstations for the execution of analysis soft-ware like Amira [2]. Our first implementation for this pur-pose is based on the VMware Horizon View [9] virtualiza-tion solution combined with the vSphere v5.5 hypervisor.We use NVidia GRID K2 GPU [10] resources, which areallocated to the VMs via Virtualized Dedicated GraphicsAcceleration (vDGA) [12] using PCI passthrough. Each VMgets physical access to a dedicated high-end Kepler corewith access to 4GB GDDR5 memory. Combined with thePCoIP [13] remote protocol, all important visualization APIslike OpenGL 4.1x or DirectX 11 are supported. Carefreeinteractive usage without annoying time delays is alreadypossible from home office with a common DSL connectionabout 16 Mbit/s and better.Access and processing of data sets should take place onremote network-attached storage systems, which requires afast network interconnect. Therefore we will provide Infini-Band support within VMs. Each VM gets direct access to ahardware-virtualized InfiniBand VF provided by an SR-IOVcapable InfiniBand Host Channel Adapter. Deployment andmanagement of running VMs and templates is performantenough with a NFS shared storage system accessible via 10Gbit/s Ethernet.All VMware resources are managed by our software layercalled Cloud Service Bus, which was originally developedfor the transparent provision of public and private IaaS cloudresources [14]. It utilizes VMware SDK and the AmazonWeb Services (AWS) SDK for the control of the correspond-ing VMs/templates and appends value-added services. Aunified SOAP-API provides all common functionalities.Currently VMware does not provide any intelligent sched-ulers for initial placement supporting VMs with allocatedPCIe devices within a multi-node cluster. Furthermore,allocation and release of PCIe resources during differentVM/cluster states has to be triggered manually. Thereforewe extended our management software functionality withan initial placement mechanism for VMs with PCIe devices.The placement decision of new created and existing VMswithin the cluster is based on free available PCIe, RAMand CPU resources. This is an important feature for anautomated VM deployment as cluster load balancing vialive-migration of VMs1 is not possible with direct hardwareaccess to PCIe devices.Our management software provides further essential fea-tures for the integration of IaaS resources into a holisticanalysis workflow. Several prepared VM templates are avail-able with different analysis software packages. Users areable to define their specific VM hardware configuration con-cerning their needs on-demand. Customization mechanismswithin the Cloud Service Bus allow to pre-configure newcreated VMs with corresponding specific network mountpoints, license server information and more, as availableVM templates are cleaned up by all personalized data.Users can authenticate themselves with their existing Ac-tive Directory (AD) account towards the Cloud Service Bus.The complete resource usage and load statistics is monitoredfor general liability and billing. These informations are alsoaccessible by the users for their personal purposes. It is pos-sible to define billing algorithms based on the CPU, RAMand DISK consumption respectively wall time reservationof a VM. To prevent resource allocation by inactive users,credits can be assigned. In the case of exhausted credits,VMs will be shut down after a warning email.Based on the available SOAP-API, we developed a cross-platform web service called YVAINE [15], with a specialfocus on a user friendly and intuitive design. Users shouldcreate and manage their resources with a few mouse clicksonly. This service is the basis for the ASTOR web portal andwill be combined with an overview concerning the users’data sets. The simple choice of an existing data set will createa specific VMwith all necessary software packages, network1 Also known as VMware vMotion and used by the VMware Distributed Re-sources Scheduler (DRS) within a multi-note virtualization environmentfor load balancing.Proceedings of PCaPAC2014, Karlsruhe, Germany FCO202User Interfaces and Data DisplaysISBN 978-3-95450-146-5235 Copyright©2014CC-BY-3.0andbytherespectiveauthorsHost 01 Host 02VM1 VM3Virtualization Infrastructure(VMware ESXi & NVidia GPU)ASTORWEB-PortalVMs & Data SetsVMware Storage(Images & VMs)Scratch Storage Processing StorageInfiniBand10GEVM4VM2User Management Active Directory Authentication Load Statistics / Measurement Billing User Interfaces (over SOAP-API) Web Service / .NET Client Command Line InterfaceVMware vSphere Adapter GPU & I/O-Device PCIe Initial Placement Logic Image Management / ProvisionCloud Service BusAnalysis-as-a-Service (AaaS)HTTPSHorizon Clientmounts and required computing resources and provide thecorresponding access credentials to the users. Figure 1. illus-trates the architecture of our current Analysis-as-a-Serviceworkflow implementation.Figure 1: Analysis-as-a-Service Architecture. Users loginto the ASTOR web portal, choose their specific data sets andget access credentials to their specific virtual machines foranalysis, best suited to their needs.CONCLUSION & OUTLOOKThe first prototype of the ASTOR "Analysis-as-a-Service"infrastructure is already a significant improvement comparedto the traditional strategy with dedicated analysis worksta-tions. Scientific users are flexible concerning their scheduledtime and get professional high-end resources and fast accessto their large data sets for data analysis at arbitrary locations.Future plans foresee the support of more federated accountsolutions like Shibboleth. Furthermore we will improve theinitial placement mechanisms based on the past and expectedresource consumption of running VMs. The evaluation ofalternative storage solutions like noSQL databases is anotherresearch aspect within the ASTOR project.ACKNOWLEDGMENTWe want to thank the Federal Ministry of Education andResearch in Germany for the funding of our research withinthe project ASTOR.REFERENCES[1] ANKA – the Synchrotron Radiation Facility at the KarlsruheInstitute of Technology (KIT)http://www.anka.kit.edu/[2] Amira 3D Software for Life Scienceshttp://www.fei.com/software/amira-3d-for-life-sciences/[3] VG Studio MAXhttp://www.volumegraphics.com/produkte/vgstudio-max/[4] OpenGL - The Industry’s Foundation for High PerformanceGraphicshttp://www.opengl.org/[5] CUDA – Parallel Programming and Computing Platformhttp://www.nvidia.com/object/cuda_home_new.html[6] OpenCL – The open standard for parallel programming ofheterogeneous systemshttp://www.khronos.org/opencl/[7] Citrix XenDestkophttp://www.citrix.com/products/xendesktop/overview.html[8] Microsoft RemoteFXhttp://technet.microsoft.com/en-us/library/ff817578(v=ws.10).aspx[9] VMware Horizon Viewhttp://www.vmware.com/products/horizon-view[10] NVidia GRID - Graphics-Accelerated Virtualizationhttp://www.nvidia.com/object/grid-technology.html[11] Single Root - I/O Virtualization (SR-IOV)https://www.pcisig.com/specifications/iov/single_root/[12] Graphics Acceleration in View Virtual Desktopshttp://www.vmware.com/files/pdf/techpaper/vmware-horizon-view-graphics-acceleration-deployment.pdf[13] PC over IP remote protocolhttp://www.teradici.com/pcoip-technology[14] B2B- Infrastructure Integration via Cloud Service BusiX Kompakt: IT-Management in der Cloud, 04-2012, Pages48–51, September, 2012[15] YVAINE, SCC News 01/2013, Karlsruhe, Germanyhttp://www.scc.kit.edu/publikationen/scc-news.phpFCO202 Proceedings of PCaPAC2014, Karlsruhe, GermanyISBN 978-3-95450-146-5236Copyright©2014CC-BY-3.0andbytherespectiveauthorsUser Interfaces and Data Displays

OpenGL-based data analysis in virtualized self-service environments

https://publikationen.bibliothek.kit.edu/170101031/18803379

OpenGL-based data analysis in virtualized self-service environments

Abstract

Similar works

Full text

Available Versions

KITopen