456 research outputs found
Dynamic Provision of Heterogeneous Computing Resources for Computation- and Data-intensive Particle Physics Analyses
Schweizerisches Wirtschaftsarchiv, Basel
Bestandesübersicht des Schweizerischen Wirtschaftsarchivs in Basel
Dynamic Resource Extension for Data Intensive Computing with Specialized Software Environments on HPC Systems
Modern High Energy Physics (HEP) requires large-scale processing of extensive
amounts of scientific data. The needed computing resources are currently
provided statically by HEP specific computing centers. To increase the number
of available resources, for example to cover peak loads, the HEP computing development
team at KIT concentrates on the dynamic integration of additional
computing resources into the HEP infrastructure. Therefore, we developed ROCED,
a tool to dynamically request and integrate computing resources including
resources at HPC centers and commercial cloud providers. Since these resources
usually do not support HEP software natively, we rely on virtualization and container
technologies, which allows us to run HEP workflows on these so called
opportunistic resources. Additionally, we study the efficient processing of huge
amounts of data on a distributed infrastructure, where the data is usually stored
at HEP specific data centers and is accessed remotely over WAN. To optimize
the overall data throughput and to increase the CPU efficiency, we are currently
developing an automated caching system for frequently used data that is transparently
integrated into the distributed HEP computing infrastructure
Transparent Integration of Opportunistic Resources into the WLCG Compute Infrastructure
The inclusion of opportunistic resources, for example from High Performance Computing (HPC) centers or cloud providers, is an important contribution to bridging the gap between existing resources and future needs by the LHC collaborations, especially for the HL-LHC era. However, the integration of these resources poses new challenges and often needs to happen in a highly dynamic manner. To enable an effective and lightweight integration of these resources, the tools COBalD and TARDIS are developed at KIT.
In this contribution we report on the infrastructure we use to dynamically offer opportunistic resources to collaborations in the World Wide LHC Computing Grid (WLCG). The core components are COBalD/TARDIS, HTCondor, CVMFS and modern virtualization technology. The challenging task of managing the opportunistic resources is performed by COBalD/TARDIS. We showcase the challenges, employed solutions and experiences gained with the provisioning of opportunistic resources from several resource providers like university clusters, HPC centers and cloud setups in a multi VO environment. This work can serve as a blueprint for approaching the provisioning of resources from other resource providers
Advancing throughput of HEP analysis work-flows using caching concepts
High throughput and short turnaround cycles are core requirements for efficient processing of data-intense end-user analyses in High Energy Physics (HEP). Together with the tremendously increasing amount of data to be processed, this leads to enormous challenges for HEP storage systems, networks and the data distribution to computing resources for end-user analyses. Bringing data close to the computing resource is a very promising approach to solve throughput limitations and improve the overall performance. However, achieving data locality by placing multiple conventional caches inside a distributed computing infrastructure leads to redundant data placement and inefficient usage of the limited cache volume. The solution is a coordinated placement of critical data on computing resources, which enables matching each process of an analysis work-flow to its most suitable worker node in terms of data locality and, thus, reduces the overall processing time. This coordinated distributed caching concept was realized at KIT by developing the coordination service NaviX that connects an XRootD cache proxy infrastructure with an HTCondor batch system. We give an overview about the coordinated distributed caching concept and experiences collected on prototype system based on NaviX
HEPScore: A new CPU benchmark for the WLCG
HEPScore is a new CPU benchmark created to replace the HEPSPEC06 benchmark
that is currently used by the WLCG for procurement, computing resource pledges
and performance studies. The development of the new benchmark, based on HEP
applications or workloads, has involved many contributions from software
developers, data analysts, experts of the experiments, representatives of
several WLCG computing centres, as well as the WLCG HEPScore Deployment Task
Force. In this contribution, we review the selection of workloads and the
validation of the new HEPScore benchmark.Comment: Paper submitted to the proceedings of the Computing in HEP Conference
2023, Norfol
Proceedings of the 4th bwHPC Symposium
The bwHPC Symposium 2017 took place on October 4th, 2017, Alte Aula, Tübingen. It focused on the presentation of scientific computing projects as well as on the progress and the success stories of the bwHPC realization concept. The event offered a unique opportunity to engage in an active dialogue between scientific users, operators of bwHPC sites, and the bwHPC support team
Reconstruction of signal amplitudes in the CMS electromagnetic calorimeter in the presence of overlapping proton-proton interactions
A template fitting technique for reconstructing the amplitude of signals produced by the lead tungstate crystals of the CMS electromagnetic calorimeter is described. This novel approach is designed to suppress the contribution to the signal of the increased number of out-of-time interactions per beam crossing following the reduction of the accelerator bunch spacing from 50 to 25 ns at the start of Run 2 of the LHC. Execution of the algorithm is sufficiently fast for it to be employed in the CMS high-level trigger. It is also used in the offline event reconstruction. Results obtained from simulations and from Run 2 collision data (2015-2018) demonstrate a substantial improvement in the energy resolution of the calorimeter over a range of energies extending from a few GeV to several tens of GeV.Peer reviewe
- …