We report on a large scale simulation of beam-beam effects for the CERN Large Hadron Collider (LHC). The stability of particles which experience head-on and long-range beam-beam effects was investigated for different optical configurations and machine imperfections. To cover the interesting parameter space required computing resources not available at CERN. The necessary resources were available in the LHC@home project, based on the BOINC platform. At present, this project makes more than 60000 hosts available for distributed computing. We shall discuss our experience using this system during a simulation campaign of more than six months and describe the tools and procedures necessary to ensure consistent results. The results from this extended study are presented and future plans are discussed

Herr, Werner

Kaltchev, D I

McIntosh, E

Schmidt, F

CERN Document Server

EUROPEAN ORGANIZATION FOR NUCLEAR RESEARCHEuropean Laboratory for Particle PhysicsLarge scale beam-beam simulations for the CERN LHC using distributedcomputing resourcesWe report on a large scale simulation of beam-beam effects for the CERN Large Hadron Collider (LHC).The stability of particles which experience head-on and long-range beam-beam effects was investigated fordifferent optical configurations and machine imperfections. To cover the interesting parameter spacerequired computing resources not available at CERN. The necessary resources were available in theLHC@home project, based on the BOINC platform. At present, this project makes more than 60000 hostsavailable for distributed computing. We shall discuss our experience using this system during a simulationcampaign of more than six months and describe the tools and procedures necessary to ensure consistentresults. The results from this extended study are presented and future plans are discussed.Large Hadron Collider ProjectAbstractW. Herr, D. Kaltchev1), E. McIntosh and F. SchmidtLHC Project Report 927CERN,CH-1211 Geneva 23,SwitzerlandGeneva, June 20061)  TRIUMF, Vancouver, CanadaCERN, Geneva, SwitzerlandPresented atEPAC'06, Edinburgh, UK, June 26-30, 2006LARGE SCALE BEAM-BEAM SIMULATIONS FOR THE CERN LHCUSING DISTRIBUTED COMPUTINGW. Herr, E. McIntosh and F. Schmidt, CERN, Geneva, Switzerland,D. Kaltchev, TRIUMF, Vancouver, CanadaAbstractWe report on a large scale simulation of beam-beam ef-fects for the CERN Large Hadron Collider (LHC). The sta-bility of particles which experience head-on and long-rangebeam-beam effects was investigated for different opticalconfigurations and machine imperfections. To cover the in-teresting parameter space required computing resources notavailable at CERN. The necessary resources were availablein the LHC@home project, based on the BOINC platform.At present, this project makes more than 60000 hosts avail-able for distributed computing. We shall discuss our ex-perience using this system during a simulation campaignof more than six months and describe the tools and pro-cedures necessary to ensure consistent results. The resultsfrom this extended study are presented and future plans arediscussed.MOTIVATIONIn the final design phase of the LHC it is required to ver-ify the long term stability, in particular in the presence ofthe beam-beam forces which eventually limit the perfor-mance of the LHC. An overview of the various aspects wasgiven in [1]. In this study we should like to answer a fewquestions with impact on the design parameters:• Which crossing scheme is preferred [1] ?• What is the effect of triplet errors ?• Which is the preferred working point ?Required Computing ResourcesFor the above, we need to study separately the optionswith alternating and non-alternating crossing schemes [1].Errors on the focusing triplet have to be assigned in a sta-tistically significant number and the steps in the tune scanhave to be chosen fine enough to not miss bad workingpoints. Beyond these we need to understand the behaviourof so-called PACMAN bunches [1] and to assess the pa-rameter set proposed for the LHC commissioning. For theabove study we have estimated about 600000 jobs runningabout 10 hrs each, corresponding to 6 million CPU hours.This amount of CPU capacity for accelerator design stud-ies is not available at CERN, where development of GRIDcomputing is dedicated to experiment event simulation andprocessing. The relatively small amount of input/output re-quired, typically less than half a megabyte of input (lessthan 50 kilobytes when compressed) and a few megabytesof output, make the application rather suitable for grosslyparallel distributed computing. Indeed for such massivestudies it is necessary to return only the summary outputof a few tens of kilobytes.The CPSS system at CERN [2] which uses the idle timeon desktop Windows PCs provides a suitable infrastruc-ture, however with only a few thousand systems available50% of the time. The CERN IT Department developed theLHC@HOME system [3], based on the Berkeley Open In-frastructure for Network Computing (BOINC) [4], provid-ing access to more than 60000 home computers, donated bymore than 30000 volunteers. It is however necessary to runeach job at least three times to guarantee the correctness ofthe results.SixTrackThe tracking code SixTrack [5] was developed to studythe long term stability of the LHC beams in the presence ofmagnetic non-linearities as well as beam-beam effects [6].Typically SixTrack simulates 60 particles at a time as theytravel around the ring, and runs the simulation for up to1 million turns. This allows to test whether the beam isgoing to remain on a stable orbit for a much longer time.Details on the procedure to find the so-called dynamic aper-ture can be found in [7]. To maximise efficiency, and toprovide minimum inconvenience to the LHC@HOME vol-unteers, SixTrack was modified to incorporate an extremelyefficient checkpoint/restart facility.THE RUN ENVIRONMENTA set of existing SHELL scripts [8] for submitting workto the CERN LSF Batch System were extensively modifiedto provide• Job submission to BOINC and CPSS• A simple text file database for managing and monitor-ing the work• Result retrieval, an essential extension, since the re-mote jobs have no access to the CERN file systems• Extensive logging and error checking• Additional physics options to easily handle multipleamplitudes, tunes, and anglesRunning on a Heterogeneous Computing SystemAn important discovery made with CPSS, and confirmedwith LHC@home, is that different Intel and Intel compati-ble PC processors can return different results, even when a32-bit static linked absolute binary executable is used. Thisproblem was tracked down to the evaluation of elementaryfunctions such as exponential and logarithm. A solutionwas found in the elementary function library CRlibm fromENS Lyon [9] which not only provides identical results,but guarantees to provide the correctly rounded double pre-cision results. 1THE PHYSICS CASEWhile at low energy the main limitation for the beamlifetime comes from the machine non-linearities, i.e. themagnetic field errors, at collision energy the limiting ef-fects are caused by the beam-beam interaction [10]. Dueto the large number of bunches, parasitic, so-called longrange interactions, are unavoidable and are eventually thelimiting factor for the LHC performance.Crossing SchemesAn active beam-beam compensation scheme is not fore-seen for the nominal LHC and the parameters are chosen tominimize the detrimental effects. A passive compensationof the first order effects due to the long range interactionscan be accomplished using an alternating crossing scheme[1], i.e. crossing in different planes in the different inter-action regions. In particular the tune shift and orbit effectscan be compensated very efficiently [1], thus reducing therequired operational tune space and avoiding off-centredcollisions. In the present layout [11] the long range inter-actions are important only in the high luminosity (i.e. lowβ∗) interactions regions. To study their effect on the beamstability we have simulated collisions only in these two in-teraction points.PACMAN EffectsAn important feature of the LHC is the very differentcollision pattern of the bunches. This is caused by the non-regular bunch filling and the collision scheme. Only abouthalf of the bunches experience all long range and head-on collisions and are considered as ”nominal” bunches.Other bunches have fewer long range (one third of nomi-nal) or head-on (two instead of four) collisions [1]. Someof these effects are compensated by the alternating crossingscheme. These non-nominal bunches are often called PAC-MAN bunches. Non-alternating schemes (i.e. all crossingsin the same plane) may enhance these effects, depending onthe distribution of the collision points. It is therefore nec-essary to investigate the dynamic behaviour separately forthese different classes of bunches. In this study we evaluatethe dynamic aperture for nominal bunches and separatelyfor the ”extreme” PACMAN bunches, i.e. those buncheswhich experience the least number of beam-beam interac-tions (head-on as well as long range).1The topic of portability and precision and practical solutions will becovered in a forthcoming paper to be presented at ICAP Chamonix.Magnetic Field ErrorsAt injection energy the field errors from the main dipoleare the main limit while at collision energy the emittancehas decreased and the bad field region of the dipoles isavoided. However, due to the small β ∗ at the collisionpoint, the beam size becomes large in the final focusingmagnets and their field errors need to be considered [11].PROCEDUREParameter set Since the stability further depends onthe relative horizontal and vertical amplitudes, different an-gles in the x-y plane are tested. In this study we expect animportant effect since the x-y symmetry is broken in somecases, we use angular steps of 5 degrees, i.e. 17 angles inthe x-y plane. For each set of amplitude and angle a hor-izontal and vertical tune is assigned and the particles aretracked through the elements of the machine.Magnetic field errors The magnetic field errors in thefinal focusing quadrupoles were set up according to thespecifications in [11]. A correction algorithm was ap-plied to minimize their impact, assuming the errors havebeen measured and are known. Typically 20 different sets(seeds) of the field errors are used for each case.Scan of working point In order to find the best work-ing points, the particles are tracking for different tune val-ues. The nominal working point of the LHC in collisionis 64.31 and 59.32 in the horizontal and vertical planes.For the tune scan the horizontal tune was varied in stepsof 0.001. The tune difference of 0.01 between the hori-zontal and vertical tune was maintained, i.e. the scan wasperformed parallel to the diagonal in the working diagram.MAIN RESULTSThe minimum dynamic aperture in units of the beam sizeσ as a function of the horizontal tune and for different an-gles in the x-y plane are shown in Figs. 1 and 2. In Fig. 1we have used an alternating crossing scheme while in Fig. 2we show the results when the beams cross always in thehorizontal plane. The inherent symmetry in the alternatingcrossing scheme shows a weak dependence on the angle.The largest aperture can be found around the design work-ing point. In the non-alternating case the angular depen-dence is very strong with a significantly larger aperture inthe plane orthogonal to the crossing plane. Considering allangles, the available tune space is however smaller than inthe alternating case. A similar behaviour can be observedin Figs. 3 and 4 where we show the results of the tune scanfor PACMAN bunches.SUMMARY• We have set up an environment for large scale accel-erator physics studies.Figure 1: Minimum dynamic aperture for horizontal-vertical crossingFigure 2: Minimum dynamic aperture for horizontal-horizontal crossing• The foreseen alternating crossing schemes is superiorto a crossing in the same plane.ACKNOWLEDGEMENTSWe should like to thank all the volunteers who havemade their computers available to provide the necessarycomputing resources. Without their help this study wouldnot have been possible.REFERENCES[1] W. Herr, “Dynamic behaviour of nominal and PACMANbunches for different LHC crossing schemes”, LHC ProjectNote 856, CERN (2005).[2] E. McIntosh and A. Wagner, “CERN Modular PhysicsScreensaver or Using spare CPU cycles of CERN’s Desk-top PCs”, CHEP 2004, Oct. 2004, Interlaken, p. 1055.[3] “LHC@Home”, http://lhcathome.cern.ch/.Figure 3: Minimum dynamic aperture for PACMANbunches for horizontal-vertical crossingFigure 4: Minimum dynamic aperture for PACMANbunches for horizontal-horizontal crossing[4] “BOINC: Berkeley Open Infrastructure for Network Com-puting”, http://boinc.berkeley.edu/.[5] F. Schmidt; SixTrack: Version 3, Single particle trackingcode treating transverse motion with synchrotron oscilla-tions in a symplectic manner.[6] Y. Luo and F. Schmidt; Weak-Strong Beam-Beam Trackingfor LHC V6.0 , Proc. Workshop on Beam-Beam Effects, Fer-milab, June 2001.[7] M. Hayes, E. McIntosh and F. Schmidt, “The Influence ofComputer Errors on Dynamic Aperture results Using Six-Track”, LHC Project Note 309, CERN (2003).[8] E. McIntosh and F. Schmidt; A new run environment for Six-Track.[9] CRlibm, a library of correctly rounded elementaryfunctions in double-precision. http://lipforge.ens-lyon.fr/www/crlibm/.[10] LHC beam-beam studies, http://cern.ch/lhc-beam-beam/[11] LHC Design Report, CERN-2004-003, (2004).

Large Scale Beam-beam Simulations for the CERN LHC using Distributed Computing

Abstract

Similar works

Full text

Available Versions

CERN Document Server