The interconnections between the Large Hadron Collider (LHC) magnets have required some 40 000 TIG welded joints and 65 000 electrical splices. At the level of single joints and splices, non-destructive techniques find limited application: quality control is based on the qualification of the process and of operators, on the recording of production parameters and on production samples. Visual inspection and process audits were the main techniques used. At the level of an extended chain of joints and splices - from a 53.5 m half-cell to a complete 2.7 km arc sector - quality control is based on vacuum leak tests, electrical tests and RF microwave reflectometry that progressively validated the work performed. Subsequent pressure tests, cryogenic circuits flushing with high pressure helium and cool-downs revealed a few unseen or new defects. This paper presents an overview of the quality control techniques used, seeking lessons applicable to similar large, complex projects

Bertinelli, F

Borowiec, P

Bozzini, D

Cruikshank, P

Fessia, P

Grimaud, A

Kotarba, A

Maan, W

Olek, S

Poncet, A

Russenschuck, Stephan

Savary, F

Sulek, Z

Tock, J P

Tommasini, D

Vaudaux, L

Williams, L

CERN Document Server

LHC-PROJECT-REPORT-113120Aug20081 CERN, Accelerator Technology Department, Geneva, Switzerland2 HNINP, Krakow, Poland3 ALL43, St-Genis-Pouilly, France4 IEG, St-Genis-Pouilly, FranceEUROPEAN ORGANIZATION FOR NUCLEAR RESEARCHEuropean Laboratory for Particle PhysicsTHE QUALITY CONTROLOF THE LHC CONTINUOUS CRYOSTAT INTERCONNECTIONSF. Bertinelli1, P. Borowiec2, D. Bozzini1, P. Cruikshank1, P. Fessia1, A. Grimaud3 ,A. Kotarba2, W. Maan1, S. Olek2, A. Poncet1, S. Russenschuck1, F. Savary1, Z. Sulek2,J.P. Tock1, D. Tommasini1, L. Vaudaux4, L. Williams1The interconnections between the Large Hadron Collider (LHC) magnets have required some 40 000 TIG weldedjoints and 65 000 electrical splices. At the level of single joints and splices, non-destructive techniques find limitedapplication: quality control is based on the qualification of the process and of operators, on the recording ofproduction parameters and on production samples. Visual inspection and process audits were the main techniquesused. At the level of an extended chain of joints and splices - from a 53.5 m half-cell to a complete 2.7 km arcsector - quality control is based on vacuum leak tests, electrical tests and RF microwave reflectometry thatprogressively validated the work performed. Subsequent pressure tests, cryogenic circuits flushing with highpressure helium and cool-downs revealed a few unseen or new defects. This paper presents an overview of thequality control techniques used, seeking lessons applicable to similar large, complex projects.Presented at the 11th European Particle Accelerator Conference (EPAC'08)23-27 June 2008, Genoa, ItalyGeneva, Large Hadron Collider ProjectCERNCH - 1211 Geneva 23SwitzerlandLHC Project Report 1131Abstract20 August 2008THE QUALITY CONTROL OF THE LHC  CONTINUOUS CRYOSTAT INTERCONNECTIONS F. Bertinelli, D. Bozzini, P. Cruikshank, P. Fessia, W. Maan, A. Poncet, S. Russenschuck, F. Savary, J.P. Tock, D. Tommasini, L. Williams, CERN, Geneva, Switzerland P. Borowiec, A. Kotarba, S. Olek, Z. Sulek, HNINP, Krakow, Poland A. Grimaud, ALL43, St-Genis-Pouilly, France, L. Vaudaux, IEG, St-Genis-Pouilly, France.Abstract The interconnections between the Large Hadron Collider (LHC) magnets have required some 40 000 TIG welded joints and 65 000 electrical splices. At the level of single joints and splices, non-destructive techniques find limited application: quality control is based on the qualification of the process and of operators, on the recording of production parameters and on production samples. Visual inspection and process audits were the main techniques used. At the level of an extended chain of joints and splices – from a 53.5 m half-cell to a complete 2.7 km arc sector – quality control is based on vacuum leak tests, electrical tests and RF microwave reflectometry that progressively validated the work performed. Subsequent pressure tests, cryogenic circuits flushing with high pressure helium and cool-downs revealed a few unseen or new defects. This paper presents an overview of the quality control techniques used, seeking lessons applicable to similar large, complex projects. INTRODUCTION The Large Hadron Collider (LHC), its installation and the interconnection work between magnets have been extensively described at various stages of the project [1]. The project management aspects of the interconnection of the arc continuous cryostat are described elsewhere [2]: this paper specifically presents the quality control aspects. Three main joining technologies are involved in interconnection work: induction soldering for ~ 10 000 13 kA splices, ultrasonic welding for ~ 53 000 600 A splices and TIG welding for ~ 37 000 joints. Other activities involve essentially assembly.  QA STRATEGY AND ORGANISATION Interconnections (IC) in the continuous cryostat have been designed to be performed using mechanical/automatic processes. Indeed the main contractor IEG employed “operators” for TIG welding, not “welders”. Moreover, the geometry of these interconnections implies that non-destructive testing methods find only limited application. Consequently the Quality Assurance (QA) strategy is based on four concepts: - Prior qualification of the processes, the equipment and the operators on samples; - On-line monitoring and recording of critical parameters of all junctions; - Strict application of the qualified procedures and full traceability; - Production samples that can be tested off-line using destructive methods. The human resources directly involved in the quality control (QC) of interconnection work in the period January 2005 to June 2008 represent ~ 120 man-years, or ~ 38% of the total workload. At peak, in the period January to June 2007, ~ 72 persons were involved simultaneously in QC. The relative contribution of the different teams is shown in Fig. 1: - IEG: operators were responsible for the first level of control, followed by their team leaders and dedicated inspectors. - ICIT: Project Associates under a CERN-HNINP collaboration agreement, who were trained to perform visual inspection work. - AT-MCS: the organisation of the quality control was ensured by a team of CERN staff from AT-MCS, together with a few technicians from the Institut de Soudure. - VAC: responsible for the vacuum performance and leak testing, CERN staff from AT-VAC together with the contractor ALL43. - ELQA: responsible for the electrical testing, CERN staff from AT-MEL together with Project Associates from the same CERN-HNINP collaboration.  AT-MCS19%ICIT28%IEG Inspectors10%ELQA21%VAC22% Figure 1: Teams involved in Interconnection Quality Control at peak activity, January-June 2007. QC EXPERIENCE & LESSONS LEARNT “Organise the QC structure early!” The first IC work on the continuous cryostat was performed in May 2005. Delays originating from other LHC project areas implied that the intense IC activity only started later in January 2006, up to November 2007: IC activity in later months was associated to critical consolidation work following hardware commissioning. 1Project management pressure was on IC work to progress as fast as possible. New team members were introduced monthly. The turnover of operator staff was inevitably high. The international nature of teams often introduced language communication difficulties. Consequently, once the intense project pressure was on, the QC organisation required continuous and rapid adaptation, see Fig. 2. For example, the CERN AT-MCS team set up a Hotline Call Centre for the reporting and management of non-conformities in September 2006. However the structure of QC needs to be operational and tested early enough in order to be ready to “ride the storm” when it comes!  Interconnection work - continuous cryostat:Quality Control resources and NCR opened0102030405060708090100Jan-05Apr-05Jul-05Oct-05Jan-06Apr-06Jul-06Oct-06Jan-07Apr-07Jul-07Oct-07Jan-08Apr-08TimeQC manpower0100200300400500600700Number of NCR opened per monthQuality Control PersonnelNCR opened per month Figure 2: Evolution of QC personnel and number of NCR opened per month with time Traceability of work Traceability of work (operator, equipment, date) was essentially based on daily, manual reports recorded onto Excel files or the Manufacturing and Test Folder (MTF) computer tool developed by CERN. While powerful database applications indeed provide endless analysis possibilities, it was the simpler software tools that were favoured for widest usage, flexible analysis and urgent decision-making: for example, identifying the work performed in a certain time-period by a specific operator. Traceability has value in the details! For example, the use of 7 different ultrasonic machines was traced, but experience showed this needed to extend further to their sonotrode tooling and maintenance. Also, while records of joining operations are available, subsequent minor interventions and repairs were not, but should have been, systematically monitored. An extensive usage of activity recording through barcode reading (activity, operator, tooling, components) was not implemented: this is an example of a structural tool discussed in September 2005 that came too late to be implemented and tested in time, and could have played an important role in both project management and quality issues. Monitoring and recording of parameters The equipment for all three joining technologies included the facility to monitor and record the main parameters throughout each joining operation. In practice this proved a mixed success with respect to expectations. A large percentage of recorded data - up to ~ 60-70% - from induction soldering and TIG welding was lost in acquisition, probably due to the fragility to high-frequency interference of the flash card storage support used. Also, the expectation of systematically identifying joint defects from the recordings was not met, since the recorded sampling rate was too slow: for example TIG data was recorded every 100 of rotation, see Fig. 3.   Figure 3: Example of TIG recording data: this case illustrates a poor positioning of the electrode height.  However, the recording of production parameters remains important to avoid large errors (wrong production parameters used, repeated mistakes by operators).  Production samples and process auditing Production samples were used particularly for electrical splices. Sample loops were performed to qualify each machine and new operators: the loops were then tested off-line, for example for their mechanical strength and their electrical contact resistance at 4.2K.  Samples were taken as routine production checks: for example 460 loops from ultrasonic welding - 40 loop samples per month - corresponding to ~ 1% of the splices made, see Fig. 4.  Production Samples: contact resistance,  ultrasonic welding of M spools0510152025303540 7-8  4-5  8-1  3-4  5-6  2-3  6-7  1-2SectorNumber off0 - 2 nΩ2 - 3 nΩ3 - 4 nΩ4 - 5 nΩabove 5 nΩ Figure 4: production samples from ultrasonic welding  Associated with production sampling is the auditing of the process and the operator. Auditing, i.e. the control of the process (as distinct from inspecting - the control of the result) took major importance starting January 2007: it was implemented by dedicated IEG and CERN AT-MCS staff. This allowed faster improvement of quality issues and was highly appreciated by operators also as a means of information and feedback. The constant presence of CERN staff in the tunnel at the workplace was essential. 2Visual inspections and non-conformities Following transport and alignment of magnets, the first pre-inspection was performed by ICIT. Initially limited to the reporting of non-conformities, the role of ICIT was increased to include some immediate corrective actions, in particular over repetitive issues (removal of dirt particles, repair of Multi Layer Insulation). A critical role of ICIT was the visual inspection and endoscopy of the beam lines together with the necessary interventions in clean conditions to remove the objects found, for example thin plastic shavings residues from beam screen packing material. Once CERN released an interconnection for work, IEG performed its preliminary inspection in order to take responsibility for its subsequent work. Small repairs were handled by IEG directly as internal non-conformities: ~ 1 800 such cases are estimated to have been treated, with limited formal traceability. After having passed final inspection by IEG, the finished joint was inspected again by ICIT and if necessary by CERN AT-MCS.  ICIT IEG CERN AT-MCS ELQA VAC Total1970(859 dust, 507 MLI)PIMs 35 35 40 110Busbars (13 kA) 7 155 23 185US spools (600 A) 9 194 17 8 228TIG welds 75 289 41 7 4121067(381 welds, 300 shocks)Line-N 2 30 10 4 46Cryo instrum. 3 3 1 7Final inspection 3 148 150 11 92 404W closure 26 4 30HWC 10 5 15Total 3168 981 307 32 102 4590206954 4 2 1084Pre-inspectionInspection after work947 Table 1: Number of Non-conformity Reports (NCR) opened per activity and QC Team  The particular lip joint geometry implies that radiography was ineffective as a QC technique for TIG welds: ICIT performed 100% visual inspection of welds. The 600 A electrical splices for the line-N were 100% visually inspected by CERN AT-MCS. However, because of the tightness of the workflow organisation, the 13 kA splices were 100% visually inspected by IEG and immediately insulated, making visual inspection by ICIT impossible. CERN finally introduced a non-destructive ultrasonic technique to control the 13 kA splices of the last ~ 15% splices made [3]: the positive “psychological” effect of this control on operators was noteworthy. The CERN AT-MCS team who followed non-conformities handled ~ 4 500 cases, see Table 1. Tests on chains of interconnected magnets The workflow was organised to interconnect adjacent magnets into small chains that could be tested as complete circuits. The ends of these chains were interconnected after successful completion of the tests. ICIT was responsible to perform microwave reflectometry tests - and if necessary endoscopy - on the beam lines, on chains of 16 magnets. The vacuum leak testing identified ~ 380 “defects”, see Fig. 5: typically an IC defect level from TIG welding of ~ 0.2% was experienced. Contrary to initial predictions, a large proportion originated from “imported leaks”, i.e. from components previously tested on the surface (flexible hoses, base material of flanges). Also interesting were a few material leaks caused by pollution of the base metal by silver or tin residues from previous soldering.  Fig 5(a): IC leaks identified0.2%3.3%0.9%0.5% 0.4%1.6%050100150200250C'/k X E M & N Extern. W VType of connectionNumber off IC leaks Imported leaksFig. 5(b): IC leaks size0102030405060701.E-01 1.E-02 1.E-03 1.E-04 1.E-05 1.E-06 1.E-07 1.E-08 1.E-09 1.E-10Leak size (mbar l s-1)Number offC'/k XE M & NExtern. W V Figure 5: (a) “defect” level for leaks and (b) leak size  The electrical quality assurance, ELQA, identified ~ 180 “defects” [4]. Typically an IC defect level of ~ 0.1% was experienced. Once a defect was detected in an IC chain, the subsequent step was its precise localisation for repair: this often involved a time-consuming, large effort implying more specific competences and resources. Extensive testing performed after the pressure test of the completed 3.3 km sectors, after helium flushing and after cool-down still identified remaining defects, at the level of ~ 5-10 cases, in particular electrical problems associated to insulation damage or polluting metal residues. CONCLUSIONS The quality control effort and resources employed on the LHC interconnections have been considerable and their effectiveness in progressively identifying defects has been described. Experience so far suggests that a correct compromise between quality control costs and results has been achieved. REFERENCES [1] J. Ph. Tock et. al., “Interconnections of the LHC Cryomagnets at CERN: Strategy Applied and First Results of the Industrialization Process”, IEEE Transactions on Applied Superconductivity, MT20 Conference, Philadelphia, USA, 27-31 August, 2007. [2] P. Fessia et. al., “The LHC Continuous Cryostat Interconnections: the Organisation of a Logistically Complex Worksite Requiring Strict Quality Standards and High Output”, EPAC’08 Conference, 23-27 June, 2008, Genova, Italy. [3] F. Caspers et. al. “Ultrasound Diagnostics of the Superconducting Cable Connections between the Main Ring Magnets of LHC”, PAC07 Conference, June 25-29, 2007, Albuquerque, USA. [4] S. Russenschuck et al., “Electrical Quality Assurance of the Superconducting Circuits during LHC Machine Assembly”, EPAC’08 Conference, 23-27 June, 2008, Genova, Italy.  3

The Quality Control of the LHC Continuous Cryostat Interconnections

The Quality Control of the LHC Continuous Cryostat Interconnections

Abstract

Similar works

Full text

Available Versions

CERN Document Server