CERN uses several liquid helium cryoplants (total of 21) for cooling large variety of superconducting devices namely: accelerating cavities, magnets for accelerators and particle detectors. The cryoplants are remotely operated from several control rooms using industrial standard supervision systems, which allows the instant display of all plant data and the trends, over several days, for the most important signals. The monitoring of the cryoplant performance during transient conditions and normal operation over several months asks for a long-term recording of all plant parameters. An historical data recording system has been developed, which collects data from all cryoplants, stores them in a centralized database over a period of one year and allows an user-friendly graphical visualization. In particular, a novel tool was developed for debugging causes of plant failures by comparing selected reference data with the simultaneous evolution of all plant data. The paper describes the new system, already in operation with 11 cryoplants

Barbeau, H

Barth, K

Baud, R

Dauvergne, J P

Delikaris, D

CERN Document Server

EUROPEAN ORGANIZATION FOR NUCLEAR RESEARCHLaboratory for Particle Physics(ECR)New Long-term Historical Data Recording and Failure Analysis system for theCERN CryoplantsH. Barbeau*, K. Barth, R. Baud, J.-P. Dauvergne,D. DelikarisCERN uses several liquid helium cryoplants (total of 21) for cooling large variety ofsuperconducting devices namely: accelerating cavities, magnets for accelerators and particledetectors. The cryoplants are remotely operated from several control rooms using industrialstandard supervision systems, which allows the instant display of all plant data and the trends,over several days, for the most important signals. The monitoring of the cryoplant performanceduring transient conditions and normal operation over several months asks for a long-termrecording of all plant parameters. An historical data recording system has been developed,which collects data from all cryoplants, stores them in a centralized database over a period ofone year and allows an user-friendly graphical visualization. In particular, a novel tool wasdeveloped for debugging causes of plant failures by comparing selected reference data with thesimultaneous evolution of all plant data. The paper describes the new system, already inoperation with 11 cryoplants.*Institut Universitaire de Technologie de Belfort, 90 016 Belfort,  FranceCERN LHC/99-520 th International Congress of Refrigeration (ICR99)19 - 24  September, 1999, Sydney, AustraliaGeneva, Switzerland 18 October 1999Divisional ReportAdministrative SecretariatLHC DivisionCERNCH - 1211 Geneva 231NEW LONG-TERM HISTORICAL DATA RECORDING AND FAILURE ANALYSISSYSTEM FOR THE CERN CRYOPLANTSK. BARTH(1) , R. BAUD(1), J.-P. DAUVERGNE(1), D. DELIKARIS (1), H. BARBEAU(2)(1) CERN, LHC Division, CH-1211 Geneva 23, Switzerland(2) Institut Universitaire de Technologie de Belfort, 90016 Belfort CEDEX, FranceABSTRACTCERN uses several liquid helium cryoplants (total of 21) for cooling large variety ofsuperconducting devices namely: accelerating cavities, magnets for accelerators and particledetectors. The cryoplants are remotely operated from several control rooms using industrialstandard supervision systems, which allows the instant display of all plant data and the trends, overseveral days, for the most important signals. The monitoring of the cryoplant performance duringtransient conditions and normal operation over several months asks for a long-term recording of allplant parameters. An historical data recording system has been developed, which collects data fromall cryoplants, stores them in a centralized database over a period of one year and allows an user-friendly graphical visualization. In particular, a novel tool was developed for debugging causes ofplant failures by comparing selected reference data with the simultaneous evolution of all plantdata. The paper describes the new system, already in operation with 11 cryoplants.1. INTRODUCTIONAt CERN in total 21 Helium cryoplants are operated, ranging from 400 W to 18 kW coolingpower at 4.5 K entropy equivalent. Six refrigerators (4 x 18 kW at 4.5 K and 2 x 400 Watt at 4.5 K)are used to cool the superconducting cavities and magnets of the Large Electron Positron collider(LEP) and Super Proton Synchrotron (SPS). The other 15 Helium liquefiers supply superconductingmagnets of various LEP and SPS experiments [1] as well as large cryogenic test facilities for thenew Large Hadron Collider (LHC) [2]. All installations are fully automated and remote controlledfrom dedicated control rooms (in total 5). Table 1 shows an overview of the presently operatedHelium cryogenics for experiments and test facilities. For each plant the number of analog anddigital process signals to be controlled and monitored is given.Table 1: Overview of presently operated Helium cryogenics for experiments and test facilitiesUSER Helium Plant Analog I/O's Digital I/O'sLEP, PA4(ALEPH solenoid, quadrupoles)Sulzer TCF200 + Linde Purifier + Bauer recovery compressor + 3000 m3 He pure & impure storage 230 580LEP, PA8 (DELPHI solenoid, quadrupoles)Sulzer TCF200 + Linde Purifier + Bauer recovery compressor + 3000 m3 He pure & impure storage 240 640NA 49.1 (Vertex) CNA 100 140 285NA49.2 (Vertex) CNA 100 140 285NA-ATLAS/H8 CNA 100 140 285NA-CMS/RD5 CNA 100 120 260NA-COMPASS CNA 100 120 260NA-LHC/Bl.4 Test Facility CNA 100 130 320SM18 LHC Test Facility 6 kW Air Liquide 530 650SM18 LHC Test Facility 6 kW Linde 300 540Cryolab LHC Cable Test Facility Sulzer TCF50 350 660Cryolab Dewar Filling Station Sulzer TCF50 150 280SM18 He Gas Recovery 2 Linde purifers+3 Sulzer Burckhardt recovery compressors +15000 m3 He pure & impure storage 70 400Cryolab & NA Gas Recovery 2 Linde purifers + 3 Sulzer Burckhardt recovery compressors + 35000 m3 He pure & impure storage 140 360Total: 2800 58052For the control and short term monitoring of all the cryoplants in operation a industrial standardsupervisory system from ASEA Brown Boveri (ABB) [3] is used. The normal operation of thecryogenic installation during a continuous period of 9-11 month, with relatively slow (over 2-3weeks) transient conditions, demands for a long term historical data acquisition system. Archivingseveral 100 megabytes of process data, which are accumulated from each installation listed in table1 over 12 month and keeping it in one centralized database, exceeds the capacity of the ABBsupervisory system. Therefore a special adapted data acquisition system was developed, based ontwo DEC workstations running a self-written software package. The DEC system reads out every 8-15 minutes all process data, event and alarm lists from the various decentralized ABB controlsystems, stores  the information in one central database and  provides two user-friendly operatorinterfaces to display the data. In addition to a World Wide WEB interface, a special tool for failureanalysis was developed to allow backward and forward display of all data by the same animatedgraphical representation used for monitoring of the normal operation.2. ORGANISATION OF THE CRYO CONTROL AND DATA ACQUISITION SYSTEM2.1 The cryogenic process control and supervisory systemThe ABB supervisory system contains three standardized components to provide a de-centralizedcontrol of the cryogenic plants:The so called Master Piece unit (MP) connect the system to the real process by exchangingdigital and analog I/O signals and stores them in its process database. It contains also completeprogrammable logic controllers (PLC's) for the control of machines and processes. The MPexecutes the control logic with cycle times down to 20 msec and stores the last 200 process alarmsor events in a dedicated memory area. The MP contains also 4 storage files (logs), each with roomfor 10 analog process variables, to do the short-term trending of the most important plantparameters (max. time period: 240 h, min. sample rate: 15 sec)[4].The Master View unit (MV) is responsible for the man-machine communication using colorgraphic video display terminals and printers. The MV station displays the cryoplant status andallows manual control of the installation, using animated process (max. 86), object (max. 2600) andtrend displays (max. 20). It provides also the alarm- and event list handling from all the MPsconnected to it.[3]The Master View station communicates with it's subordinated Master Piece unit(s) over amultidrop, master-slave bus called Master Bus 200 (MB). The max. communication speed of thisasynchronous bus, complying with the RS 422 standard, is 153.5 kbit/sec. Figure 1 shows a typicalexample of  such a widely distributed system structure, used to control and monitor the cryogenicinstallations of the North Area and LEP section from one dedicated cryo control room. There are intotal 3 MV units, each configured as master. The first MV station is assigned to the North Areaexperiment cryogenics and controls via 9 MP units, all connected in parallel and configured asslaves, the compressors, cold boxes and infrastructure of the cryogenics for SPS fixed-targetexperiments. A second MV unit is dedicated for the cryogenics of the two LEP experimentsALEPH (Apparatus for Lepton Physics) and DELPHI (Detector with Lepton, Photon and HadronIdentification). Both installations are located far away from the cryo control room (ALEPH 15 km,DELPHI 10 km) and therefore apart from the two MP units a second MV station situated in a smalllocal control room has been added to each network. The local control room is used only duringmaintenance and all normal operation is done via the MV station in the main control room. Toprovide reliable and fast communication, the LEP MV unit is connected using a fiber optical link tothe two local Master Bus networks. A third MV station, connected to two MP units, controls aliquefier, supplying the LHC short magnet test facility. This network as well as the one dedicated tothe North Area experiment cryogenics uses analog lines and where the distances between twoMaster Bus nodes are greater then 30 m, standard modem links (RS232C) are used.3Master PieceZN Comp. 1-4Master PieceZN Comp. 5-7Master PieceZN InfrastructureMaster PieceZN CB CMS/RD5Master PieceZN CB COMPASSMaster PieceZN CB ATLAS/H8Master PieceZN CB NA49.1Master PieceZN CB NA49.2Master PieceZN Comp. LHC/Block4Master PieceZN CB LHC/Block4ABBMASTER BUSABBMASTER BUSMaster View 850ZN Cryo Control RoomZNMaster View 820ZN Cryo Control RoomLHC/Block4Master PieceLEP1 PA4 SurfaceMaster PieceLEP1 PA4 UndergroundMaster PieceLEP1 PA8 SurfaceMaster PieceLEP1 PA8 UndergroundMaster View 820PA4 Cryo Control RoomMaster View 820PA8 Cryo Control RoomABBMASTER BUSABBMASTER BUSTerminalPA4 Cryo Control RoomTerminalPA4 Cryo Control RoomTDM TDMMaster View 850ZN Cryo Control RoomLEP100TDMTDMSignalTDMTerminal ZNNA Cryo Control RoomTerminal LHC/Bl.4NA Cryo Control RoomTerminal LHC/Bl.4NA Cryo Control RoomRouterRS232DEC Server 90 TLDEC Workstation 2ZN Cryo Control RoomDEC Workstation 1ZN Cryo Control RoomDisk storagePrinterCERN WideNetworkCryo DataAccquisitionCryoWEB ServerApplication"Cryo_Offline"North Area / LEP100 Control &Data Accquisition SystemTCP/IPTCP/IPFigure 1: Structure of the North Area and LEP control and data acquisition system2.2 The long-term historical data acquisition systemThe control and supervisory system, as described above, covers well all short term monitoring ofthe cryogenic installation needed to control it. In addition a second system was developed, whichreads out all the various process signals and alarm/event lists of the ABB MP units. The data is timetagged and stored with a repetition rate of 8-15 minutes in one central database. This allows to keepa trace of all analog and digital process information from the plants and allows performancemonitoring over periods of up to one year.Figure 1 shows in detail, how the long term data acquisition system is connected to the NorthArea and LEP ABB control system. Each ABB MP unit stores all analog and digital signal valuesfrom the sub-processes, controlled by it, in a local process database. The database of each MP isread out from a self programmed data acquisition software running on a DEC workstation. The datais transferred from the ABB MP units to the DEC workstation via a special ABB hardware portcalled ASEA Master EXCOM[5]. The EXCOM module is installed in the MV station and allowsthe external computer to read out indirectly all process databases of the MP units connected to itsABB Master Bus 200. To avoid additional cabling between the MV stations in the different controlrooms and the DEC workstation, a Ethernet communications server[6] is installed in each localzone. This device provides a routing between the asynchronous serial link port and a thin wireEthernet, thus allowing the transfer of the various MP process data files using the CERN LocalArea network (LAN). Figure 2 illustrates how all the different control rooms are connected via theCERN LAN to the DEC workstation located in the North Area & LEP control room. The centraldatabase, which is continuously updated by the data acquisition program is kept on a networkaccessible hard disk (13 Gbytes), providing sufficient space to keep data accumulated by all plantsduring one year of operation.4Router ABBMASTER BUSDEC Server 90 TLDEC Workstation 2ZN Cryo Control RoomDEC Workstation 1ZN Cryo Control RoomDisk storagePrinterCERN WideNetworkCryo DataAccquisitionCryoWEB ServerApplication"Cryo_Offline"TCP/IPTCP/IPNA & LEP100Cryo ABBControl SystemDEC Server 90 TL DEC Server 90 TLDEC Server 90 TL Gateway/RouterSM18 CryoABBControl SystemWest Area CryoABBControl SystemCryolab CryoABBControl SystemLHC Exp. CryoControl System???TCP/IPTCP/IPTCP/IP TCP/IPRS232RS232???BUSRS232Figure 2: Detailed structure of the long term historical data acquisition system3. INTERFACES TO THE LONG TERM HISTORICAL DATA ACCQUISITION SYSTEMThe enormous amount of process information stored in the central database, demanded also for auser-friendly operator access to the data for analyzing it off-line. Two interfaces have beenestablished to display and analyze the process data.3.1 The WEB interfaceThe WEB server has been set-up on a second DEC workstation. The software package extractscontinuously process data from the central database and presents it in form of hypertext  documents(HTML). Figure 3 shows the homepage of the cryogenics for experiments and test facilities WEBserver (http://afcryo02.cern.ch), w ich is presenting an overview with the main status informationfor all cryoplants in operation. All animated WEB pages are updated automatically every 15minutes with the latest process information of the central process database.  By clicking on the texthyperlinks, the user has access to all analog and digital process signals as well as alarm and eventlists of the installation for any period in the past 12 month. Every signal can be exported into astandard ASCII text file, which permits to produce reference curves for all important plantparameters and allows the long-term monitoring of the plant performance.The WEB interface is used to provide the different users of the cryogenics with the latest statusinformation, since any computer equipped with a Internet access and a WEB browser has access toit. In case of plant failures outside normal working hours, the responsible operation engineer canalso perform a first diagnosis from home. This allows him to judge, if it is necessary to come in andsupport the on-call operator, who is alarmed automatically via his mobile telephone using anindependent working Alarm Notification System (ANS).5Figure 3: Homepage of the cryogenics operation section for experiments and test facilities A special WEB page has been set up, containing level, pressure and safety information of allliquid nitrogen storage tanks (19 in total with capacities between 6000 - 50000 l) at the variousCERN sites. With it, a single person can manage and plan the liquid nitrogen distribution using aspecial contract set-up with the suppliers.3.2 The "CRYO-OFFLINE" interfaceIn case of any cryoplant malfunction a detailed off-line analysis is performed to find the origin ofthe problem and to avoid it in the future. For an exact problem debugging, the time evolution ofseveral plant parameter has to be displayed in parallel. Using the WEB interface, this can be onlydone by exporting all the necessary data and displaying them in multi-graphs, using a standardgraphical software package (for example MS-EXCEL). The disadvantage of this method is thelimited number of curves, which can be displayed in one graph without loosing the overview.Therefore a second access interface to the central process database was developed, called "CRYO-OFFLINE" [7]. The software was programmed using a high level programming language (GNU C)and a graphical modeling system (SL/GMS) available for the DEC platform.6Figure 4: Animated graphical representation of the LEP PA8 cold box in "cryo-offline"The tool allows to display time evolution of the signals with the same animated graphicalrepresentation, used in the ABB control system. Figure 4 shows a hardcopy of such an animatedprocess diagram. Instead of displaying on-line signals, data from the long-term data acquisitionsystem for a user-defined time period is used and can be browsed forwards or backwards in time.For all signals displayed in the mimic, also a curve over the pre-defined time period can bedisplayed. This form of displaying the information has the advantage that all relevant parameterscan be shown on one screen, while at the same time the overview over their relation to the processis kept.4. CONCLUSIONS AND OUTLOOKThe industrial standard ABB control and supervisory systems, presently used at CERN forcryogenics, covers very well the needs for short-term monitoring of process information. It has beencomplemented by a self-made data acquisition system monitoring long time periods of up to 1 year,due to the demand of the specific applications of cryogenics at CERN. Nevertheless such a long-term data monitoring could be also interesting in other fields of process control.The system has been tested and is now under full operation for more then 2 years. Especiallytogether with the two user-interfaces, it has been shown, that it is a powerful tool for problemdebugging and plant performance analysis. The system is presently extended to include all the newcryogenic installations, which have been installed to supply various test facilities for the LHC.75. REFERENCES1. Dauvergne J.P. et al., 1996, Cryogenics For CERN Experiments. Past, Present And Future,ICEC 16, IoP, p. 326 - 3322. Barth K., Delruelle N. et al., 1998, New Cryogenic Facilities For Testing SuperconductingEquipments For The CERN Large Hadron Collider, ICEC 17, IoP, p. 623 - 6263. ASEA Electronics, ASEA Master View Buyer's Guide, Ref. 7650 046-201/24. ASEA Electronics, ASEA Master Piece 200 Users Manual, Ref. 7650 039-202/05. ASEA Electronics, ASEA Master-EXCOM Users Manual, Ref. 7650 056-2016. DIGITAL Equipment Corporation, DECserver 90TL Owner's Manual, Ref.: EK-DSRVE-OM7. Barbeau H., 1998, Consultation "Offline" Des Synoptiques Et Des Graphiques Des InstallationsCryogeniques, Rapport de stage, IUT Belfort - Départment Informatique

New Long-term Historical Data Recording and Failure Analysis System for the CERN Cryoplants

Abstract

Similar works

Full text

Available Versions

CERN Document Server