Jelezov, A

Ressing, D

Rybnikov, V

Sánchez, F

Uwer, U

Zweizig, J

CERN Document Server

THE HERA-B SLOW AND RUN CONTROL SYSTEMV.Rybnikov (DESY), A.Jelezov (ITEP Moscow), D.Ressing (DESY),F.Sanchez (DESY), J.Zweizig (DESY, now in CALTECH),U.Uwer (Humboldt University Berlin)1 HERA-B EXPERIMENTThe HERA-B experiment is dedicated to measure CPviolation in the B meson system. It is built up on the HERAproton beam and uses an internal wire target.About 300 collaborators from 33 institutes participate atthe experiment.The detector consists of 10 sub-detectors providing theraw data, 16 subsystems are integrated in the DAQ systemfor building and processing events.The final number of readout channels is more then 550K.The main detector components are the Vertex Detector, In-ner and Outer Trackers, RICH, Electro-magnetic Calorime-ter and Muon Chambers.The concept of the Slow and Run control systems playsan important role for the experiment.The aim of the paper is to present the both systems, theirbasic parts, implementation, the status and plans.2 BASIC PARTS OF SLOW AND RUNCONTROL SYSTEMSThe Hera-B Slow and Run Control is a highly distribut-ed system. Estimates for the final system give about 2300processes running on 1600 different computing nodes em-ploying different operating systems such as IRIX, Solaris,Linux, LynxOS. About 1200 processors are SHARC DSPs(Digital Signal Processors) that have no operating systemat all.The basic parts to build up transparent, easy maintainedand scalable systems are: message passing system; state machine control; central services;The software design approach was also considered as anessential part for the implementation of the systems .2.1 Message Passing SystemInterprocess communication is a primary question in themodern systems. Our decision in this area was to have asingle integrated message passing interface for all the mes-saging needs of the HERA-B online systems. It’s imple-mented on all variety of UNIX platforms as well as onplatforms with no operating systems, namely the SHARCDSPs. The main features of the HERA-B message passingsystem are: UDP socket based interface; a simple naming scheme for associating each processoffering a service with a name to look up address ofthe service by using the name; two interface level options (high and low) to satisfyquite diverse requirements with 4 orders of magnituderange in latency requirements and 5 orders of magni-tude range in message length; a translation service which rearranges byte ordering,changes byte alignment and translates floating pointformats, that is hidden from the user;2.2 State MachineThe integration and synchronization of the subsystem-s within the common Run Control (RC) and Slow Control(SC) systems is the next basic point. These tasks are solvedby applying so called state machine and arranging the allprocesses in a process tree, where the state transition com-mands propagate from the top to the bottom, and the statesstream has the opposite direction. The main features of theHERA-B State Machine Control (SMC) package are: dynamic process configuration during its start-up– dynamic state transition table (STT);– dynamic state dependence table (SDT);– dynamic number of daughter processes; the possibility to have processes with diverse STT inone process tree; user defined actions for state transition.2.3 Central servicesThe major central services used by any process of theHERA-B Run and Slow Control systems are: name service (process registration, process informa-tion provision)International Conference on Accelerator and Large Experimental Physics Control Systems630International Conference  Accel rator and L rge Experim ntal Physics Control Systems, 1999, Trieste, Italy process service (remote start and process termination) error logging (error message distribution, errorrecording) 		 				-SSSw--31SSSoSERVERNAMESERVERPROCESSERRORLOGGERProc 1INFO ...INFO ...FATAL ...Proc 2Proc 1Proc 1Proc 2Proc 2”Proc 1” addressI’m ”Proc 1”DISKFATAL ...Statrt Proc NProc N terminatedWhere is ”Proc 1”Terminate Proc NFigure 1: HERA-B Central Services2.4 Software Design ApproachThe main considerations in the design of the commononline software both for the SC and RC systems are min-imizing the software development by the detector groups,facilitating maintenance of the code, providing a commonupgrade path and supporting the variety of the computertypes and operating systems running on them. To achievethese objective the template approach has been consideredas the important one. There are the following templates ofcommon use: data monitors; slow process control; slow control graphical user interfaces;3 SLOW AND RUN CONTROLSYSTEMS CONCEPT3.1 Process Tree Applying the State MachineAll the processes are arranged in a control tree. The topprocess controls and monitors the state of the whole tree. Itsends state transition commands to get all the processes inthe state that corresponds to a particular phase of the datataking. The processes in turn send their states according totheir internal status and at its change.All the processes belonging to a certain subsystem areusually registered in a separate sub-domain provided by thename service. The processes of general use are given socalled ’system’ domains. This approach allows to simplifythe identification of the processes in the tree.3.2 Slow Control as a Part of the Control TreeThe slow control system has its own process tree. Weassume that the system runs all the time. When a data tak-ing run starts a special processes called ’Branches’ connectto the corresponding branches of the slow control system toget them participating in the run. Therefore the slow con-trol processes also have some internal states that changedepending on the data taking run stage.				jjjj	jj jj6?............/7............~}iPPq1)6?6?............./7..........RIiPPq1)>=..........RIyXXz............=>	     -6?I@@@R9:UICSCSCUIUI UISCSub-detectorRun ControlSlow ControlSPCSub-detectorBRANCHCRCFigure 2: Slow Control as a Part of the Control Tree3.3 Dynamic Configuration of the RunOne of the important features of the run control systemis the ability to configure the run configuration dynamical-ly. The objects of the configuration are: subsystems (sub-detectors providing data); data logging type (disk, tape, none); trigger type (FLT, SLT, TLT, 4LT); resources in use (number of farm nodes);3.4 Parallel Runs and ControlsTo have the parallel runs for different sub-detectors isan important feature of the Run Control system. It allowsto test some sub-detectors in stand-alone mode along withthe data taking for the other part of the sub-detectors. Theparallel runs are also possible for the Slow Control of thesub-detectors.Every run has its own ’RUN’ domain provided by thename service. This excludes the interference among theruns.6314 IMPLEMENTATION OF SLOW ANDRUN CONTROL SYSTEMSEvery run has its own domain not to interfere with the oth-ers. Global DB servers provide all client applications withthe interested information. The information about all theprocesses that supposed to be started for data taking run isalso stored in the DB and the process manager starts therequired processes according to this information by meansof the process service.SERVERNAMESERVERDBERRORLOGGERSERVERPROCESSSERVERPROCESSSERVERPROCESSPROCESSMANAGER6%6%%				-6?6?-?-? ??6 6 6...........RIyXXXz............=>9:.........................................................................................................................................................................................................................................................................................................................RIyXXXz.............=>9:.........................................................................................................................................................................................................................................................................................................................RIyXXXz............=>9:..............................................................................................................................................................................................................................................................................................................run-specific processesglobal processesSUBS1SUBSNSUBS2CRCUIRUN NAMESERVERRUN ERRORLOGGERRUN PROCESSMANAGERFigure 3: Slow and Run Control Start-UpEvery run starts by launching a run control graphical us-er interface (GUI), that will control and monitor the topprocess in the control tree (CRC, CSC, see fig. 2).The operator chooses the type of the run configurationand the subsystems that are going to participate in the run.The start-up of the process tree begins with the requestfrom the GUI to the global process manager to launch thefirst portion of the run processes. They are the name server,the process manager, the local error logger, the Central RunControl (CRC) process.All the remaining processes for the run are started by thelocal process manager on requests sent by the CRC and thebranch processes.After all processes have been launched, the state ma-chine control links are established and control is grantedto the operator to start data taking.The status of processes participating in the run are be-ing monitored and controlled. There is the possibility torestart processes on their termination. In case of the termi-nation of a process that can not be restarted and consideredas ”critical” the run will be terminated.This scheme is applied to both the Run and Slow Controlsystems.5 STATUS AND PLANSThe HERA-B detector completion is scheduled by theend of 1999. The Hera-B Run Control and Slow Controlsystems has been operated as two parallel systems sincethe beginning of this year.Currently 500 - 600 processes are run by the Run Con-trol and Slow Control systems. The transition to the sys-tems with 2300 process does not mean a great effort. Themost part of the processes is just a duplication of those al-ready running. Those processes will run on the SHARCDSP nodes (switch processes) and farm nodes (fourth leveltrigger processes).They will also be commissioned by theend of the year.Completion of the systems,their integration and reliabil-ity improvement are the main points of our concern. As theexperiment gets complete, the both systems become moreand more complex. We anticipate that the issue of start-uptime and the process control reliability become more andmore important. At this respect we are trying to configurethe system in such a way that some processes will be turnedto a kind of central services reused by many runs withouttheir termination.6 REFERENCES Hera-B Message System Name Service, DAQ-Note. Proposal for Hera-B Message Handler Interface,DAQ-Note. The State Machine Control Package for the HERA-BRun Control System, DAQ-Note. Hera-B Error Logging Facility, DAQ-Note. Online framework for monitor and diagnostic rou-tines, DAQ-Note. Hera-B DAQ Web-page http://www-hera-b.desy.de/subgroup/daq/welcome.html Towards a HERA-B Slow Control System, DAQ-Note,A.Jelezov, ITEP Moscow, zhelezov@mail.desy.deU.Uwer, HU Berlin, uwer@ifh.de Parameter Control SoftwareA.Jelezov, ITEP Moscow, zhelezov@mail.desy.deU.Uwer, HU Berlin, uwer@ifh.de Hera-B Slow Control Web-pagehttp://www-hera-b.desy.de/subgroup/slowctrl/welcome.html632

The Hera-B Slow and Run Control System

Abstract

Similar works

Full text

Available Versions

CERN Document Server