Bakker, R

Birke, T

Bub, S

Müller, R

CERN Document Server

LOGGING OF SUPERVISORY DATA AT BESSY  T. Birke , R. Bakker, S. Bub, R. Mu¨ller, BESSY, Berlin, GermanyAbstractThird generation light sources are facilities requiring ulti-mate beam stability. Typically component design tries tosuppress well known perturbations. During operations acrucial element of the control system is the logging of allsignals that might be suited to identify unknown sources ofperformance degradation. As soon as correlatable data areavailable it is feasible to plan detailed additional measure-ments, characterize the systems involved and design ap-propriate countermeasures (e.g. feedback, component re-design). In this paper the status of the data archiving sys-tem in use at BESSY is described. Planned developmentsand ideas are sketched.1 SCOPE AND PRIORITIESAt dedicated measurements parameters are stepped inten-tionally and the resulting effects on other parameters arerecorded and evaluated. In contrast the precautionary mea-sure of logging parameters relevant for operational condi-tions into certain archives has no specific application inmind. Mostly the data are never extracted and analyzed.But if a certain interruption, a slow drift or any other un-expected behavior of the machine has to be analyzed thelogged data become invaluable and should be sufficientlycomplete to contain a clue to the explanation or at least ahint on what should be examined more thoroughly.1.1 Data of InterestA vast amount of parameters is required to describe the op-eration condition of a light source. The basic configurationis given by the strength of all beam guiding, kicking andwiggling fields as well as by the status and performance ofthe accelerating and damping elements. With stored beamall signals from the diagnostic system become relevant:beam intensity and lifetime, orbit displacements, sourcepoint size and shape, etc. Environmental variables like vac-uum in the beam pipe, temperature profiles and variations,performance of the cooling systems strongly influence op-eration conditions. Delivered beam quality also dependson operational modes like fill pattern, status of feed-backsystems etc. Frequently changes in the accelerator perfor-mance are due to not-intended operator actions or causedby spontaneous and volatile events typically captured byan alarm system.Funded by the Bundesministerium fu¨r Bildung, WissenschaftForschung und Technologie and by the Land BerlinEmail: birke@bii.bessy.de1.2 Format and GroupingsDifferences between various commonly used recordingmodes are basically substantiated by the meaningful fre-quencies of data gathering. The adaptive way of storingdate with a fixed, sufficiently high frequency during periodsof rapid changes and switching to the recording of monitorevents notifying the occasional occurrence of a change isfor most signals a universal compromise [1].In addition snap-shot files are needed for comparison andreload of different accelerator settings. Reading with user-adjustable frequency is needed for online visualized stabil-ity monitoring and trending. Short term history buffers helpto mediate a quick overview of the present status. Eventcapturing facilities are used to document (transient) alarmsand operator actions. Run book entries hold data measuredby hand or comments and observations.Natural structure for a grouping of the various signals torecord with one or the other method is given by the deviceclasses the monitored equipment belongs to.1.3 LatenciesAcceptable latencies between data recording and viewingvary strongly with the application. For ‘online’ trend anal-ysis the latency should be negligible, for long term statisti-cal analysis data latency is no issue.2 ARCHIVING MODES AND TOOLSAt BESSY a couple of signal logging facilities are in use.They cover most of todays needs but do not share a com-mon data format and are thus not suited for combined cor-relation analysis.2.1 Snapshot FilesOne of the central working horses for the operation is asave/restore tool that allows to keep particular well char-acterized accelerator set-ups and return to these states byreloading the appropriate data sets[2]. Focus of this toolis given to the flexibility of hierarchical, partial, additiveand filtered selection of data subsets for storing, compar-ison, reload and export. File-sets control the bundling ofdata gathering and hold the data. They reflect device hier-archies, are easily adaptable to changes in the equipmentcollections and allow to be systematically backwards cor-rected if e.g. hardware changes require the use of new units.Today a thousand of files describe ‘good’ states and can beanalyzed with respect to operational changes.International Conference on Accelerator and Large Experimental Physics Control Systems431International Conference  Accel rator and L rge Experim ntal Physics Control Systems, 1999, Trieste, ItalyGiven a ‘dense’ logging of set-points and read-backs ofall devices data duplication could be avoided. An alterna-tive save/restore solution would require only the time stampand a descriptive comment to tag and save a certain pre-accelerators and storage ring set-up. But then the snapshotdata extraction engine had to provide the functionality nowcovered by file-set configurations.2.2 Online Stability Monitor DisplaysA simple way to find trends and correlations is to con-nect the signals of interest to a software strip-chart recorderfeaturing a very flexible update frequency, time span andzooming control. Print outs or data dumps to disk fre-quently document a first observation of a correlation andtrigger often a more systematic precision measurement.Figure 1: Stability Monitor using StripTool.2.3 Continuously Monitored DataA constantly growing list of signals is collected contin-uously in the background. Initially information aboutlong term power supply stability and vacuum quality wasneeded. Later all types of beam parameters, RF operationconditions, insertion device changes etc. have been addedas the interest in these parameters became apparent.The data gathering engine is based on periodicallyrestarted SDDSmonitor[3] processes. Semi-automaticallygenerated configuration files provide the list of signals tocollect with a fixed frequency. Storage format is one fileper day and device class in a simple calendar oriented di-rectory structure.Beam position data have to be treated differently. Thefront end computer has to post the orbit measurement pe-riodically in a stream type format for technical reasons. Adedicated program collects the information and stores theordered, averaged and cleaned data in compact binary formto a similar directory structure.2.4 Sparse and Irregular DataAll relevant console applications log messages, error con-ditions, action triggers etc. via cmlog[4] to a messagedatabase in a structured, but otherwise free format. This fa-cility is an invaluable tool to pinpoint irregularities or mis-behavior within the system. It helps to reconstruct actionsSDDS MonitorSDDS MonitorSDDS MonitorSDDS MonitorFile SetsRecombinePlot WEBViewShellPresent StructureShellFigure 2: Data Collection and Extraction Engine.or events and their effect that have not been adequately re-ported by the operator. Extraction of correlations out of thisdata pool would require a dedicated application program.A WEB based electronic run book is provided by a cgiinput/output interface to ORACLE database tables set upwith a perl package. The stored data allow statistical anal-ysis of delivered beam time, failure frequencies and theircauses etc.automatically (re-)generated Thu Sep 30 19:25:00 METDST 1999actual images are (re-)generated every 30 minutesVacuum History: Thursday, Sep. 30. 1999Today previous day whole week: #1999-39 list of weeks list of months Microtron Injection Line Booster Transfer LinePostScript PostScript PostScript PostScriptStorage Ringvac. sect. 1 vac. sect. 2 vac. sect. 3 vac. sect. 4PostScript PostScript PostScript PostScriptvac. sect. 5 vac. sect. 6 vac. sect. 7 vac. sect. 8PostScript PostScript PostScript PostScriptToday previous day whole week: #1999-39 list of weeks list of months Figure 3: Data presentation on the WEB.4323 RETRIEVAL MODES AND TOOLS3.1 Statically Presented Overview DataThe parameters of common interest are periodically ex-tracted from the continuously collected archive and con-verted into static graphs. These plots are accessible in abrowsable history on the WEB (Fig. 3). These standarddisplays provide a sufficiently detailed overview of the av-erage accelerator performance.3.2 Search and CorrelationsThe numerous filter programs supplied with the SDDStoolkit[3] support a large variety of data recombinations,mathematical analysis, search for specific signatures andcoincidences. A tcl/Tk based wrapper around some SDDStools helps to scan the archive. It allows for the most com-mon operations like selection of time span, device class,signals of interest and output option. A graphical previewhelps to ensure that the data selected contain the desiredinformations. For further postprocessing these data can beexported to SDDS, ASCII or spread-sheet format.4 SHORTCOMINGS, IMPROVEMENTPLANSThe deficiencies of the BESSY logging have to be over-come in the near future but the need is not yet urgent.4.1 Adaptive Logging Frequency, OptimizedRetrieval SpeedSDDSmonitor provides only data collection at fixed rates.The compromise in data acquisition frequency misses fastintermediate events and results in a waste of disk space andretrieval speed. That problem can be solved by migrationto another collector tool and storage format[1].4.2 Universal Data Storage FormatGiven a file-set based archive networked access besidesNFS requires a specific solution. Retrieval tools have toadapt to the data format within the archive. A uniform APIrequires an additional layer of retrieval calls. In this respectrelational databases supply attractive features. With SQL astandard language is available to express correlations. TheDB data extraction tools provide networked access to thedata and allow for presentations on most common viewingtools like WEB browsers and java virtual machines.For a relational DB major points of concern are im-port/export speed especially if flexibility requires a largenumber of different tables. A possible scenario could bea staged archive: Collectors with adaptive data acquisitionrate store their information into file-sets optimized for re-trieval speed. For small latency views and rapid scans alimited functionality retrieval tool is available that has ac-cess to the specific file set format. For long term statisticalanalysis and correlation finding a ‘digest’ process continu-ously transfers the data from the file-sets into a relationalDB and makes it available for SQL based applications (Fig.4). The same process could also take care of data aging andcompression strategies.WEBFile SetsORACLESQLjava- IDEA -AdaptiveCollectorSmallLatencyViewRetrieveDigestShort TermLong TermFigure 4: Staged Archive, Universal Storage Format.4.3 Post Mortem DumpA ‘nice to have’ link between the event catching facilitiesand the continuous monitoring would be a sliding windowdata buffer holding a (short) time span of specific signalsof interest that is dumped to archive on certain triggers.That would help to analyze the vicinity of a (rare) failurecondition with high resolution. At BESSY there is no workin progress on this issue but conceptual decisions to comehave to be open for a later integration.5 SUMMARYThe present archiver situation at BESSY is a ‘minimum-effort’ solution. Basic requirements are adequately cov-ered. Maintainability, accessibility, retrieval speed andflexibility are not totally satisfactory.Due to the enormous variety of data sources and for-mats relevant for supervisory analysis universal recordingand retrieving tools seem to be not feasible. Neverthelesspromising development progress has been reported[1] to-wards a generic and portable toolkit covering most of thelogging requirements.6 REFERENCES[1] L.R.Dalesio, W.Watson, M.Bickley, M.Clausen, DataArchiving in Experimental Physics, Proceedings of the 1997ICALEPCS, Beijing, 1997, p.75[2] See http://www csr .bessy .de/control/SoftDi st/Sa veResto re/[3] M. Borland, L. Emery, N. Sereno, Proceedings of the 1995ICALEPCS, Chicago, 1995, p.382[4] cmlog has been developed by TJNAF.433

Logging of Supervisory Data at BESSY

Abstract

Similar works

Full text

Available Versions

CERN Document Server