Dalesio, L

Kasemir, K U

CERN Document Server

DATA ARCHIVING IN EPICSK. Kasemir, L. Dalesio, LANL, Los Alamos, New Mexico, USAAbstractThe Data Archiver is operational at the Low EnergyDemonstrator Accelerator (LEDA) at Los AlamosNational Laboratory (LANL).  In the original design,several data taking engines were foreseen.  In theimplementation, the periodic engine and the monitor-based engine were combined.  This paper discusses theperformance, disk utilization, and use of the Archiver.  Itwill also talk about plans for data management, theintegration of the archive viewer from JeffersonLaboratory, and modularization to support other datastorage methods.1  INTRODUCTIONSince the original design, the EPICS Archiver hasundertaken several significant changes.  Most importanthas been the rewriting of the archive engine, so that thedata retrieval and data collection would share a commonset of access methods.  This new work has beenperformed by Kay Kasemir at LANL.  The original filestructure and functions reported at the ICALEPCS in 1997have been maintained [1].  This paper now covers theperformance, disk utilization, use of the Archiver; the webbased archive control and data viewing, the code structureand the current state of archive data management.2  PERFORMANCESeveral performance tests have been performed on thechannel Archiver.  The original tests performed were tostudy the relative delays between seek, file open/close,large buffers and small buffers.  The result was reportedin the original paper.  The later tests are performed on thenew archive data taking engine and retrieval routines.Retrieval tests were done on two data sets using anUltra 1 from Sun Microsystems.  The first set containedover 3.0 Gbytes of data taken over 1 year.  A minimum ofone file per day was created.  Some of the early data hadas many as six files per day.  The second set of tests wasdone on a data file that was split from the original andcontained only the last 3 months of data with only one fileper month. This second file contained 1.3 Gbytes of data.In the first set of tests, data was fetched for the first ofevery month going back in time.  Two measurements aregiven; one for CPU time used and the other for actualelapsed time.  During tests on the large data set, wheredata was retrieved from the previous month, the CPUusage time ranged between 7 msec and 16 msec.  Elapsedtime ranged from 1 second to 2 seconds.  Testing retrievalfor each of the last ten months, the CPU usage timeranged from 120 msec up to 1,160 msec.  Elapsed timeranged from 1 second to 98 seconds.  The elapsed timebecame shorter as subsequent tests were run.  It is ourbelief that as the tests were run, more resources wereassigned to our task, but we have not studied this insufficient depth.  Another possibility for the disparity inretrieval times may be bad file links in the archive file.The delay of over a minute to retrieve archive data isunacceptable and we will continue to work to understandthe issue.  The second set of tests was done on the threemonthly files that were extracted from the original.  Thetotal range for CPU usage accessing any of the data was40 msec to 280 msec.  The elapsed time never exceededone second.  This second set of tests was well within allperformance parameters.Data collection tests were performed on a 450 MHzPentium II.  In both tests, the buffers are being flushed todisk every 15 seconds, which means that we are buffering150 samples per channel.  When archiving 500 channels at10 Hz, the CPU load was normally at 20% with peaks to100% for five seconds every 15 seconds.  Then a test wasdone for 1,000 channels at 10 Hz.  The CPU usage was at100% continually.  In the last case, we are loosing a littledata.  Tests will be rerun with circular buffers that cansupport more than the number of samples required for thespecified interval.  In the second case, the web-browserwas used to access the status of the Archiver whilearchiving was active.  Status data took about 5 seconds toappear, which was much slower than normal.  However,the Archiver continued to catch all signals.Performance requirements for the Archiver stated thatwe would archive 10K channels per second on a $5,000computer and access as many as 4 channels from as longago as 30 days in under 4 seconds.  Shortly, all of theserequirements will be met.  We still have concern about thenumbers that we observed for accessing data from morethan six months ago, however.3  DISK UTILIZATIONModifications since the original work include anincremental buffer assignment and some data compressioncapability.  When data is being taken for a new file, theoriginal buffer is allocated for 64 values.  This minimizesthe disk usage for a channel that is not changing veryrapidly.  If this buffer is filled, a new buffer in the samefile is allocated for 256 samples.  When this buffer isInternational Conference on Accelerator and Large Experimental Physics Control Systems463International Conference  Accel rator and L rge Experim ntal Physics Control Systems, 1999, Trieste, Italyfilled, all subsequent buffers are allocated at 1,024samples.  The file split utility also compresses these datafiles by combining the buffers into chunks up to 4,000samples in length.  In addition to combining the chunks, italso removes any unused buffer space.  The data beingarchived includes the process value, time stamp and alarmcondition.  If a value does not change, two samples arestored, the first time it was at this value, and the last timeis was at this value.  Other than this, nothing is currentlydone to reduce the disk space being used.4 ARCHIVER USEThe Archiver has been in use at LEDA for just over ayear.  It is also being tested at the Synchrotron LightSource (SLS) at the Paul Sherrer Institute (PSI) and ISACat TRIUMF.  At LEDA, we are archiving around 1,800signals every 30 seconds.  This is only 60 samples persecond.  In the first year, we collected over 3 gigabytes ofdata.  Since August of 1999, we have been archivingapproximately 1.2 Gbytes per month, which compressesdown to 400 Mbytes per month.  In addition, severalarchives have been set up at LEDA to find controllerresponses or study problem areas.  These studies werearchiving every change with a maximum rate of 10 Hz.5 USER INTERFACE FOR OPERATIONAND DATA VIEWINGThe goal of the user interface for both operation anddata viewing is to make status, control, and data retrievalavailable on any workstation with reasonable ease andacceptable performance.  The original data archive enginehad the user interface as an integral part of the code.  Thishas two major problems.  First, the Archiver could not bestarted without a windowing environment and thereforecould not be started when the workstation rebooted.Second, the status of the archive engine was onlyavailable at the workstation where the Archiver was run.A data viewer from Jefferson Laboratory [R] that allowsusers access to this data provides a great tool for studyingthe data in the UNIX environment.  At LEDA, we alsoneed access to this data from PCs, so an additional toolwas made. To determine the best approach to solving theseshortcomings in an expedient way, alternatives wereconsidered.  This study of alternatives is presented inTable 1.The availability of client programs for all platforms ofinterest (Win32, Unix) was considered the most importantfactor.  A defined protocol is preferred over a newlydesigned protocol since it provides better chances forfinding general tools, having a lower development cost,and ease shared development.  Therefore, HHTP waschosen because it provided the bests results for littleeffort.  The only problem with this approach is thehandling of dynamic status updates.  In this approach,there is no way of telling the browser about newinformation, since HTTP does not define a commonserver push mechanism.  Using the client pull mechanism,the browser will simply update periodically, regardless ofactual changes.  As for any Epics addition, ChannelAccess (CA) was initially considered a natural choice.The engine would then serve pseudo channels likeX_is_archived for each channel X.AS IS CASERVERHTTP CORBA BUILDITCLIENTSEXISTX11 EPICSCLIENTSWEBTOOLNONE NONEPROTOCOLDEFINEDN/A YES YES YES NO1 CLEINTPERENGINEYES YES YES YES YES> 1 CLIENTPERENGINENO YES YES YES YESCHANNELSEARCHNO YES NO ? YES>1 ENGINEPERCHANNELYES NO YES ? YESMONITOR>1 ENGINEPERCLIENTNO YES No ? YESDYNAMICSTATUSUPDATEYES YES  PUSH ? YESTable 1: Alternatives for providing a user interfaceto the Archiver for data collectionUnfortunately, only few non-UNIX display tools areavailable for CA, and none of them fit the Archiverrequirements well.  CA is currently limited to a single CAserver per computer that serves the same channel onlyonce per sub-net.  The custom TCP/IP protocol approachallows the implementation of all the desired features.However, the necessary development for protocoldefinition would definitely postpone the first release of anew archive engine.  CORBA was not investigatedthoroughly because of time constraints.  It might still bethe second best choice to HTTP for someone interested inwriting sophisticated clients that need deeper access to theengine.  The channel access server approach may be moreattractive when the current limitation of one server perworkstation is removed.  The HTTP server was chosen464and made operational in several weeks.  Figure 1 shows ascreen that is available for viewing archived data..Figure 1: Data Retrieval6 CODE STRUCTUREA set of database-style C++ classes is underdevelopment for reading and writing Channel Archiverdata files.  The top-level Archive class provides channelinformation via ChannelIterators, which in turn cangenerate ValueIterators that hold a Value for a specificpoint in time.  They are based on the current standard C++library as described in [2] with few exceptions:1) (FILE *)-I/O is used to access the Channel Archiverfiles because tests yielded poor results for seeks usingthe standard C++ fiostream classes2) The standard exception class is not used as a baseclass because the present egcs C++ compiler (used onLinux, Solaris, HPUX) does not support them.The Value class allows access to a value’s time stamp,status information and numeric value in both numeric andstring format, independent of the underlying dbr_time_xxxtype which is actually found in the archive.  An exampleof this code is shown in figure 1.  The code is currentlyrunning on Win32 with Microsoft Visual C++ 6.0 as wellas many UNIX systems, namely Linux, HPUX, andSolaris with egcs-2.91.66.Whenever possible, these iterator-type helper classesshould be used.  Not only do they allow easy sequentialaccess; they are also highly generic thus portable to amatching set of classes that could operate on acommercial relational database.// Given a Channel Archiver directory file name// and regular expression channel name pattern,// list values stamped from start to end time// for all matching channels:void listValues (const string &directory, conststring pattern,const osiTime &start,const osiTime &end){  Archive archive (directory);  ChannelIterator channel =    archive.findChannelByPattern (pattern);  while (channel){    cout << "Channel: " << channel->getName()      << endl;    ValueIterator value;    for (value =      channel.getValueAfterTime(start);      value && value->getTime() < end;      ++value){    cout << *value << endl;      }    ++channel;  }}7 DATA MANAGEMENTThere are many ways to provide data compression.  Atpresent the only one supported will split a file into piecesand remove unused buffer space.  This allows older datato be removed to secondary storage as a means of datamanagement.  It also allows data to be removed to othermachines to be studied.  In the future, a compressive toolto provide data management is planned.8 CONCLUSIONThe tool for providing long term archiving has beenrunning at LEDA for one year now.  A replacement that iswritten in C++ and produces the same file format is inbeta test and offers a network interface to runtime archiveinformation and archived data through a web browser.The new code also enables the replacement of the iteratorclass talking to an in-house file format, with data storageto a relational database or some other data store.  In thefuture, a server like CORBA or Channel Access maymake the data available though a client interface, butLANL is not planning such an effort.  Both data takingengines are near meeting the requirement of saving 10Ksamples per second.  They are able to retrieve data fromthe last eight hours in less than 1 second and retrieve thelast 30 days in less than 4 seconds.  Further work isrequired to provide an extensive set of data managementfunctionality.REFERENCES[1] L.R. Dalesio, et.al.  “Data Archiving in ExperimentalPhysics,” ICALEPCS’97, Beijing, Nov. 1997.http://www.aps.anl.gov/icalepcs97/paper97/p218.pdf[2] B. Stroustrup, The C++ Programming Language, 3rdedition, July 1997, Addison-WesleyFigure 2  of the iteratorclass to report  archived data.  Typical Use465

Data Archiving in EPICS

Abstract

Similar works

Full text

Available Versions

CERN Document Server