Poole, John

CERN Document Server

SUMMARY AND CONCLUSIONS OF THEIADBG WORKSHOP ON DATABASES FOR ACCELERATORSArgonne National Laboratory, 6-8th November 1995J. PooleCERN, Geneva, Switzerland1 IntroductionThe IADBG Workshop on Databases for Accelerators was held in the new conference centre at Argonne National Labo-ratory and was attended by over 40 people (more than 12% of the ICALEPCS attendance). The workshop consisted of 3major sessions and a summary session on the final day. The workshop programme was as follows:Part I - Functionality and DataSession Chairman: Ian MacGregor (SLAC)CERN, Josi SchinzelSLAC, George CraneFNAL, Kelley Trombly-Freytag and Tim ZingelmanBNL, Garry TrahernCEBAF, Sue WitherspoonPart II - Tools and ApplicationsSession Chairman: Roger Bailey (CERN)Oracle at CERN, Pierre StrubinOracle at SLAC, Andrea ChanSybase at RHIC, Garry TrahernSybase at FNAL, Kelley Trombly-Freytag and Tim ZingelmanObjectStore at CEBAF, Matt BickleyPart III - Techniques and Problem AreasSession Chairman: Chip Watson (CEBAF)Methodologies, Josi SchinzelOn-line databases, Mike LamontVery large databases, Ronnie BillenData analysis and visualisation, open discussionPart IV - Summary and ConclusionsSession Chairman: John Poole (CERN)Summary of Functionality and Data session, Ian MacGregorSummary of Tools and Applications session, Roger BaileySummary of Techniques and Problem Areas session, Chip WatsonOpen discussion to define wish list, sharing areas, further studies, future workshops, working groups ...A summary of the important issues and conclusions is given below. The proceedings are available electronically throughthe IADBG WWW pages1.1The IADBG home page URL is http://www.cern.ch/IADBG/Welcome.html2 Functionality and DataIn the first few talks it became clear that a wide variety of database management systems were being used and that everyconceivable type of data in accelerator laboratories was concerned. The data being handled falls into the following broadareas: Laboratory management – personnel, finance, pay roll, documentation, project management Equipment management – construction, maintenance Control – off-line reference data and on-line active data Beam related data management – measurement and physics dataSome interesting developments in the field of geographical data and relational representation of objects were reported fromBrookhaven National Laboratory.If one considers what are generally referred to as database activities, then memory resident home-made solutions dominate.In the relational field, Oracle and Sybase dominate. The only object database discussed in detail at the workshop wasObjectStore, which is being used at CEBAF.The diverse choice of solutions to the data management problem arises because there is no unique alternative. Databaseshave been built to suit particular problems or users and even where a coherent solution has been adopted for a project, itis necessary to have partitioned implementations to accommodate the conflicting requirements (high performance queryengines, development systems etc). In most cases reported there is, or will be, more than 1 type of implementation e.g.relational and memory resident or object and memory resident.The diversity itself leads to maintenance problems: expertise is required in several areas and replacement of legacy systemsbecomes increasingly difficult. This is probably an indication that the field is still in its infancy. Some ingenious imple-mentations, linking heterogeneous systems were presented (BNL, SLAC). These are necessary to enable activities whichrequire input from several disciplines.3 Tools and ApplicationsTools are used for data manipulation and the range which is used for data input is rather limited. For relational systems andexpert users, data input is often done using SQL. Whilst this is very easy to use, it is also very easy to corrupt or destroydata and this is why it is restricted to experts (several of whom admitted to making serious errors this way). Oracle providesa graphical interface (Oracle*Forms) which allows the user to interact in a very controlled way with the database but at theexpense of development effort.In the PC world it is possible to connect to relational databases through commercial products like EXCEL and severalinstances of such applications were reported. Apart from these few tools there are numerous home made interfaces whichhave been built to enable data input.The variety of tools available for data extraction is far greater and includes proprietary interfaces from Oracle (Ora-cle*Browser, Oracle*Report, Oracle*Graphics ...), EXCEL, PVWAVE, tcl/tk etc. Whilst there is a great variety therewas a general feeling that many of these products are not very popular with the users. In general, the public domain toolslike tcl/tk and WWW were more popular and users were happier with them - perhaps because they have a more familiarlook and feel and are not user hostile.In the case of ObjectStore the only way to interact with the database is through C++ and it requires a knowledge of thestructure of the database in order to save or retrieve data.Applications have been built using all of the tools mentioned above as well as programs written using programmatic in-terfaces to the database. All of the databases discussed at the workshop are delivered with a set of libraries which allowdevelopers to interact with the database from within a program.It was emphasised that the majority of the work associated with database systems is not creating the structures and maintain-ing the data, but building and maintaining the applications. The possibilityof enforcing referential integrity and constraintswithin the database (which arrived with the latest releases of the relational DBMS’s) was seen as a big step forward. Thisdevelopment removes the necessity of building in controls to enforce referential integrity and constraints in every appli-cation. However, this represents only a small part of the total effort required for application development.It was agreed that the area which holds the greatest promise for application development is that of the WWW interfaces.SLAC and BNL both reported developments, both based on in-house developments using public domain tools.4 MethodsExperience at CERN with project management using the European Space Agency (ESA) standards was discussed. Thestandard was succinctly described as ‘rigorously applied common sense’, but with the warning that one should alwaysapply methods in a pragmatic way.From the review of information technology methods the main conclusion was that it is essential to follow some methodologybut that the choice seems to be largely a matter of taste. The group of people who actually design and build the databases isa small fraction of the people involved in database activities (<33%) and most of them were following some methodologyand using tools. A number of tools were described: Teamwork, which supports Structured Analysis (SA), Structured Design (SD), Entity Relationship Modeling (ERM)and Objects (next/latest release ?) Oracle*CASE, which supports ERM and functional decomposition ERDraw, which supports ERM RIDL, which supported the Natural Information Analysis Method (NIAM) but is no longer supported commercially Object Protocol Method (OPM) Select OMT, which supports the Rumbaugh method TOT, an in-house tool at FNALThe difficulties of learning a methodology AND tool were discussed in some length. It was generally agreed that thereis a considerable investment to be made and there is a danger that new users will treat the method like a religion. All ofthe delegates who had been through the experience agreed that the discipline of a methodology is a vital ingredient in theproduction of a good implementation. It was also agreed that for smaller projects a complete analysis and design, usingformal methods, was an overkill.5 Techniques5.1 On-line Control DatabasesThe majority of such implementations are built around what can be classified as memory resident custom-built databases.The case of the LEP on-line database was examined in some detail. This has been built using Oracle and is in the controlpath: control applications write settings to the database and the process connected to the hardware receives a signal whichcauses it to get the new settings from the database and then it changes the hardware. Client applications use the database toget data, rather than polling the equipment. This development has been possible because of the latency in the LEP controlsystem, due in part to its size and in part by choice, where changing hardware settings should be done in a time<1s.Other large control systems which use memory resident systems (EPICS, SLC) often have much tighter constraints onresponse times. The performance limits for a relational database in a control system have not yet been determined.5.2 Very Large DatabasesWhen then LEP loggingdatabase was conceived a few years ago it was considered to be a very large database (10 Gbytes),whereas today people are discussing volumes of data orders of magnitude larger. However, in terms of the database systemsin use at accelerator laboratories it remains one of the largest to have been implemented in a relational system. Tables withup to a million rows are in use and several special techniques had to be employed in order to make the system performadequately.It was concluded that using an RDBMS for such a system is a reasonable solution as long as the I/O requirements arereasonable. In the LEP case around 120 Mbytes per day are being written to the database and this is some (unspecified)way below the limit.6 Problem Areas6.1 Data Visualisation and AnalysisMany tools are being used to present data to the user and for analysis (e.g. EXCEL, XRTgraph, PAW, PVWave ...). Ingeneral the mature sites were relatively happy with the facilities available. A new in-house system which is being usedextensively at the APS at Argonne is the SDDS (Self Describing Data Set) Toolkit. This uses the custom files in combinationwith a powerful scripting language which allows data analysis, reduction and visualisation. Whilst the system is veryperformant it was felt that the lack of data management was a distinct disadvantage.A new system is being designed for CEBAF and this is based on an integrated set of tools which will allow control ofequipment to be integrated with the data gathering, analysis and display.7 Future PlansIt was agreed that the IADBG should start a number of new activities in order to help members in the future. The firstidea was to establish a mail exploder so that members can send requests for help and/or advice on database problems. Thisservice will supplement the E-mail newsletters because not all members will necessarily wish to subscribe to the exploder.It was felt that some more detailed description of database implementations would be of great use to people starting newdevelopments. More detail than was presented at the workshop is needed, but the data structure definitions would be toodetailed and not particularly useful – some short descriptive paragraphs were thought to be most useful. The informationwill be made available through the IADBG WWW pages.The success enjoyed by WWW applications giving access to database information was stressed many times during theworkshop. It was therefore agreed that examples of such implementations (the code) will also be made available throughthe IADBG WWW pages.On a number of occasions there were discussions about the performance capabilities of various database systems. There areindications that for some applications object oriented databases can considerably out perform relational implementationsbut it was clear that there was no solid basis for comparison. It was decided that some kind of benchmark test will be definedso that meaningful comparisons can be made. The test should be something which is relevant to accelerator projects andwhich can serve as a guide to the relative performance of object and relational databases, for example.The IADBG was started in 1994 and had held meetings at EPAC94 and PAC95 before the workshop. It was agreed thatthe group has now established itself on a solid basis and that it should have at least one co-ordinator on each continent.J. Poole agreed to continue looking after Europe and A. Chan (SLAC) agreed to look after N. America.Finally it was unanimously agreed that the workshop had been extremely useful and that it would be appropriate to repeatit in the future. It was felt that in about 2 years it would be useful but that the situation will be reviewed at future meetingsof the IADBG. The next meeting will be at the EPAC2, which will be held in Sitges, Barcelona, Spain from 10 - 14th June1996.8 AcknowledgementI would like to express my thanks to all of the speakers and session chairmen who worked hard and made the workshop sucha success. I must also thank the organising committees of ICALEPCS for their help with the organisation of the workshop.Finally I would like to thank Bill McDowell of Argonne National Laboratory for arranging the meeting room, refreshmentsand on-site transport.2EPAC WWW pages are accessible through http://www.cern.ch/EPAC/Welcome.html

Summary and conclusions of the IADBG workshop on databases for accelerators

Abstract

Similar works

Full text

Available Versions

CERN Document Server