Bieser, F

Bordua, M

Cherney, M

Chrin, J

Ferguson, I

Fiedler, J

Ghazikhanian, Vu

Gross, J

Harper, G

Howe, M

Jacobson, S

Kravtsov, P

Lewis, S

Lin, J

Lionberger, C A

McParland, C P

McShane, T S

Meier, J

Reichhold, D M

Sakrejda, I

Yamamoto, E

Zhang, W

CERN Document Server

DEVELOPMENT OF THE HARDWARE CONTROLS SYSTEM FOR THESTAR EXPERIMENTD. Reichhold2, F. Bieser4, M. Bordua4, M. Cherney2, J. Chrin2, I. Ferguson1, J. Fiedler2,V. Ghazikhanian1, J. Gross2, G. Harper6, M.Howe6, S. Jacobson4, P. Kravtsov5, S. Lewis4, J. Lin2,C. Lionberger4, C. McParland4, T. McShane2, J. Meier2, I. Sakrejda4, E. Yamamoto1, and W. Zhang31Department of Physics, University of California, Los Angeles, California 90095, USA2Department of Physics, Creighton University, Omaha, Nebraska 68178, USA3Department of Physics, Kent State University, Kent, Ohio 64242, USA4Lawrence Berkeley National Laboratory, University of California, Berkeley, California 94720, USA5 Moscow Engineering Physics Institute, Moscow, 115409, Russia6Department of Physics, University of Washington, Seattle, Washington 98195, USAAbstractThe STAR experiment completed its first commissioningrun in August, 1999. Hardware was controlled by anEPICS-based system. Of special note is the use of asingle field bus (High-level Data Link Control - HDLC)by all STAR subsystems for monitoring and control offront-end electronics and access to front-end buffers.STAR Controls was implemented by a consortium ofinstitutions. A large portion of system development hasbeen and continues to be carried out by undergraduatestudents. Effort has been minimized by the adoption ofexperiment-wide standards and the use of pre-packagedsoftware tools.1. THE STAR EXPERIMENTThe Solenoidal Tracker At RHIC (STAR) experimentwill study relativistic heavy ion collisions of gold nucleiat 100 GeV per nucleon. STAR is also capable of studiesusing the less massive species at varying energies thatwill be delivered by the Relativistic Heavy Ion Collider(RHIC) at Brookhaven National Laboratory. STAR canmeasure single-event variables representing entropy,temperature, baryochemical potential, strangenesschemical potential, intermittency, and particle andenergy flow. High transverse momentum processes in theform of jets, mini-jets and single particles [1] andperipheral collisions [2] can also be studied.The baseline subsystems of the STAR experiment are thetime projection chamber, data acquisition (DAQ),trigger, online computing and magnet. The maintracking device in STAR is a cylindrical TPC, 4.18meters in length with inner and outer radii of 0.5 m and2 m respectively [3]. Additional subsystems (a ringimaging Cerenkov detector, an electromagneticcalorimeter, a radial time projection chamber a siliconvertex detector, and a silicon strip detector) will be addedin the next year. A second commissioning run will takeplace in December, 1999. The first collisions at RHIC forphysics studies are scheduled for January, 2000.2. STAR HARDWARE CONTROLSDEVELOPMENTThe STAR controls system was developed at a number ofremote sites with the initial system integration takingplace at Lawrence Berkeley Laboratory for cosmic raytesting of a single TPC sector [4]. Final integration istaking place at Brookhaven National Laboratory. Toexpedite the integration process, a number of designrules were instituted at the start of the process [5]. Thedevelopment tools were standardized. Software toolkitsthat would be available to all collaborating institutionswere selected. EPICS [6] was selected as well-suited tothis project and the number of EPICS tools to be usedwas limited. It was perceived that graphical databaseswould be easier to maintain over the lifetime of theexperiment. Similar efforts were made to limit thenumber of kinds of interfaces to the system. Theexperiment also adopted a common field bus. Amezzanine board was developed which could be used forall subsystems.STAR hardware is designed to be failsafe. All requiredpersonnel and property protection is mandated to beindependent of STAR Hardware Controls.Controls systems were developed as hardwareconstruction progressed for most of the baseline. Thiseliminated the need for the development of controlssystems for test setups, and gave the users and developersearly experience with the system. Databases areconstructed on the subsystem level, such that they can beused for subsystem testing and remain easily included ina larger detector configuration.The baseline system has required approximately 20 man-years to construct. Roughly two man-years were paid outof project funds. The remainder was contributed bycollaborating institutions. Completing the hardwarecontrols will require an estimated 6-8 man-years ofeffort. Project goals called not only for a reliablehardware controls system, but also for the training ofyoung physicists. To this end, expert support was enlistedonly for early special record development and systemmaintenance as well as hardware design and fabrication.Roughly 2/3 of the manpower consists of students andpostdoctoral assistants. Undergraduate students played asignificant role in the project. Their efforts were enabledby the use of standardized toolkits. These introductorystudents were able to take on projects, which, althoughInternational Conference on Accelerator and Large Experimental Physics Control Systems385International Conference  Accel rator and L rge Experim ntal Physics Control Systems, 1999, Trieste, Italyroutine to senior staff, were nevertheless interesting andcapable of producing rapid and significant results fromtheir standpoint. The use of undergraduates increased themanpower requirements, since students are onlyavailable part time, and face a significant learning curve.The structure of the project required that a postdoctoralresearcher was present full-time at the integration site.3. STAR HARDWARE CONTROLSSYSTEMThe STAR Hardware Controls system sets, monitors andcontrols all subsystems. In addition, Hardware Controlscan generate and display alarms and warnings for eachsubsystem. A real-time VxWorks environment is utilizedfor this purpose and a controls software package, EPICS,is used to provide a common interface to all subsystems.The baseline STAR detector consists of time projectionchamber subsystems (anode high voltage, cathode fieldcage, gating grid, front-end electronics power supply,HDLC link, gas, laser, interlocks, and VME-cratecontrol), mechanisms for the exchange of informationwith the STAR trigger and online, as well as externalmagnet and accelerator systems. Approximately 10,000parameters governing experiment operation are currentlycontrolled and monitored. The communication of theSTAR Hardware Controls with the other systems isshown in Figure 1.Figure 1. Communication between STAR real-timesystems.Magnet status and accelerator information is obtainedfrom the accelerator controls database using a ControlDEVice (CDEV) client/server interface [7]. Beam scalarinformation, STAR experiment status and STARmeasurements of the magnetic field can also be providedto the RHIC control group using CDEV.The set of databases for a particular run configuration ismaintained by Hardware Controls. The set of all possiblerun-time databases are stored as part of OnlineComputing. The appropriate set is transferred toHardware Controls as Run Control transitions betweenvarious operating states. At present, this connectionconsists only of an interface between EPICS andObjectivity. This connection will be completed by theDecember run.4. STAR CONTROLS FIELD BUSThe majority of the experiment’s electronics is located onthe detector. In the case of the TPC alone there are over4000 front-end electronics (FEE) boards in the endcapregions of the TPC for amplifying, shaping anddigitizing signals from the TPC pad plane. FEE cards areconnected to 144 readout boards also located on the TPCendcaps. Readout boards were designed with an internalmemory and alternate readout path for diagnosticpurposes and installation testing. The readout boardmemory can be written to and read from the HardwareControls link as well as the normal data stream.In order to have reliable links to the readout boards,easily identify any malfunctioning readout boards,communicate with a large bandwidth over a distance ofabout 30 meters under a strong magnetic field andminimize cabling using a multi-drop topology, an HDLCprotocol (running at 1 Mbit/s) was selected as the fieldbus for the experiment. The link is based on a 68302processor serial port. This link is mounted on a separatePC daughter board [8].The VME interface, a Radstone SBCC-1 board, supportsup to four HDLC channels. Each HDLC channelconfigures the readout of a single extended region of theTPC by communicating with six readout boards over anRS-485 link. In addition, the controls link can monitorand adjust voltages on the TPC readout boards andcontrol the power on FEE cards (in groups of four).When testing, a bit pattern can be sent using the HDLClink to a readout board to initiate the creation of ananalog pulse. The FEE cards that are connected to thereadout board will receive the pulse signal, then amplifyand digitize it. A comparison of the output data with theinput data facilitates an examination of the operation ofthe event memory buffer and the electronics. To detectcabling errors each of the FEE boards and readout boardsis tagged with a geographical address. The HDLC linkcontrols the routing of the digitized data to the buffer onthe readout boards or directly to DAQ via a fiber opticslink. If the data is sent to the buffer, it can be read backby Hardware Controls via the HDLC link. Data can alsobe sent from the buffer to DAQ. A comparison of thedata between Hardware Controls and DAQ allows theintegrity of the data acquisition and HDLC paths to beverified. HDLC will also be used for four upgradedetector subsystems.5. EPICS AT STAREPICS was selected as the foundation for the STARcontrol software environment because it incorporates astandard means of sharing information and services,graphical display and control interfaces, and a robust,real-time operating system. At STAR, EPICS is run onSun workstations connected to VME processors runningthe VxWorks operating system. The components ofEPICS used at STAR are the Motif Editor and DisplayOther TPC SubsystemsSTAR Hardware ControlsTPC FEESTAR TriggerSTAROnlineDAQAcceleratorMagnetHDLCC++ CDEV386Manager (MEDM), the Graphics Database ConfigurationTool (GDCT), the sequencer, the alarm handler, and thedata archiver.A number of I/O devices are included in the system [9].The anode high-voltage system controls an Arcnetconnection to the LeCroy 1458 power supplies. TheArcnet driver is adopted from the one developed byTJLAB-Hall-B. The communication between control-panels and the Arcnet is accomplished using subroutinerecords. A Glassman high-voltage power supply iscontrolled using four VMIC modules: a 64-bitdifferential digital input board (1111), a 32-channel relayoutput board (2232), an analog to digital converter(3122) and a digital to analog converter (4116). Precisionmeasurements of the voltage drops and current aremonitored using a new EPICS driver providing a GPIBconnection to a Keithley electrometer. FEE power iscontrolled by six Acromag digital I/O boards that areconnected to solid state relays. A new EPICS driver hasbeen written for this purpose. The HDLC link has beeninterfaced with the EPICS software from which a numberof control and monitoring tasks can be performed. AnEPICS record and related EPICS driver support havebeen written for this purpose. Specialized controlelectronics were developed for the TPC gating grid.EPICS drivers have been written to control the potentialson the gating grid and to monitor its status. The gascontrol program runs on a PC using the Windows NToperating system. (Data is saved every 20 seconds onto across-mounted disk, which is accessed by HardwareControls. The values are displayed in EPICS.) This is theonly TPC subsystem with a front-end not under the directcontrol of an EPICS interface. The implementation of anEPICS front-end was not cost effective as the gas systemwas developed in Russia using interfaces developedspecifically for the STAR experiment. Laser calibrationis monitored and controlled through a VMIC 32-channelrelay output board. A LeCroy 1440 high-voltage powersupply monitors and sets the phototube voltages via anRS232 serial connection. Communication isaccomplished using the subroutine records. Newsubroutine functions were written for this purpose.Additional support for VME access to STAR-specifictrigger boards has been written. The safety interlocksystem is controlled using an Allen-Bradley PLC system.This stand-alone system passes status information viaTTL signals to a STAR front-end, an Acromag 948Xdigital I/O board. Additional PLC systems monitortemperatures, humidity and cooling water flow.At STAR, EPICS front-end readout (input-outputcontroller) programs are distributed among twelve VMEprocessors (of the types MVME 147, 162, 167). STARelectronics controls are housed in eleven 6-u and three 9-u Wiener VME crates. Since there is no access to theplatform where these crates are physically located duringrun time, remote control of the VME crates is required.This is achieved through a built-in VME-CANbusinterface. The crate housing the CANbus interface,platform ethernet hub, terminal server and platformbased workstation are provided with uninterruptiblepower to minimize the impact of a brief loss of power. Asecond system oversees the 14 VME crates used for dataacquisition.The integrated system operated effectively during thefirst commissioning run. The single item of concernresulted from system developers being located at remotesites during run time, delaying the process of upgradingcode. This difficulty should be reduced as more systemexperts are trained.6. SUMMARYAn EPICS-based control system has been developed forthe STAR experiment at RHIC. STAR HardwareControls maintains the system-wide control of the STARdetector with its EPICS databases at the subsystem level.The system developed contains a number of new EPICSimplementations. An HDLC link provides the field busused by the experiment for controls and as an alternatedata path. The system began operation as part of anengineering run during the summer of 1999.7. ACKNOWLEDGMENTSThis work was supported in part by the United StatesDepartment of Energy under contract numbers DE-FG03-96ER40991 and DE-AC03-76SF00098, theCreighton College of Arts and Sciences and the Dean ofthe Graduate School, Creighton University.8. REFERENCES[1] J. W. Harris et al., “The STAR Experiment atRelativistic Heavy Ion Collider,” Nucl. Phys. A566, 277c(1994); STAR Conceptual Design Report, LawrenceBerkeley Laboratory Report, PUB-5347 (1992).[2] S. R. Klein and J. Nystrand, “Exclusive VectorMeson Production in Relativistic Heavy Ion Collisions,”Phys. Rev. C60, 014903 (1999).[3] H. Wieman et al., “STAR TPC at RHIC,” IEEETrans. Nucl. Sci. 44, 671 (1997).[4] J. Chrin et al., “Results from the STAR TPC SystemTest”, IEEE Trans. Nucl. Sci. 44, 592 (1997).[5] J. Gross et al., “A Unified Control System for theSTAR Experiment”, IEEE Trans. Nucl. Sci., 41 184(1994).[6] A.J. Kozubal et al., “Experimental Physics andIndustrial Control System,” ICALEPCS89 Proceedings,288 (Vancouver, 1989).[7] J. Chen et al., “CDEV: An Object-Oriented ClassLibrary for Developing Device Control Applications,”ICALEPCS95 Proceedings, 97 (Chicago, 1995).[8] S. R. Klein et al., “Front End Electronics for theSTAR TPC,” IEEE Trans. Nucl. Sci. 43, 1768 (1996).[9] J. Lin et al., “Hardware Controls for the STARExperiment at RHIC,” Real Time Proceedings (Santa Fe,1999) to be published.387

Development of the Hardware Controls System for the STAR Experiment

Development of the Hardware Controls System for the STAR Experiment

Abstract

Similar works

Full text

Available Versions

CERN Document Server