Fredian, Thomas

Stillerman, Joshua

DSpace@MIT

PSFC/JA-01-20CompactPCI Based Data Acquisition with MDSplusJ.A. Stillerman, T.W. FredianJuly 2001Plasma Science and Fusion CenterMassachusetts Institute of TechnologyCambridge, MA 02139 USAThis work was supported by the U.S. Department of Energy, Cooperative Grant No.DE-FC02-99ER54512. Reproduction, translation, publication, use and disposal, in wholeor in part, by or for the United States government is permitted.Submitted for publication to Fusion Engineering and Design.CompactPCI based Data acquisition withMDSplusJoshua A. Stillerman and Thomas W.FredianMIT Plasma Science and Fusion CenterNW17-268190 Albany St.Cambridge, MA, USA 02139AbstractAlcator C-Mod is incorporating CompactPCIbased data acquisition cards into the MDSplusdata acquisition system. Each crate includes adiskless computer running the mdsip serverfrom MDSplus under the linux operating system,and one or more digitizer cards with onboardmemory. A minimal set of software isdownloaded to this computer at boot timeallowing it to arm and read out the cards whenrequested by the host data acquisition computers.This diskless design is very attractive in our highfield environment and simplifies the maintenanceand configuration of the embedded computers.Separating the digitizers from the dataacquisition computers allows the data acquisitionto be done in a platform independent way.Ethernet provides robust, inexpensivecommunications. The initial digitizer cards havethirty-two 16 bit, 250KHz digitizers, 128Msamples of memory, two arbitrary waveformgenerators and 8 programmable digital outputs.CompactPCI provides an attractive alternative toour aging CAMAC based data acquisitionequipment.IntroductionThe data acquisition hardware in use at AlcatorC-Mod is predominantly based on the CAMACstandard. It consists of approximately 700modules of 75 different types in 62 crates on 5SCSI serial highways. The system is aging, andsuffers from both robustness and performanceproblems. Though we would like to replace it,the large installed base makes it infeasible, bothfor economic and for technical reasons. Untilrecently hardware for new diagnostics anddiagnostic upgrades was chosen to fit into ourCAMAC environment. The existinginfrastructure and expertise made other choicestoo expensive. New applications withrequirements for large numbers of channels havemade the investigation of other data acquisitionplatforms attractive. The lower hybrid currentdrive system will need approximately 400channels and the replacement of the digitizers forthe magnetics diagnostic will require another 200channels. Once a new system is in place, it canbe applied to smaller scale applications atrelatively low cost. The alternatives of VME[1],standard PCI, new CAMAC modules, andCompactPCI[2] were investigated, andCompactPCI was chosen for the platform.BackgroundThe CAMAC standard[31 was designed in thelate 1960s and extensively used in experimentalthe physics community. The hardware ismodular, new devices can be plugged intoexisting crates and operated fairly independentlyfrom the modules in place. A wide range oftransient recorders is available with onboardmemory to hold recorded events. Manyspecialized modules, such as phase digitizers,time of flight analyzers and high speed timershave been developed for physics diagnosticsapplications. Crates can be distributed around anexperiment or laboratory, and optically isolatedfrom each other using optical u-ports. Theadvent block mode transfers in the 1980s andSCSI serial highways in the 1990s allow fortheoretical transfer rates of about 3Mbytes/Second, in practice we can achieve onlyabout 0.5 Mbyte/second from crates withenhanced mode controllers and 0.08Mbyte/second from unenhanced crates.The main technical shortcoming is theinterdependence of the crates on a given serialhighway. If any crate or optical Uport on ahighway fails, none of the devices on thathighway can be accessed. There arecommunication schemes that overcome this,such as Ethernet crate controllers and directconnections from each crate back to the hostcomputers. These will need to be pursued for theongoing support of the existing equipment.There are additional factors that make CAMACunattractive for new applications.The market for CAMAC equipment is small; thiscauses prices to be relatively high, and theselection of available modules to be fairlylimited. Much of the equipment in use at AlcatorC-Mod is no longer manufactured. Thecommunication speed is becoming less adequateas data rates and memory sizes increase. The1limited processing power available in the crates,both complicate the communication protocols,and restrict the data processing that can beoffloaded from the data acquisition computers.There are vendors willing to develop new customhardware for particular applications, but there isnot a lot of externally motivated development ofnew modules.CompactPCI was chosen for these newapplications after considering VME, standardPCI, and new CAMAC modules. VME wasrejected for several reasons. The chassis, andthe real-time software that runs on the embeddedcomputers are expensive. Most of the availableboards require the embedded computer'smemory for data acquisition. The embeddedcomputer required external communicationhardware to overcome the backplane'sbandwidth limitations and a large amount devicespecific programming. Finally, the VMEstandard was developed in 1981 and is starting toshow its age. A system based on standard PCIcards initially looked very good. Theinfrastructure consists of inexpensive personalcomputers with Ethernet communications.Ethernet provides fast and robust data transfers,and data acquisition boards are available frommany manufacturers. The main drawbacks arethat the boards and connectors are relativelyfragile, and that the computer needs to beremoved from the rack and disassembled toaccess or replace the boards; though CPCI issomewhat more expensive and is not asubiquitous as standard PCI, it nicely addressesthese concerns.SolutionThe platform we chose is based around a CPCIbased computer running linux[4]. A disklessboot scenario was chosen to avoid damage fromthe high magnetic fields in our environment andto simplify the management of these systems.The operating system and root file system areloaded over Ethernet using etherboot[5] to aramdisk. The computer in each crate loads thesame boot image. The systems differentiatethemselves by getting dhcp assigned ipaddresses. The operating system image includesthe MDSplus[6] client library, a telnet deamonfor diagnostic purposes, and will eventuallyinclude a web server for access to configurationand status information. Using the same bootimage on all of the crates makes upgrades andmaintenance very simple.The support for specific data acquisition boardshas been written in the MDSplus built-ininterpreter called tdi. This scheme was firstdeveloped at EPFL[7] for use on the TCVexperiment. The code is written so that it can beused either locally on the CPCI computer, orremotely, driven by a central data acquisitioncomputer. If operated locally, with local storage,one of these crates could be used as a completestandalone data acquisition system. When usedremotely, as we plan to do, the crate can betreated as a backplane with an intelligentcontroller. In this remote model, the users canview the new equipment as a simple collection ofdigitizers; very similar to the existing CAMACbased modules. The powerful embeddedcomputer allows for the data to be preprocessedwhile it is being read out. The initialimplementation supports both subsampling anddigital filtering. The platform is flexible, makingit be easy to experiment with a variety ofimportance sampling schemes.The user interface for controlling the devices isvery similar to the existing MDSplus devicesupport. A board specific setup form is used tospecify the hardware and software settings to beused. From the users perspective the onlydistinction between these new devices and theexisting equipment is that an ip node name andboard number are used instead of a designationfor CAMAC highway, crate and slot.Figure 1 - CPCI device setup2The initialization routine processes the user-specified settings, optionally connects usingmdsip to the remote computer where thehardware is attached, and then evaluates anexpression that sets up and arms the hardware.The store routine again optionally connects, andfor each channel reads the data, subsampling anddigitally filtering them if requested, and writesthem to MDSplus. The filters are specified bythe user as an array of coefficients to beconvolved with the data before they aresubsampled. The data can be re-read from theboards at the full time resolution aroundinteresting events. The boards retain the data intheir on board memory until the next time theyare armed.Hardware DetailsThe first applications of this system are usingdigitizers purchased from D-Tacq Solutions[8]and timebase decoders being designed by theRFX group at CNR in Padova Italy. Thedigitizers have thirty-two 16 bit, 250KHzdigitizers, 128 Msamples of memory, twoarbitrary waveform generators and 8programmable digital outputs. Analog inputsignal processing is done on a mezzanine board,which also houses the front panel and inputconnectors. The timers are being designed to bebackwards compatible with our existing opticallydistributed timing system[9]. These new timershave 64 bit counters and run at 10 MHz insteadof the original modules built by MPB which ranat 1 MHz with 16 bit counters. They will be ableto drive the CPCI backplane timing signalsdirectly.The performance of these digitizers, both interms of data transfer speed and analog to digitalconversion is quite good. The boards can beread by the embedded computer at more than 5megabytes/second. The data can be read overour local area network at a rate of 2.5megabytes/second. Figure 2 shows an acquiredsignal that was digitized at 250 Khz. The inputsignal was a 10 Hz sine wave with a trianglewave superimposed on it. The added signal'sOfrequency was swept from 3.1 to more than 125kHz to see the 50 kHz cutoff frequency of thefour pole Butterworth filters in the input circuit.The top panels show the signal sampled at thefull rate. The middle panels show the effect ofdecimating the data to every 10& point. Thebottom traces show the effect of subsampling toevery 10 ' point after applying a 41 point digitalfilter. On the right size of the figure is expandedaround at time where the input frequency isabout 21 kHz, which is well above the Nyquistfrequency of the subsampled data. The digitalfilter attenuates the aliased data by a factor of100. This clearly demonstrates both thenecessity and power of digitally filtering the datawhen it is being under sampled.-Figure 2 Acquired signals aliasing andfilteringConclusionCPCI provides an attractive platform for dataacquisition under MDSplus. While it would beprohibitively expensive to replace all of ourexisting CAMAC based equipment, CPCI willwork well for new applications and in instanceswhere existing equipment needs to be replacedfor maintenance reasons. By choosing disklesslinux for the operating system, the crates can beput into place with very low managementoverhead and should operate robustly in our highmagnetic field environment. From the naiveuser's perspective this equipment will appear tobe very similar to existing device, but embeddedcomputer and network-based communicationswill provide them significantly betterperformance.This work was performed with the support of theAlcator C-Mod staff. Significant contributionswere also made by Xavier Llobet and BasilDuval from EPFL, Gabriele Manduchi of CNRand John Mclean and Peter Miline from D-TacqSolutions. We would like to thank them.3VIV, V[1] VITA STANDARDS ORGANIZATION.The VMEbus, Handbook, Fourth Edition,Scottsdale, AZ, (1999)[2] PCI Industrial Computer ManufacturersGroup (PICMG), The CompactPCI Standard,http://www.picmg.org/.[3]Modular Instrumentation and InterfaceStandards (CAMAC), ANSI/IEEE Std 583-1982[4] Linux @ is a registered trademark of LinusTorvalds. http://www.linux.org/[5] Yap, K., Etherboot: Etherboot home page,http://etherboot.sourceforge.net/[6] Stillerman,J.A.,Fredian,T.W.,Klare,K.A.,Manduchi,G.,MDSplus data acquisition system, Review ofScientific Instruments Vol 68 No 1 January 1997[7] B.P. Duval, et.al., Evolution not Revolutionin the TCV Tokamak Control andAcquisition System, Proceedings of the 21stSymposium on FusionTechnology, Madrid, 11-15 September 2000, toappear.[8] D-TACQ Solutions Ltd. East Kilbride,SCOTLAND. http://www.d-tacq.com/[9]V. Schmidt, G. Flor, G. Manduchi, I.Piacentini, "The timing system of the RFXNuclear Fusion Experiment", Proceeding of theInternational Conference on Accelerators andLarge Experimental Physics Control Systems.KEK, Tsukuba, Japan,November 11 - 15, 1991pp367-370.4

CompactPCI based Data acquisition with MDSplus

http://dspace.mit.edu/bitstream/1721.1/93983/1/01ja020_full.pdf

CompactPCI based Data acquisition with MDSplus

Abstract

Similar works

Full text

Available Versions

DSpace@MIT