Multi-Layer Distributed Storage of LHD Plasma Diagnostic Database

At the end of LHD experimental campaign in 2003, the amount of whole plasma diagnostics raw data had reached 3.16 GB in a long-pulse experiment. This is a new world record in fusion plasma experiments, far beyond the previous value of 1.5 GB/shot. The total size of the LHD diagnostic data is about 21.6 TB for the whole six years of experiments, and it continues to grow at an increasing rate. The LHD diagnostic database and storage system, i.e. the LABCOM system, has a completely distributed architecture to be sufficiently flexible and easily expandable to maintain integrity of the total amount of data. It has three categories of the storage layer: OODBMS volumes in data acquisition servers, RAID servers, and mass storage systems, such as MO jukeboxes and DVD-R changers. These are equally accessible through the network. By data migration between them, they can be considered a virtual OODB extension area. Their data contents have been listed in a “facilitator” PostgreSQL RDBMS, which now contains about 6.2 million entries, and informs the optimized priority to clients requesting data. Using the “glib” compression for all of the binary data and applying the three-tier application model for the OODB data transfer/retrieval, an optimized OODB read-out rate of 1.7 MB/s and effective client access speed of 3?25 MB/s have been achieved. As a result, the LABCOM data system has succeeded in combination of the use of RDBMS, OODBMS, RAID, and MSS to enable a virtual and always expandable storage volume, simultaneously with rapid data access

Nonstop Lose-Less Data Acquisition and Storing Method for Plasma Motion Images

Plasma diagnostic data analysis often requires the original raw data as they are, in other words, at the same frame rate and resolution of the CCD camera sensor. As a non-interlace VGA camera typically generates over 70 MB/s video stream, usual frame grabber cards apply the lossy compression encoder, such as mpeg-1/-2 or mpeg-4, to drastically lessen the bit rate. In this study, a new approach, which makes it possible to acquire and store such the wideband video stream without any quality reduction, has been successfully achieved. Simultaneously, the real-time video streaming is even possible at the original frame rate. For minimising the exclusive access time in every data storing, it has adopted the directory structure to hold every frame files separately, instead of one long consecutive file. The popular ‘zip’ archive method improves the portability of data files, however, the JPEG-LS image compression is applied inside by replacing its intrinsic deflate/inflate algorithm that has less performances for image data

Steady-state data acquisition method for LHD diagnostics

The LHD experiment has gone through 5 campaign periods over the past 4 years, during which the diagnostics data continues to grow and the primary 28 measurements produce about 620 MB/shot in 150 shot/day 3-min cycles. In 2002, 30-min long-pulse experiments will be carried out in LHD, where real-time operations are indispensable for plasma measurements and data acquisition. The new scheme for utilizing conventional CAMAC digitizers in long-pulse experiments has been discussed and examined. As a result, in LHD, CAMACs will shift into 120?180 s cyclic operation, synchronized by the diagnostic timing system. The new CompactPCI-based digitizer frontend has performed about 84 MB/s continuous acquisition in benchmarks, and has been formulated with the conventional CAMAC system to make concurrent acquisitions

Adaptive data migration scheme with facilitator database and multi-tier distributed storage in LHD

Recent “data explosion” induces the demand for high flexibility of storage extension and data migration. The data amount of LHD plasma diagnostics has grown 4.6 times bigger than that of three years before. Frequent migration or replication between plenty of distributed storage becomes mandatory, and thus increases the human operational costs. To reduce them computationally, a new adaptive migration scheme has been developed on LHD’s multi-tier distributed storage. So-called the HSM (Hierarchical Storage Management) software usually adopts a low-level cache mechanism or simple watermarks for triggering the data stage-in and out between two storage devices. However, the new scheme can deal with a number of distributed storage by the facilitator database that manages the whole data locations with their access histories and retrieval priorities. Not only the inter-tier migration but also the intra-tier replication and moving are even manageable so that it can be a big help in extending or replacing storage equipment. The access history of each data object is also utilized to optimize the volume size of fast and costly RAID, in addition to a normal cache effect for frequently retrieved data. The new scheme has been verified its effectiveness so that LHD multi-tier distributed storage and other next-generation experiments can obtain such the flexible expandability

Inter-application communication during LHD consecutive short pulse discharge experiment

LHD short pulse experiments are executed every three minutes. After the end of the discharge, the scientists must collect, analyze, visualize the last acquired data of the discharge, and prepare for the next discharge. From the beginning, the computer environment of the LHD (Large Helical Device) experiment has been built as a network distributed system, and various computers have been used for data acquisition or physical analysis. When one program is finished on one computer, that computer must send the results in order to the other computers to run programs. Smooth communication is required in order to finish all the tasks before the next discharge. To exchange the information among the applications running on the different computers, the authors have tried various methods, such as a commercial software to share the memory over the network, simple network file sharing method, IP multicast, web interfaces, and others. The purpose of this paper is to share our experiences of trial and error to build the network distributed systems for the consecutive plasma discharge experiments

Integrated radiation monitoring and interlock system for the LHD deuterium experiments

The Large Helical Device (LHD) successfully started the deuterium experiment in March 2017, in which further plasma performance improvement is envisaged to provide a firm basis for the helical reactor design. Some major upgrades of facilities have been made for safe and productive deuterium experiments. For radiation safety, the tritium removal system, the integrated radiation monitoring system, and the access control system have been newly installed. Each system has new interlock signals that will prevent any unsafe plasma operation or plant condition. Major interlock extensions have been implemented as a part of the integrated radiation monitoring system, which also has an inter-connection to the LHD central operation and control system. The radiation monitoring system RMSAFE (Radiation Monitoring System Applicable to Fusion Experiments) is already operating for monitoring γ(X)-rays in LHD. Some neutron measurements have been additionally applied for the deuterium experiments. The LHD data acquisition system LABCOM can acquire and process 24 h every day continuous data streams. Since γ(X)-ray and neutron measurements require higher availability, the sensors, controllers, data acquisition computers, network connections, and visualization servers have been designed to be duplicated or multiplexed for redundancy. The radiation monitoring displays in the LHD control room have been carefully designed to have excellent visual recognition, and to make users immediately aware of several alerts regarding the dose limits. The radiation safety web pages have been also upgraded to always show both dose rates of γ(X)-rays and neutrons in real time