System Development Project Development of a Prototype of RCAN P2P System

RCAN1 [1, 2] is a self-organizing peer-to-peer (P2P) overlay under development within KDE Lab2. RCAN is based on pure and flat P2P system design. RCAN uses a self-scaling multi-ring routing infrastructure

Software Development for High Speed Data Recording and Processing

The European XFEL beam delivery defines a unique time structure that requires acquiring and processing data in short bursts of up to 2700 images every 100 ms. The 2D pixel detectors being developed produce up to 10 GB/s of 1-Mpixel image data. Efficient handling of this huge data volume requires large network bandwidth and computing capabilities. The architecture of the DAQ system is hierarchical and modular. The DAQ network uses 10 GbE switched links to provide large bandwidth data transport between the front-end interfaces (FEI), data handling PC layer servers, and storage and analysis clusters. Front-end interfaces are required to build images acquired during a burst into pulse ordered image trains and forward them to PC layer farm. The PC layer consists of dedicated high-performance computers for raw data monitoring, processing and filtering, and aggregating data files that are then distributed to on-line storage and data analysis clusters. In this contribution we give an overview of the DAQ system architecture, communication protocols, as well as software stack for data acquisition pre-processing, monitoring, storage and analysis

Detectors and Calibration Concept for the European XFEL

The European X-ray Free Electron Laser (XFEL.EU) is an international research facility presently under construction in the area of Hamburg, Germany, which will start its operation at the end of 2016. The superconducting linear accelerator of the facility will deliver electron bunches with an energy of up to 17.5 GeV, arranged in trains of typically 2700 bunches at a repetition rate of 4.5 MHz. Each train will be followed by a gap of 99.4 ms. Spatially coherent X-rays are generated from the electron bunches in a series of undulators based on the Self-Amplified Spontaneous Emission (SASE) process, in three photon beamlines extending over a length of up to 200 m. Each beamline serves two experiments with different scientific goals

Karabo: An Integrated Software Framework Combining Control, Data Management, and Scientific Computing Tasks

The expected very high data rates and volumes at the European XFEL demand an efficient concurrent approach of performing experiments. Data analysis must already start whilst data is still being acquired and initial analysis results must immediately be usable to re-adjust the current experiment setup. We have developed a software framework, called Karabo, which allows such a tight integration of these tasks. Karabo is in essence a pluggable, distributed application management system. All Karabo applications (called “Devices”) have a standardized API for self-description/configuration, program-flow organization (state machine), logging and communication. Central services exist for user management, access control, data logging, configuration management etc. The design provides a very scalable but still maintainable system that at the same time can act as a fully-fledged control or a highly parallel distributed scientific workflow system. It allows simple integration and adaption to changing control requirements and the addition of new scientific analysis algorithms, making them automatically and immediately available to experimentalists

Calibration and Calibration Data Processing Concepts at the European XFEL

The European X-ray Free Electron Laser (Altarelli, 2006) is a high-intensity X-ray light source currently being constructed in Hamburg, Germany, that will provide spatially coherent X-rays in the energy range between 0.25 keV and 25 keV. The machine will deliver a unique time structure, consisting of up to 2700 pulses, with a 4.5 MHz repetition rate, 10 times per second at very high photon fluxes up to 1017 photons/s (Tschentscher, 2012). The LPD (Hart, 2012; Koch, 2013), DSSC (Porro, 2010, 2012; Lutz, 2010) and AGIPD (Graafsma, 2009) detectors are being developed to provide Mpixel imaging capabilities at the aforementioned repetition rates for a dynamic range spanning from single photon sensitivity to 104 –105 photons per pixel. The detectors are optimized for specific energy ranges. A direct consequence of the aforementioned detectors’ characteristics is that they generate raw data volumes unprecedented in photon science, ranging up to 1Mpixel x 640 memory cells x 10 pulse/s x 16 bit, i.e. 12.8 Gbyte/s. On-detector vetoing may not necessarily lower these rates much - a memory cell freed by a vetoed pulse may be used by data from one of the remaining 2700 pulses a train consists of. The PC-layer may reduce this data amount by additional software triggering, but this is not guaranteed. Figure 1 gives an overview of the different data products at European XFEL, as well as their flows and involved user roles, under the assumption that processing takes place within XFEL’s Karabo framework (Heisen, 2013). In addition to the high data rates, the Mpixel detectors’ on-sensor memory-cell and multi-gain-stage architectures necessary for the high dynamic range, pose unique challenges in detector-specific data corrections and calibration (Weidenspointner, 2012; Sztuk-Dambietz, 2013a). These challenges are addressed by providing a dedicated and thoroughly characterized set of test stands, which utilize continuous sources (Fe-55, X-ray tubes) as well as a pulsed setup: PulXar (Sztuk-Dambietz, 2013b), which is designed to produce X-ray pulses of 50-150 ns duration, within a 0.6 ms burst followed by a 99.4 ms gap. The radiation it produces thus closely matches the XFEL pulse structure. Additionally, simulation tools are being developed to assist in detector characterization (Bohlen and Joy, 2013)

Data reduction activities at European XFEL: early results

The European XFEL is a megahertz repetition-rate facility producing extremely bright and coherent pulses of a few tens of femtoseconds duration. The amount of data generated in the context of user experiments can exceed hundreds of gigabits per second, resulting in tens of petabytes stored every year. These rates and volumes pose significant challenges both for facilities and users thereof. In fact, if unaddressed, extraction and interpretation of scientific content will be hindered, and investment and operational costs will quickly become unsustainable. In this article, we outline challenges and solutions in data reduction

The Karabo distributed control system

The Karabo distributed control system has been developed to address the challenging requirements of the European X-ray Free Electron Laser facility, including complex and custom-made hardware, high data rates and volumes, and close integration of data analysis for distributed processing and rapid feedback. Karabo is a pluggable, distributed application management system forming a supervisory control and data acquisition environment as part of a distributed control system. Karabo provides integrated control of hardware, monitoring, data acquisition and data analysis on distributed hardware, allowing rapid control feedback based on complex algorithms. Services exist for access control, data logging, configuration management and situational awareness through alarm indicators. The flexible framework enables quick response to the changing requirements in control and analysis, and provides an efficient environment for development, and a single interface to make all changes immediately available to operators and experimentalists