Bründermann, E.

Caselle, M.

Funkner, S.

Kopmann, A.

Mexner, W.

Niehues, G.

Tan Jerome, N.

Vogelgesang, M.

English

KITopen

NEW COLLABORATIVE APPROACH TO SCIENTIFIC DATAMANAGEMENT WITH NOVAW. Mexner∗, E. Bründermann, M. Caselle, S. Funkner, A. Kopmann,G. Niehues, N. Tan Jerome, M. VogelgesangKarlsruhe Institute of Technology, Karlsruhe, GermanyAbstractAccelerator physics studies at the storage ring KARA atKIT produce terabytes of diagnostics data per day, whichis recorded once and then reused on a long-term basis toanswer different research questions at KIT. Finally, raw dataand intermediate analysis results should be published alongwith scientific results. Thus storing from the very beginningthe data of all analysis steps and its metadata in a centralportal would be very beneficial. Similar requirements existfor synchrotron X-ray micro tomography at the KIT imagingcluster and there is an interest to share the large data analysiseffort. By using a new collaborative approach, the NOVAproject aims to create tools, to enable an efficient use ofvaluable beam time. For micro tomography beamlines theproject will build up a comprehensive database of variousdemonstrator organisms for the morphological analysis ofanimals. The NOVA portal is integrated in the local datahandling procedures and the datasets automatically appear inthe NOVA portal as they are recorded. For both applications,accelerator diagnostics and X-ray tomography, the NOVAportal will offer new collaborative tools to enable synergeticdata analysis.INTRODUCTIONScientific data management is becoming increasingly im-portant for large scale physics facilities. The European Com-mission points out already in 2016 that e-science is essentialto meet the challenges of the 21st century in scientific discov-ery and learning [1]. The emerging question is: Is your datauseful for somebody in 10 years? Experiments nowadaysproduce data at extremely high rates, which are put highdemands to the whole data acquisition chain with respect todata analysis, curation, storage and usages. While the firsttwo steps are naturally in the focus of the scientists, the latersteps are often not handled with the same effort. However,the basics of the data lifecycle must be considered as early aspossible so that curated data sets with high-quality contentare stored. The experimental boundary conditions must bedescribed in form of metadata as completely as possible inorder to be able to analyze data later and reuse it interdisci-plinary. For providing open access to these data, the FAIRdata principles, as introduced by Wilkinson et al. [2], are im-portant. Data have to be Findable, Accessible, Interoperableand Reusable. To be findable, data should have a globallyunique and eternally persistent identifier. To be accessible,also (meta)data have to be online. For being accessible,∗ wolfgang.mexner@kit.edu(meta)data have to be retrievable by their identifier using astandardized open, free and platform independent protocol,e.g. a public repository. To be interoperable, (meta)data usea formal, accessible, shared, and broadly applicable languagefor knowledge representation. To be reusable, metadata havea plurality of accurate and relevant attributes and should bepublished together with a clear and accessible data usagelicense.Detectors with high temporal and spatial resolution arenowadays available and used in an increasing number ofexperiments. The resulting data rates are faster growingthan data handling technologies. Thus, we move to an erawhere data becomes to large to copy and more and morescientists in smaller organizations are excluded from latesttechnologies. During the last ten years several large efforthas been taken to find solutions for improved analysis, man-agement and access to large data sets. Synchrotron X-raytomography has served as an example for imaging applica-tions in general. The integration of online data processing inDAQ systems is essential for high data rate applications. Itimproves the quality of recorded data and enables advancedexperimental control. For the demonstrator application X-ray tomography, a suite of modular software components,called the UFO GPU computing platform, has been designedand implemented. It is intended to execute complex beam-line protocols, including real-time data investigation andon-site data processing [3–6]. New methods are added toenhance reconstruction quality and to compute acceptablereconstructions from few data [7, 8]. To manage scientificdata at all processing levels, we started the NOVA project. InNOVA, the Network for Online Visualization and SynergisticAnalysis, a group of X-ray experts, engineers, computer sci-entists, mathematicians and biologists teams up to advanceanalysis tools for tomographic data. The project aims forsynergistic data analysis and is building up a comprehensivedata portal for morphologic images of small insects [9, 10].The instrumentation in accelerator physics is currentlyundergoing a dramatic change. With the new instrumentsKAPTURE and KALYPSO developed at KIT continuousmonitoring of electron beams is possible [11]. These toolsuncover phenomena in elctron bunch dynamics and enableadvanced beam control [12]. Similar to the imaging appli-cations mentioned before multidimensional datasets withsizes up to the Terabyte level are recorded. Common to allthis high rate and high resolution measurements is the needto describe datasets carefully and to provide hierarchicalexploration of these datasets. Due to the size of the datasets,storage is costly and access is complex and time consuming.12th Int. Workshop on Emerging Technologies and Scientific Facilities Controls PCaPAC2018, Hsinchu, Taiwan JACoW PublishingISBN: 978-3-95450-200-4 doi:10.18429/JACoW-PCaPAC2018-FRCA1FRCA1224ContentfromthisworkmaybeusedunderthetermsoftheCCBY3.0licence(©2018).Anydistributionofthisworkmustmaintainattributiontotheauthor(s),titleofthework,publisher,andDOI.Data Acquisition and Data Storage, Data AnalysisRepositories like the European projects EUDAT or ZEN-ODO allow to store datasets with searchable metadata, butcurrently there are no suitable tools available to browse thiskind of datasets and to inspect multi-scale phenomena. Inorder to share analysis effort in international communitiesand to provide access to raw data with scientific publicationswe will adapt the technologies developed with the NOVAweb portal and investigate methods to browse in multidimen-sional datasets. We are convinced, complex datasets needprecise textual metadata and visual exploration. Improvedvisualization is the key to the development of better metadatafor multidimensional datasets.As a first prototype of the NOVA portal for the tomo-graphic datasets has been already developed. In the nextstep we will adapt the NOVA environment for electron beamdiagnostics at KIT. It is intended to explore with a secondcommunity the benefits of collaborative data analysis. Thesystem is constantly evolving and integrated with both user-visible front-end components to enhance the view on thedata as well as back-end components that connect the systemwith the actual experiment setup. In the future, the systemwill enable scientists to structure their data automatically ina hierarchical manner, which is a prerequisite for efficientstorage of large data volumes.NOVA DATA PORTALSynchrotron X-ray microtomography offers unique op-portunities for the morphological analysis of small animals.Internal structures become observable even in opaque organ-isms in a non-invasive, three-dimensional way at sub-micronresolution. By using a new collaborative approach, NOVAaims to create new possibilities allowing for a more effi-cient use of valuable beam time at tomographic synchrotronbeamlines. At the same time the project establishes a com-prehensive database of various demonstrator organisms anddevelops the NOVA data portal to archive, analyze, and sharethese datasets.The NOVA data portal is split into a core system andthird-party services. The core system provides login, au-thentication, and management facilities as well as a RESTAPI to interact with those facilities remotely. Third-partyservices can register with the core system to provide addi-tional functionality and features that should or could not runon the same machine as the portal backend itself. Two mainservices that are currently implemented are the thumbnailservice and a 3D visualization service. The thumbnail ser-vice is used to generate iconic images to get a quick ideawhat kind of data is contained in a data set. The service readsthe middle slice of a dataset, shrinks it to a user-defined sizeand maps the grey values to an RGB triple and saves theresulting JPEG in an on-disk cache from which a browserrequest is served. Instead of using a generic icon, part ofthe data with a distinguishing color helps to identify a previ-ously seen data quicker (Fig. 1, left). The 3D visualizationservice generates on-demand slice maps from the raw imagedata for client-side rendering using a 3D WebGL browserlibrary WAVe. The service receives a description of the de-sired dataset and its volume origin and region, then readsthe required slices, rescales them and re-orders them in a 2Dslice map. This process happens in asynchronous fashion, i.e.the client requests the generation and waits for the service tofinish processing and get the final slice map URLs. The slicemap’s filename is determined by a hash computed from thesize requirements for efficient on-disk storage and retrieval.On the front-end side the 3D visualization is started as soonas the slice maps for the entire volume is generated. Theuser can then translate, rotate, and scale the volume anddefine a bounding box to “zoom in” to load a smaller partof the volume with a higher resolution. This zoom requestwill then generate a new task for the slice map server with anew origin and a new region. The right-hand side of Fig. 1shows the front-end after zooming in on a particular area ofthe volume.The WAVe Library for 3D Web VisualizationThe WAVe library is a web-based 3D visualization appli-cation that renders volumetric slice map data as 3D objects.It consists of multiple modules which render the data in dif-ferent forms, i.e. using surface rendering or direct volumerendering. These modules are implemented as shaders thatenable a high flexibility in fine-tuning the GPU usage suchas varying the ray-casting steps or performing various in-terpolation schemes. The library is based on the Three.jsframework which simplifies the 3D scene management. Fig-ure 2 shows the architecture of the library. The API layerallows users to integrate the library into any user interfacedesigns.ADAPTING NOVA FOR ACCELERATORRESEARCHThe presented data management workflow has been de-veloped from the beginning in a modular and portable way.It is meant as a generic approach for applications with largemulti-dimensional datasets. While being developed for orga-nizing tomographic datasets of biological origin, it can easilybeing adopted to other fields. We have identified a seconduse-case for accelerator physics data recorded at the KARAlight source, which was also used to acquire tomographicdata in the ASTOR campaigns. The accelerator researchdata e.g. Terahertz research, consists of one Terabyte permachine physics day, with the capability of one Terabyteper hour, and has been acquired by different detectors likeEO-Laser, Streak camera, HEB detector, KAPTURE andKALYPSO. The development of fast detection methods forcomprehensive monitoring of electron bunches is a prereq-uisite to gain comprehensive control over the synchrontronemission in storage rings with their MHz repetition rate. Forexample the “KArlsruhe Linear arraY detector for MHz-rePetition rate SpectrOscopy” (KALYPSO) developed atKIT allows to detect longitudinal electron bunch profiles viasingle-shot,near-field electro-optical sampling at the Karl-sruhe Research Accelerator (KARA) [12]. For one analysis,12th Int. Workshop on Emerging Technologies and Scientific Facilities Controls PCaPAC2018, Hsinchu, Taiwan JACoW PublishingISBN: 978-3-95450-200-4 doi:10.18429/JACoW-PCaPAC2018-FRCA1Data Acquisition and Data Storage, Data AnalysisFRCA1225ContentfromthisworkmaybeusedunderthetermsoftheCCBY3.0licence(©2018).Anydistributionofthisworkmustmaintainattributiontotheauthor(s),titleofthework,publisher,andDOI.100.000 samples with 0,91 Mhz are taken with signal andanother set of 100.000 samples as calibration measurementsand ar stored structured as tables in the container formatHDF5. The organization of the data takes place on file sys-tem level with a structuring by date / filling number / detectorand measuring station. The metadata is recorded separatelyin a digital log book (elog) and the accelerator’s operat-ing data is automatically logged into a NoSQL Cassandradatabase.The interesting use case for NOVA is, that these datahas been recorded only once and can be reused on a long-term base for different accelerator physic research questions.Informations like the shift of the bunch profile center in rela-tion to different machine parameters could be automaticallycalculated and displayed in the portal. Thus, offering allthese data enriched with metadata in a central portal wouldbe a big benefit for accelerator research. The core infras-tructure is capable of hosting this data, the question thatarose was how to ingest it into the system. For the alreadystored data, some metadata is partially digitally no available(e.g. paper logbooks) or differently structured depending oneach experiment type. This means importing can only hap-pen case-by-case. To alleviate this, we identified commonmetadata structures and began writing an import descriptionformat and an import tool that is capable of transforminginput data to a common format usable by the NOVA portal.This tool processes an easy extensible job list in JSON for-mat (Fig. 3), which collects all available metadata of thedatasets of a certain experiment type and could be added byspecific filters collecting data from additional sources likee-logbooks and the Cassandra database.SUMMARY AND OUTLOOKExisting repositories like EUDAT or Zenodo allow one topublish digital data with a digital object identifier and makeit searchable. The NOVA portal is not only a repository,but allows to browse digital 3D datasets. We are planningto adapt the NOVA collaborative research approach to theFigure 1: Overview of the NOVA web portal. Left. Thumbnails shown are generated by the thumbnail service allow foreasier and quicker distinction between different datasets. Right. Interactive 3D visualization for a selected dataset as surface(upper image) and voxel rendering (lower image). The two smaller blue boxes denote the current zoom area in the x-y- andy-z plane, the blue bounding box in the 3D view shows the current slicing operation.Scheme of the FLUTE accelerator withall installed and planned components.12th Int. Workshop on Emerging Technologies and Scientific Facilities Controls PCaPAC2018, Hsinchu, Taiwan JACoW PublishingISBN: 978-3-95450-200-4 doi:10.18429/JACoW-PCaPAC2018-FRCA1FRCA1226ContentfromthisworkmaybeusedunderthetermsoftheCCBY3.0licence(©2018).Anydistributionofthisworkmustmaintainattributiontotheauthor(s),titleofthework,publisher,andDOI.Data Acquisition and Data Storage, Data AnalysisFigure 2: The image represents the high-level architectureof the WAVe library.Figure 3: Concept of importing metadata to NOVA portal.accelerator community for the new generation of detectorsfor accelerator beam diagnostics.ACKNOWLEDGMENTWe like to thank the german Federal Ministry of Educationand Research (BMBF) supporting the development of theNOVA portal with grant 05K16VBK.REFERENCES[1] https://www.naturvardsverket.se/upload/kalendarium/2016/open-data/jiri-pilar-eu-commission.pdf[2] M. D. Wilkinson, M. Dumontier, et al., “The FAIR GuidingPrinciples for scientific data management and stewardship”,Scientific Data, vol. 3, pp. 160018, 2016. doi:10.1038/sdata.2016.18[3] M. Vogelgesang, T. Farago, T. dos Santos Rolo, A. Kopmann,and T. Baumbach, “When hardware and software work inconcert”, in Proc. IPACLEPCS’13, San Francisco, CA, USA,Oct. 2013, paper TUPPC044, pp. 661–664.[4] T. Dritschler, S. Chilingaryan, T. Farago, A. Kopmann,and M. Vogelgesang, “InfiniBand interconnects for high-throughput data acquisition in a TANGO environment”, inProc. PCaPAC’14, Karlsruhe, Germany, paper FPO001, pp.161–163.[5] M. Vogelgesang, T. Farago, T. F. Morgeneyer, L. Helfen,T. dos Santos Rolo, et al., “Real-time image-content-basedbeamline control for smart 4D X-ray imaging”, Journal ofSynchrotron Radiation, vol. 23, no. 5, pp. 1254–1263, 2016.doi:10.1107/S1600577516010195[6] M. Vogelgesang, L. Rota, L. E. A. Perez, M. Caselle, S. Chilin-garyan, et al., “High-throughput data acquisition and pro-cessing for real-time x-ray imaging”, in Proc. SPIE, SanDiego, CA, USA, 2016, vol. 9967. doi.org/10.1117/12.2237611[7] A. Shkarin, E. Ametova, S. Chilingaryan, T. Dritschler,A. Kopmann, et al., “An Open Source GPU AcceleratedFramework for Flexible Algebraic Reconstruction at Syn-chrotron Light Sources”, Fundamenta Informaticae, vol. 141,no. 2–3, pp. 259–274, 2015. doi:10.3233/FI-2015-1275[8] R. Shkarin, E. Ametova, S. Chilingaryan, T. Dritschler,A. Kopmann, et al., “GPU-optimized direct Fourier methodfor on-line tomography”, Fundamenta Informaticae, vol. 141,no. 2–3, pp. 245–258, 2015. doi:10.3233/FI-2015-1274[9] S. Schmelzle, M. Heethoff, V. Heuveline, P. Loesel, J. Becker,et al., “The NOVA project: maximizing beam time efficiencythrough synergistic analyses of SRµ CT data”, in SPIE 2017,San Diego, CA, USA, 2017.[10] NOVA web page, http://ufo.kit.edu/dis/project/nova.php[11] M. Caselle et al., “A high-speed DAQ framework for fu-ture high-level trigger and event building clusters”, JINST,vol. 12, pp. C03015, 2017. http://iopscience.iop.org/article/10.1088/1748-0221/12/03/C03015[12] S. Funkner et al., “High throughput data streaming of in-dividual longitudinal electron bunch profiles in a storagering with single-shot electro-optical sampling”, arXiv.org,arXiv:1809.07530v1 [physics.acc-ph], 2018.12th Int. Workshop on Emerging Technologies and Scientific Facilities Controls PCaPAC2018, Hsinchu, Taiwan JACoW PublishingISBN: 978-3-95450-200-4 doi:10.18429/JACoW-PCaPAC2018-FRCA1Data Acquisition and Data Storage, Data AnalysisFRCA1227ContentfromthisworkmaybeusedunderthetermsoftheCCBY3.0licence(©2018).Anydistributionofthisworkmustmaintainattributiontotheauthor(s),titleofthework,publisher,andDOI.

New collaborative approach to scientific data management with NOVA

https://publikationen.bibliothek.kit.edu/1000098086/39413704

New collaborative approach to scientific data management with NOVA

Abstract

Similar works

Full text

Available Versions

KITopen