An efficient strategy for the collection and storage of large volumes of data for computation

In recent years, there has been an increasing amount of data being produced and stored, which is known as Big Data. The social networks, internet of things, scientific experiments and commercial services play a significant role in generating a vast amount of data. Three main factors are important in Big Data; Volume, Velocity and Variety. One needs to consider all three factors when designing a platform to support Big Data. The Large Hadron Collider (LHC) particle accelerator at CERN consists of a number of data-intensive experiments, which are estimated to produce a volume of about 30 PB of data, annually. The velocity of these data that are propagated will be extremely fast. Traditional methods of collecting, storing and analysing data have become insufficient in managing the rapidly growing volume of data. Therefore, it is essential to have an efficient strategy to capture these data as they are produced. In this paper, a number of models are explored to understand what should be the best approach for collecting and storing Big Data for analytics. An evaluation of the performance of full execution cycles of these approaches on the monitoring of the Worldwide LHC Computing Grid (WLCG) infrastructure for collecting, storing and analysing data is presented. Moreover, the models discussed are applied to a community driven software solution, Apache Flume, to show how they can be integrated, seamlessly

First measurements with the CMS DAQ and timing hub prototype-1

The DAQ and Timing Hub is an ATCA hub board designed for the Phase-2 upgrade of the CMS experiment. In addition to providing high-speed Ethernet connectivity to all back-end boards, it forms the bridge between the sub-detector electronics and the central DAQ, timing, and trigger control systems. One important requirement is the distribution of several high-precision, phasestable, and LHC-synchronous clock signals for use by the timing detectors. The current paper presents first measurements performed on the initial prototype, with a focus on clock quality. It is demonstrated that the current design provides adequate clock quality to satisfy the requirements of the Phase-2 CMS timing detectors

Optimised lambda architecture for monitoring scientific infrastructure

Brunel University London College of Engineering, Design and Physical Sciences Thomas Gerald Gray postgraduate research scholarship

Monitoring WLCG with lambda-architecture: a new scalable data store and analytics platform for monitoring at petabyte scale.

Monitoring the WLCG infrastructure requires the gathering and analysis of a high volume of heterogeneous data (e.g. data transfers, job monitoring, site tests) coming from different services and experiment-specific frameworks to provide a uniform and flexible interface for scientists and sites. The current architecture, where relational database systems are used to store, to process and to serve monitoring data, has limitations in coping with the foreseen increase in the volume (e.g. higher LHC luminosity) and the variety (e.g. new data-transfer protocols and new resource-types, as cloud-computing) of WLCG monitoring events. This paper presents a new scalable data store and analytics platform designed by the Support for Distributed Computing (SDC) group, at the CERN IT department, which uses a variety of technologies each one targeting specific aspects of big-scale distributed data-processing (commonly referred as lambda-architecture approach). Results of data processing on Hadoop for WLCG data activities monitoring are presented, showing how the new architecture can easily analyze hundreds of millions of transfer logs in a few minutes. Moreover, a comparison of data partitioning, compression and file format (e.g. CSV, Avro) is presented, with particular attention given to how the file structure impacts the overall MapReduce performance. In conclusion, the evolution of the current implementation, which focuses on data storage and batch processing, towards a complete lambda-architecture is discussed, with consideration of candidate technology for the serving layer (e.g. Elasticsearch) and a description of a proof of concept implementation, based on Apache Spark and Esper, for the real-time part which compensates for batch-processing latency and automates problem detection and failures

Not Available

Not AvailablePowdery mildew caused by Erysiphe necator is one of the most important diseases of grapevine in India. It affects the grape production by causing significant reduction in yield and quality of grapes. Chitosan is a deactivated derivative of chitin and the efficacy of three chitosan formulations, Chitosan fulvate @ 2ml/L, Chitosan oligosaccharides @ 2ml/L, Chitosan @ 4ml/L were evaluated against powdery mildew. The physiological loss in weight of berries was also assessed. During the fruiting seasons of 2018-2019 and 2019- 2020, chitosan formulations were evaluated both as solo treatments and in alternation with Ampelomyces quisqualis @ 5ml/L. All Chitosan formulations effectively inhibited powdery mildew on leaves and berries when compared to untreated control. Among solo treatments, chitosan exhibited lower PDI (17.76 and 5.29) on leaves than untreated control (24.28 and 13.12) during 2018-2019 and 2019-2020 respectively. In case of bunches similar trend was observed with PDI 23.39 and 9.90 against untreated control 30.31 and 17.18 in both seasons. However, the formulations were found more effective when used in alternation with Ampelomyces quisqualis. During 2018-2019 and 2019-2020, Chitosan / Ampelomyces quisqualis treatment recorded significantly lower PDI of powdery mildew on leaves i.e.12.55 and 4.77than the untreated control with PDI 24.28 and 13.22 respectively. In case of bunches, treatment Chitosan / Ampelomyces quisqualis showed the lowest disease severity with PDI 19.28 and 8.98 against untreated control with PDI 31.60 and 17.18 in both the seasons respectively. The applications of formulation containing chitosan alternated with sprays of Ampelomyces quisqualis (5ml/L) showed least disease as well physiological loss in weight of grapes. All chitosan formulations were effective in minimizing physiological loss in weight of grapes as compared to control. Among all treatments, Chitosan / Ampelomyces quisqualis treatment recorded the lowest PLW i.e.13.94 and 13.90, whereas untreated control showed the highest PLW i. e. 15.85 and 15.87 during both seasons respectively. Results showed that chitosan can be effectively used in powdery mildew management, either alone or in alternation with the bioformulation of Ampelomyces quisqualis.Not Availabl