46,797 research outputs found
The Locus Algorithm III: A Grid Computing system to generate catalogues of optimised pointings for Differential Photometry
This paper discusses the hardware and software components of the Grid
Computing system used to implement the Locus Algorithm to identify optimum
pointings for differential photometry of 61,662,376 stars and 23,799 quasars.
The scale of the data, together with initial operational assessments demanded a
High Performance Computing (HPC) system to complete the data analysis. Grid
computing was chosen as the HPC solution as the optimum choice available within
this project. The physical and logical structure of the National Grid computing
Infrastructure informed the approach that was taken. That approach was one of
layered separation of the different project components to enable maximum
flexibility and extensibility
A Mediated Definite Delegation Model allowing for Certified Grid Job Submission
Grid computing infrastructures need to provide traceability and accounting of
their users" activity and protection against misuse and privilege escalation. A
central aspect of multi-user Grid job environments is the necessary delegation
of privileges in the course of a job submission. With respect to these generic
requirements this document describes an improved handling of multi-user Grid
jobs in the ALICE ("A Large Ion Collider Experiment") Grid Services. A security
analysis of the ALICE Grid job model is presented with derived security
objectives, followed by a discussion of existing approaches of unrestricted
delegation based on X.509 proxy certificates and the Grid middleware gLExec.
Unrestricted delegation has severe security consequences and limitations, most
importantly allowing for identity theft and forgery of delegated assignments.
These limitations are discussed and formulated, both in general and with
respect to an adoption in line with multi-user Grid jobs. Based on the
architecture of the ALICE Grid Services, a new general model of mediated
definite delegation is developed and formulated, allowing a broker to assign
context-sensitive user privileges to agents. The model provides strong
accountability and long- term traceability. A prototype implementation allowing
for certified Grid jobs is presented including a potential interaction with
gLExec. The achieved improvements regarding system security, malicious job
exploitation, identity protection, and accountability are emphasized, followed
by a discussion of non- repudiation in the face of malicious Grid jobs
Extending DIRAC File Management with Erasure-Coding for efficient storage
The state of the art in Grid style data management is to achieve increased
resilience of data via multiple complete replicas of data files across multiple
storage endpoints. While this is effective, it is not the most space-efficient
approach to resilience, especially when the reliability of individual storage
endpoints is sufficiently high that only a few will be inactive at any point in
time. We report on work performed as part of GridPP\cite{GridPP}, extending the
Dirac File Catalogue and file management interface to allow the placement of
erasure-coded files: each file distributed as N identically-sized chunks of
data striped across a vector of storage endpoints, encoded such that any M
chunks can be lost and the original file can be reconstructed. The tools
developed are transparent to the user, and, as well as allowing up and
downloading of data to Grid storage, also provide the possibility of
parallelising access across all of the distributed chunks at once, improving
data transfer and IO performance. We expect this approach to be of most
interest to smaller VOs, who have tighter bounds on the storage available to
them, but larger (WLCG) VOs may be interested as their total data increases
during Run 2. We provide an analysis of the costs and benefits of the approach,
along with future development and implementation plans in this area. In
general, overheads for multiple file transfers provide the largest issue for
competitiveness of this approach at present.Comment: 21st International Conference on Computing for High Energy and
Nuclear Physics (CHEP2015
AliEnFS - a Linux File System for the AliEn Grid Services
Among the services offered by the AliEn (ALICE Environment
http://alien.cern.ch) Grid framework there is a virtual file catalogue to allow
transparent access to distributed data-sets using various file transfer
protocols. (AliEn File System) integrates the AliEn file catalogue as
a new file system type into the Linux kernel using LUFS, a hybrid user space
file system framework (Open Source http://lufs.sourceforge.net). LUFS uses a
special kernel interface level called VFS (Virtual File System Switch) to
communicate via a generalised file system interface to the AliEn file system
daemon. The AliEn framework is used for authentication, catalogue browsing,
file registration and read/write transfer operations. A C++ API implements the
generic file system operations. The goal of AliEnFS is to allow users easy
interactive access to a worldwide distributed virtual file system using
familiar shell commands (f.e. cp,ls,rm ...) The paper discusses general aspects
of Grid File Systems, the AliEn implementation and present and future
developments for the AliEn Grid File System.Comment: 9 pages, 12 figure
Efficient computation of hashes
The sequential computation of hashes at the core of many distributed storage systems and found, for example, in grid services can hinder efficiency in service quality and even pose security challenges that can only be addressed by the use of parallel hash tree modes. The main contributions of this paper are, first, the identification of several efficiency and security challenges posed by the use of sequential hash computation based on the Merkle-Damgard engine. In addition, alternatives for the parallel computation of hash trees are discussed, and a prototype for a new parallel implementation of the Keccak function, the SHA-3 winner, is introduced
EviPlant: An efficient digital forensic challenge creation, manipulation and distribution solution
Education and training in digital forensics requires a variety of suitable
challenge corpora containing realistic features including regular
wear-and-tear, background noise, and the actual digital traces to be discovered
during investigation. Typically, the creation of these challenges requires
overly arduous effort on the part of the educator to ensure their viability.
Once created, the challenge image needs to be stored and distributed to a class
for practical training. This storage and distribution step requires significant
time and resources and may not even be possible in an online/distance learning
scenario due to the data sizes involved. As part of this paper, we introduce a
more capable methodology and system as an alternative to current approaches.
EviPlant is a system designed for the efficient creation, manipulation, storage
and distribution of challenges for digital forensics education and training.
The system relies on the initial distribution of base disk images, i.e., images
containing solely base operating systems. In order to create challenges for
students, educators can boot the base system, emulate the desired activity and
perform a "diffing" of resultant image and the base image. This diffing process
extracts the modified artefacts and associated metadata and stores them in an
"evidence package". Evidence packages can be created for different personae,
different wear-and-tear, different emulated crimes, etc., and multiple evidence
packages can be distributed to students and integrated into the base images. A
number of additional applications in digital forensic challenge creation for
tool testing and validation, proficiency testing, and malware analysis are also
discussed as a result of using EviPlant.Comment: Digital Forensic Research Workshop Europe 201
Distributed Computing Grid Experiences in CMS
The CMS experiment is currently developing a computing system capable of serving, processing and archiving the large number of events that will be generated when the CMS detector starts taking data. During 2004 CMS undertook a large scale data challenge to demonstrate the ability of the CMS computing system to cope with a sustained data-taking rate equivalent to 25% of startup rate. Its goals were: to run CMS event reconstruction at CERN for a sustained period at 25 Hz input rate; to distribute the data to several regional centers; and enable data access at those centers for analysis. Grid middleware was utilized to help complete all aspects of the challenge. To continue to provide scalable access from anywhere in the world to the data, CMS is developing a layer of software that uses Grid tools to gain access to data and resources, and that aims to provide physicists with a user friendly interface for submitting their analysis jobs. This paper describes the data challenge experience with Grid infrastructure and the current development of the CMS analysis system
Technical support for Life Sciences communities on a production grid infrastructure
Production operation of large distributed computing infrastructures (DCI)
still requires a lot of human intervention to reach acceptable quality of
service. This may be achievable for scientific communities with solid IT
support, but it remains a show-stopper for others. Some application execution
environments are used to hide runtime technical issues from end users. But they
mostly aim at fault-tolerance rather than incident resolution, and their
operation still requires substantial manpower. A longer-term support activity
is thus needed to ensure sustained quality of service for Virtual Organisations
(VO). This paper describes how the biomed VO has addressed this challenge by
setting up a technical support team. Its organisation, tooling, daily tasks,
and procedures are described. Results are shown in terms of resource usage by
end users, amount of reported incidents, and developed software tools. Based on
our experience, we suggest ways to measure the impact of the technical support,
perspectives to decrease its human cost and make it more community-specific.Comment: HealthGrid'12, Amsterdam : Netherlands (2012
- âŠ