The swiss army knife of job submission tools: grid-control

Grid-control is a lightweight and highly portable open source submission tool that supports virtually all workflows in high energy physics (HEP). Since 2007 it has been used by a sizeable number of HEP analyses to process tasks that sometimes consist of up 100k jobs. grid-control is built around a powerful plugin and configuration system, that allows users to easily specify all aspects of the desired workflow. Job submission to a wide range of local or remote batch systems or grid middleware is supported. Tasks can be conveniently specified through the parameter space that will be processed, which can consist of any number of variables and data sources with complex dependencies on each other. Dataset information is processed through a configurable pipeline of dataset filters, partition plugins and partition filters. The partition plugins can take the number of files, size of the work units, metadata or combinations thereof into account. All changes to the input datasets or variables are propagated through the processing pipeline and can transparently trigger adjustments to the parameter space and the job submission. While the core functionality is completely experiment independent, integration with the CMS computing environment is provided by a small set of plugins.Comment: 8 pages, 7 figures, Proceedings for the 22nd International Conference on Computing in High Energy and Nuclear Physic

The PAX Toolkit and its Applications at Tevatron and LHC

At the CHEP03 conference we launched the Physics Analysis eXpert (PAX), a C++ toolkit released for the use in advanced high energy physics (HEP) analyses. This toolkit allows to define a level of abstraction beyond detector reconstruction by providing a general, persistent container model for HEP events. Physics objects such as particles, vertices and collisions can easily be stored, accessed and manipulated. Bookkeeping of relations between these objects (like decay trees, vertex and collision separation, etc.) including deep copies is fully provided by the relation management. Event container and associated objects represent a uniform interface for algorithms and facilitate the parallel development and evaluation of different physics interpretations of individual events. So-called analysis factories, which actively identify and distinguish different physics processes and study systematic uncertainties, can easily be realized with the PAX toolkit. PAX is officially released to experiments at Tevatron and LHC. Being explored by a growing user community, it is applied in a number of complex physics analyses, two of which are presented here. We report the successful application in studies of t-tbar production at the Tevatron and Higgs searches in the channel t-tbar-Higgs at the LHC and give a short outlook on further developments

Object level physics data replication in the Grid

To support distributed physics analysis on a scale as foreseen by the LHC experiments, 'Grid' systems are needed that manage and streamline data distribution, replication, and synchronization. We report on the development of a tool that allows large physics datasets to be managed and replicated at the granularity level of single objects. Efficient and convenient support for data extraction and replication at the level of individual objects and events will enable for types of interactive data analysis that would be too inconvenient or costly to perform with tools that work on a file level only. Our tool development effort is intended as both a demonstrator project for various types of existing Grid technology, and as a research effort to develop Grid technology further. The basic use case supported by our tool is one in which a physicist repeatedly selects some physics objects located at a central repository, and replicates them to a local site. The selection can be done using 'tag' or 'ntuple' analysis at the local site. The tool replicates the selected objects, and merges all replicated objects into a single single coherent 'virtual' dataset. This allows all objects to be used together seamlessly, even if they were replicated at different times or from different locations. The version of the tool that is reported on in this paper replicates ORCA based physics data created by CMS in its ongoing high level trigger design studies. The basic capabilities and limitations of the tool are discussed, together with some performance results. Some tool internals are also presented. Finally we will report on experiences so far and on future plans

A Flexible Consent Management System for Master Person Indices

In healthcare, a Master Person Index (MPI) is a system that integrates information of individual from multiple data sources. To ensure confidentiality, such systems, particularly in healthcare, need to respect individual and organizational constraints on the sharing of data. This report describes a reusable consent management system that enforces such constraints and how it has been tested in the context of the Utah Department of Health (UDOH) MPI for public health

The cloud hovering over the virtual campus

The Virtual Campus has been around for about 20 years. It provides an online environment that mimics the processes and services of the physical campuses and classrooms. Its adoption is almost complete in countries where Internet access has become ubiquitous. For a time seemed like the innovation in education was happening in the Virtual Campus, but this is no more. Personal Learning Environments, Life Long Learning, MOOCS, Open Educational Resources, Mobile Apps, Gamification, Social Networks, free Cloud based services... al of the above and even more hint that not all the learning today is happening at school or in the Virtual Campus.Peer ReviewedPostprint (author’s final draft

Data Access for LIGO on the OSG

During 2015 and 2016, the Laser Interferometer Gravitational-Wave Observatory (LIGO) conducted a three-month observing campaign. These observations delivered the first direct detection of gravitational waves from binary black hole mergers. To search for these signals, the LIGO Scientific Collaboration uses the PyCBC search pipeline. To deliver science results in a timely manner, LIGO collaborated with the Open Science Grid (OSG) to distribute the required computation across a series of dedicated, opportunistic, and allocated resources. To deliver the petabytes necessary for such a large-scale computation, our team deployed a distributed data access infrastructure based on the XRootD server suite and the CernVM File System (CVMFS). This data access strategy grew from simply accessing remote storage to a POSIX-based interface underpinned by distributed, secure caches across the OSG.Comment: 6 pages, 3 figures, submitted to PEARC1

Sorting Through and Sorting Out: The State of Content Sharing in the E-Learning

On 22-24 September 2002, a group of 22 education and information technology specialists gathered on the campus of the University of California at Irvine (UCI), for a symposium on the state of educational "content sharing." (See participant list.) The meeting was sponsored by the William and Flora Hewlett Foundation Education Program and the UCI Distance Learning Center. This paper summarizes the themes that emerged from that gathering. Most papers can be characterized as collaborative, but this one is particularly deserving of that adjective. The presentation here is an attempt to synthesize the ideas of all the participants, expressed in numerous conversational and written exchanges pre-, during and post-meeting. While every effort has been made to present the range of views, surely not all participants would agree with the emphases and interpretations herein.This report includes a hyper-linked bibliography and footnotes for additional web-based material on e-learning topics. Links are provided for the reader's convenience only, and represent neither an endorsement nor a guarantee of the accuracy of the content of the associated sites. Comments and questions about this document are welcomed, however, and should be directed to the author or the meeting sponsors