Coffea-casa: an analysis facility prototype

Data analysis in HEP has often relied on batch systems and event loops; users are given a non-interactive interface to computing resources and consider data event-by-event. The "Coffea-casa" prototype analysis facility is an effort to provide users with alternate mechanisms to access computing resources and enable new programming paradigms. Instead of the command-line interface and asynchronous batch access, a notebook-based web interface and interactive computing is provided. Instead of writing event loops, the column-based Coffea library is used. In this paper, we describe the architectural components of the facility, the services offered to end-users, and how it integrates into a larger ecosystem for data access and authentication.Comment: Submitted as proceedings fo 25th International Conference on Computing in High-Energy and Nuclear Physics (https://indico.cern.ch/event/948465/

Hadoop distributed file system for the Grid

Data distribution, storage and access are essential to CPU-intensive and data-intensive high performance Grid computing. A newly emerged file system, Hadoop distributed file system (HDFS), is deployed and tested within the Open Science Grid (OSG) middleware stack. Efforts have been taken to integrate HDFS with other Grid tools to build a complete service framework for the Storage Element (SE). Scalability tests show that sustained high inter-DataNode data transfer can be achieved for the cluster fully loaded with data-processing jobs. The WAN transfer to HDFS supported by BeStMan and tuned GridFTP servers shows large scalability and robustness of the system. The hadoop client can be deployed at interactive machines to support remote data access. The ability to automatically replicate precious data is especially important for computing sites, which is demonstrated at the Large Hadron Collider (LHC) computing centers. The simplicity of operations of HDFS-based SE significantly reduces the cost of ownership of Petabyte scale data storage over alternative solutions

Router and Firewall Redundancy with OpenBSD and CARP

As more reliance is placed on computing and networking systems, the need for redundancy increases. The Common Address Redundancy Protocol (CARP) protocol and OpenBSD’s pfsync utility provide a means by which to implement redundant routers and firewalls. This paper details how CARP and pfsync work together to provide this redundancy and explores the performance one can expect from the open source solutions. Two experiments were run: one showing the relationship between firewall state creation and state synchronization traffic and the other showing how TCP sessions are transparently maintained in the event of a router failure. Discussion of these simulations along with background information gives an overview of how OpenBSD, CARP, and pfsync can provide redundant routers and firewalls for today’s Internet

A Survey of Security Issues In Wireless Sensor Networks

Wireless Sensor Networks (WSNs) are used in many applications in military, ecological, and health-related areas. These applications often include the monitoring of sensitive information such as enemy movement on the battlefield or the location of personnel in a building. Security is therefore important in WSNs. However, WSNs suffer from many constraints, including low computation capability, small memory, limited energy resources, susceptibility to physical capture, and the use of insecure wireless communication channels. These constraints make security in WSNs a challenge. In this article we present a survey of security issues in WSNs. First we outline the constraints, security requirements, and attacks with their corresponding countermeasures in WSNs. We then present a holistic view of security issues. These issues are classified into five categories: cryptography, key management, secure routing, secure data aggregation, and intrusion detection. Along the way we highlight the advantages and disadvantages of various WSN security protocols and further compare and evaluate these protocols based on each of these five categories. We also point out the open research issues in each subarea and conclude with possible future research directions on security in WSNs

SNAG: SDN-managed Network Architecture for GridFTP Transfers

Software Defined Networking (SDN) is driving transformations in Research and Education (R&E) networks, enabling innovations in network research, enhancing network performance, and providing security through a policy-driven network management framework. The Holland Computing Center (HCC) at the University of Nebraska-Lincoln (UNL) supports scientists studying large datasets, and has identified a need for flexibility in network management and security, particularly with respect to identifying data flows. This problem is addressed through the deployment of a production SDN with a focus on integrating network resource management for large-scale GridFTP data transfers. We propose SNAG (SDN-managed Network Architecture for GridFTP transfers), an architecture that enables the SDN-based network management of GridFTP file transfers for large-scale science datasets. We also show how SNAG can efficiently and securely identify science dataset transfers from projects such as Compact Muon Solenoid (CMS) and Laser Interferometer Gravitational-Wave Observatory (LIGO). We focus on exposing an Application Program Interface (API) between the trusted GridFTP process and the network layer allowing the network to track flows via application metadata. <br><br>D. Nadig, et al., “SNAG: SDN-managed Network Architecture for GridFTP Transfers,” in INDIS ’16, SLC, Utah, November 2016.<br

SNAG: SDN-managed Network Architecture for GridFTP Transfers

Software Defined Networking (SDN) is driving transformations in Research and Education (R&E) networks, enabling innovations in network research, enhancing network performance, and providing security through a policy-driven network management framework. The Holland Computing Center (HCC) at the University of Nebraska-Lincoln (UNL) supports scientists studying large datasets, and has identified a need for flexibility in network management and security, particularly with respect to identifying data flows. This problem is addressed through the deployment of a production SDN with a focus on integrating network resource management for large-scale GridFTP data transfers. We propose SNAG (SDN-managed Network Architecture for GridFTP transfers), an architecture that enables the SDN-based network management of GridFTP file transfers for large-scale science datasets. We also show how SNAG can efficiently and securely identify science dataset transfers from projects such as Compact Muon Solenoid (CMS) and Laser Interferometer Gravitational-Wave Observatory (LIGO). We focus on exposing an Application Program Interface (API) between the trusted GridFTP process and the network layer allowing the network to track flows via application metadata. <br><br>D. Nadig, et al., “SNAG: SDN-managed Network Architecture for GridFTP Transfers,” in INDIS ’16, SLC, Utah, November 2016.<br

Collaborative Computing Support for Analysis Facilities Exploiting Software as Infrastructure Techniques

Prior to the public release of Kubernetes it was difficult to conduct joint development of elaborate analysis facilities due to the highly non-homogeneous nature of hardware and network topology across compute facilities. However, since the advent of systems like Kubernetes and OpenShift, which provide declarative interfaces for building fault-tolerant and self-healing deployments of networked software, it is possible for multiple institutes to collaborate more effectively since resource details are abstracted away through various forms of hardware and software virtualization. In this whitepaper we will outline the development of two analysis facilities: “Coffea-casa” at University of Nebraska Lincoln and the “Elastic Analysis Facility” at Fermilab, and how utilizing platform abstraction has improved the development of common software for each of these facilities, and future development plans made possible by this methodology