Leveraging Your Local Resources and National Cyberinfrastructure Resources without Tears

Compute resources for conducting research inhabit a wide range from researchers' personal computers, servers in labs, campus clusters and condos, regional resource-sharing models, and national cyberinfrastructure. Researchers agree that there are not enough resources available on a broad scale, and significant barriers exist for getting analyses moved from smaller- to larger-scale cyberinfrastructure. The XSEDE Campus Bridging program disseminates a several tools that assist researchers and campus IT administrators in reducing barriers to the effective use of national cyberinfrastructure for research. Tools for data management, job submission and steering, best practices for building and administering clusters, and common documentation and training activities all support a flexible environment that allows cyberinfrastructure to be as simple to utilize as a plug-and-play peripheral. In this paper and the accompanying poster we provide an overview of Campus Bridging, including specific challenges and solutions to the problem of making the computerized parts of research easier. We focus particularly on tools that facilitate management of campus computing clusters and integration of such clusters with the national cyberinfrastructure

XSEDE Data Management Use Cases L3 Architectural Response

This document is the Level 3 Architectural response for the XSEDE Data Management Use Cases.National Science Foundation OCI-1053575Ope

XSEDE Scientific Workflow Use Case L3 Architectural Response

This document is the Level 3 Architectural Response for the XSEDE Scientific Workflow Use Case.National Science Foundation OCI-1053575Ope

Enabling Scalable Data Processing and Management through Standards-based Job Execution and the Global Federated File System

Emerging challenges for scientific communities are to efficiently process big data obtained by experimentation and computational simulations. Supercomputing architectures are available to support scalable and high performant processing environment, but many of the existing algorithm implementations are still unable to cope with its architectural complexity. One approach is to have innovative technologies that effectively use these resources and also deal with geographically dispersed large datasets. Those technologies should be accessible in a way that data scientists who are running data intensive computations do not have to deal with technical intricacies of the underling execution system. Our work primarily focuses on providing data scientists with transparent access to these resources in order to easily analyze data. Impact of our work is given by describing how we enabled access to multiple high performance computing resources through an open standards-based middleware that takes advantage of a unified data management provided by the the Global Federated File System. Our architectural design and its associated implementation is validated by a usecase that requires massivley parallel DBSCAN outlier detection on a 3D point clouds dataset

XSEDE High Throughput Computing Use Case L3 Architectural Response

This document is the Level 3 Architectural Response for the XSEDE High Throughput Computing use case.National Science Foundation OCI-1053575Ope

Enabling Scalable Data Processing and Management through Standards-based Job Execution and the Global Federated File System

Emerging challenges for scientific communities are to efficiently process big data obtained by experimentation and computational simulations. Supercomputing architectures are available to support scalable and high performant processing environment, but many of the existing algorithm implementations are still unable to cope with its architectural complexity. One approach is to have innovative technologies that effectively use these resources and also deal with geographically dispersed large datasets. Those technologies should be accessible in a way that data scientists who are running data intensive computations do not have to deal with technical intricacies of the underling execution system. Our work primarily focuses on providing data scientists with transparent access to these resources in order to easily analyze data. Impact of our work is given by describing how we enabled access to multiple high performance computing resources through an open standards-based middleware that takes advantage of a unified data management provided by the the Global Federated File System. Our architectural design and its associated implementation is validated by a usecase that requires massivley parallel DBSCAN outlier detection on a 3D point clouds dataset.Accepte