Experiences Building Globus Genomics: A Next-Generation Sequencing Analysis Service using Galaxy, Globus, and Amazon Web Services

ABSTRACT We describe Globus Genomics, a system that we have developed for rapid analysis of large quantities of next-generation sequencing (NGS) genomic data. This system achieves a high degree of end-to-end automation that encompasses every stage of data analysis including initial data retrieval from remote sequencing centers or storage (via the Globus file transfer system); specification, configuration, and reuse of multi-step processing pipelines (via the Galaxy workflow system); creation of custom Amazon Machine Images and on-demand resource acquisition via a specialized elastic provisioner (on Amazon EC2); and efficient scheduling of these pipelines over many processors (via the HTCondor scheduler). The system allows biomedical researchers to perform rapid analysis of large NGS datasets in a fully automated manner, without software installation or a need for any local computing infrastructure. We report performance and cost results for some representative workloads

ASYMPTOTIC MODELS OF THE HEAT TRANSFER IN LAMINATED CONDUCTORS

International audienceLaminated materials play an important role in civil engineering. The contribution is focused on the modelling of heat conduction in these materials. The analysis is carried out in the framework of the tolerance averaging technique, [1]. A new asymptotic procedure for finding solutions to the specific heat conduction problems is proposed. General results are illustrated by some numerical examples and compared with those derived from homogenization technique, [2]

E3SM-Project/e3sm_to_cmip: v1.10.0rc3

What's Changed Refactor Refactor the API design for cmor_handlers by @tomvothecoder in https://github.com/E3SM-Project/e3sm_to_cmip/pull/140 CMOR Handlers are now defined in a single yaml file under cmor_handlers/handlers.yaml Visit the documentation for more information on this change. Documentation Add example script by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_to_cmip/pull/174 Update README.md by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_to_cmip/pull/180 Remove CWL workflow docs and update conda env dependencies by @tomvothecoder in https://github.com/E3SM-Project/e3sm_to_cmip/pull/193 CWL scripts and documentation have been moved to the datasm repo since that is where they are primarily used. CWL scripts were never included in the conda package and therefore probably never used by any other developer or repo. Bug Fixes Add os.markdirs() for /logs directory by @tomvothecoder in https://github.com/E3SM-Project/e3sm_to_cmip/pull/187 Fix deprecated numpy.warnings by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_to_cmip/pull/201 Update KeyError raised for missing handlers or non-derivable variables to logger warning by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_to_cmip/pull/203 Fix handling sftlf/LANDFRAC by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_to_cmip/pull/209 add new formula for rsut and rsutcs by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_to_cmip/pull/210 Full Changelog: https://github.com/E3SM-Project/e3sm_to_cmip/compare/v1.9.1...v1.10.0rc

E3SM-Project/e3sm_to_cmip: v1.10.0

What's Changed Refactor Refactor the API design for cmor_handlers by @tomvothecoder in https://github.com/E3SM-Project/e3sm_to_cmip/pull/140 CMOR Handlers are now defined in a single yaml file under cmor_handlers/handlers.yaml Visit the documentation for more information on this change. Documentation Add example script by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_to_cmip/pull/174 Update README.md by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_to_cmip/pull/180 Remove CWL workflow docs and update conda env dependencies by @tomvothecoder in https://github.com/E3SM-Project/e3sm_to_cmip/pull/193 CWL scripts and documentation have been moved to the datasm repo since that is where they are primarily used. CWL scripts were never included in the conda package and therefore probably never used by any other developer or repo. Bug Fixes Add os.markdirs() for /logs directory by @tomvothecoder in https://github.com/E3SM-Project/e3sm_to_cmip/pull/187 Fix deprecated numpy.warnings by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_to_cmip/pull/201 Update KeyError raised for missing handlers or non-derivable variables to logger warning by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_to_cmip/pull/203 Fix handling sftlf/LANDFRAC by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_to_cmip/pull/209 add new formula for rsut and rsutcs by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_to_cmip/pull/210 Full Changelog: https://github.com/E3SM-Project/e3sm_to_cmip/compare/v1.9.1...v1.10.

Coordinating an operational data distribution network for CMIP6 data

The distribution of data contributed to the Coupled Model Intercomparison Project Phase 6 (CMIP6) is via the Earth System Grid Federation (ESGF). The ESGF is a network of internationally distributed sites that together work as a federated data archive. Data records from climate modelling institutes are published to the ESGF and then shared around the world. It is anticipated that CMIP6 will produce approximately 20 PB of data to be published and distributed via the ESGF. In addition to this large volume of data a number of value-added CMIP6 services are required to interact with the ESGF; for example the citation and errata services both interact with the ESGF but are not a core part of its infrastructure. With a number of interacting services and a large volume of data anticipated for CMIP6, the CMIP Data Node Operations Team (CDNOT) was formed. The CDNOT coordinated and implemented a series of CMIP6 preparation data challenges to test all the interacting components in the ESGF CMIP6 software ecosystem. This ensured that when CMIP6 data were released they could be reliably distributed. No. DE-ACO2-05CH11231 and authors at Lawrence Livermore National Laboratory (LLNL) under contract DE-AC52-07NA27344 with the U.S. Department of Energy. The United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a non-exclusive, paid-up, irrevocable, world-wide license to publish or reproduce the published form of this manuscript, or allow others to do so, for United States Government purposes. The Department of Energy will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan (http://energy.gov/downloads/doe-public-access-plan).Funding Agencies|US Department of EnergyUnited States Department of Energy (DOE) [DE-AC02-05CH11231, DE-AC52-07NA27344]; European UnionEuropean Commission [824084]; French National Research Agency project CONVERGENCEFrench National Research Agency (ANR) [ANR-13-MONU-0008-02]; National Collaborative Research Infrastructure Strategy (NCRIS)-funded National Computational Infrastructure (NCI) Australia; Australian Research Data Commons (ARDC)</p