We are in the midst of a transition in the way that computing resources are obtained in the sciences. Cloud-based architectures play an increasing role in scientific processing as well as scientific communication, leveraging the considerable technical investment in these platforms. Typically astronomers have believed that the economics drive them away from this model, however these calculations often fall short of the total cost of delivering computing infrastructure. Moreover there is often
a false dichotomy between commercial cloud services (like Amazon Web Services) and traditional bare-metal “special snowflake” hardware; privately deployed clouds (such as university OpenStack clusters) represent a path forward with many of the technological advantages of working in an open standardized infrastructure while avoiding some of the direct costs of the commercial clouds. Here we describe how we have navigated some of these issues in the context of engineering the LSST Data Management’s developer infrastructure, which includes a cross-platform cloud-based continuous integration architecture
