1 research outputs found
Managing Cloud networking costs for data-intensive applications by provisioning dedicated network links
Many scientific high-throughput applications can benefit from the elastic
nature of Cloud resources, especially when there is a need to reduce time to
completion. Cost considerations are usually a major issue in such endeavors,
with networking often a major component; for data-intensive applications,
egress networking costs can exceed the compute costs. Dedicated network links
provide a way to lower the networking costs, but they do add complexity. In
this paper we provide a description of a 100 fp32 PFLOPS Cloud burst in support
of IceCube production compute, that used Internet2 Cloud Connect service to
provision several logically-dedicated network links from the three major Cloud
providers, namely Amazon Web Services, Microsoft Azure and Google Cloud
Platform, that in aggregate enabled approximately 100 Gbps egress capability to
on-prem storage. It provides technical details about the provisioning process,
the benefits and limitations of such a setup and an analysis of the costs
incurred.Comment: 8 pages, 7 figures, 4 tables, to be published in proceedings of
PEARC2