110,838 research outputs found
Uncertainty-Aware Workload Prediction in Cloud Computing
Predicting future resource demand in Cloud Computing is essential for
managing Cloud data centres and guaranteeing customers a minimum Quality of
Service (QoS) level. Modelling the uncertainty of future demand improves the
quality of the prediction and reduces the waste due to overallocation. In this
paper, we propose univariate and bivariate Bayesian deep learning models to
predict the distribution of future resource demand and its uncertainty. We
design different training scenarios to train these models, where each procedure
is a different combination of pretraining and fine-tuning steps on multiple
datasets configurations. We also compare the bivariate model to its univariate
counterpart training with one or more datasets to investigate how different
components affect the accuracy of the prediction and impact the QoS. Finally,
we investigate whether our models have transfer learning capabilities.
Extensive experiments show that pretraining with multiple datasets boosts
performances while fine-tuning does not. Our models generalise well on related
but unseen time series, proving transfer learning capabilities. Runtime
performance analysis shows that the models are deployable in real-world
applications. For this study, we preprocessed twelve datasets from real-world
traces in a consistent and detailed way and made them available to facilitate
the research in this field
Towards Data-Driven Autonomics in Data Centers
Continued reliance on human operators for managing data centers is a major
impediment for them from ever reaching extreme dimensions. Large computer
systems in general, and data centers in particular, will ultimately be managed
using predictive computational and executable models obtained through
data-science tools, and at that point, the intervention of humans will be
limited to setting high-level goals and policies rather than performing
low-level operations. Data-driven autonomics, where management and control are
based on holistic predictive models that are built and updated using generated
data, opens one possible path towards limiting the role of operators in data
centers. In this paper, we present a data-science study of a public Google
dataset collected in a 12K-node cluster with the goal of building and
evaluating a predictive model for node failures. We use BigQuery, the big data
SQL platform from the Google Cloud suite, to process massive amounts of data
and generate a rich feature set characterizing machine state over time. We
describe how an ensemble classifier can be built out of many Random Forest
classifiers each trained on these features, to predict if machines will fail in
a future 24-hour window. Our evaluation reveals that if we limit false positive
rates to 5%, we can achieve true positive rates between 27% and 88% with
precision varying between 50% and 72%. We discuss the practicality of including
our predictive model as the central component of a data-driven autonomic
manager and operating it on-line with live data streams (rather than off-line
on data logs). All of the scripts used for BigQuery and classification analyses
are publicly available from the authors' website.Comment: 12 pages, 6 figure
Towards Operator-less Data Centers Through Data-Driven, Predictive, Proactive Autonomics
Continued reliance on human operators for managing data centers is a major
impediment for them from ever reaching extreme dimensions. Large computer
systems in general, and data centers in particular, will ultimately be managed
using predictive computational and executable models obtained through
data-science tools, and at that point, the intervention of humans will be
limited to setting high-level goals and policies rather than performing
low-level operations. Data-driven autonomics, where management and control are
based on holistic predictive models that are built and updated using live data,
opens one possible path towards limiting the role of operators in data centers.
In this paper, we present a data-science study of a public Google dataset
collected in a 12K-node cluster with the goal of building and evaluating
predictive models for node failures. Our results support the practicality of a
data-driven approach by showing the effectiveness of predictive models based on
data found in typical data center logs. We use BigQuery, the big data SQL
platform from the Google Cloud suite, to process massive amounts of data and
generate a rich feature set characterizing node state over time. We describe
how an ensemble classifier can be built out of many Random Forest classifiers
each trained on these features, to predict if nodes will fail in a future
24-hour window. Our evaluation reveals that if we limit false positive rates to
5%, we can achieve true positive rates between 27% and 88% with precision
varying between 50% and 72%.This level of performance allows us to recover
large fraction of jobs' executions (by redirecting them to other nodes when a
failure of the present node is predicted) that would otherwise have been wasted
due to failures. [...
Autonomic Cloud Computing: Open Challenges and Architectural Elements
As Clouds are complex, large-scale, and heterogeneous distributed systems,
management of their resources is a challenging task. They need automated and
integrated intelligent strategies for provisioning of resources to offer
services that are secure, reliable, and cost-efficient. Hence, effective
management of services becomes fundamental in software platforms that
constitute the fabric of computing Clouds. In this direction, this paper
identifies open issues in autonomic resource provisioning and presents
innovative management techniques for supporting SaaS applications hosted on
Clouds. We present a conceptual architecture and early results evidencing the
benefits of autonomic management of Clouds.Comment: 8 pages, 6 figures, conference keynote pape
- …