16,051 research outputs found
Managing Uncertainty: A Case for Probabilistic Grid Scheduling
The Grid technology is evolving into a global, service-orientated
architecture, a universal platform for delivering future high demand
computational services. Strong adoption of the Grid and the utility computing
concept is leading to an increasing number of Grid installations running a wide
range of applications of different size and complexity. In this paper we
address the problem of elivering deadline/economy based scheduling in a
heterogeneous application environment using statistical properties of job
historical executions and its associated meta-data. This approach is motivated
by a study of six-month computational load generated by Grid applications in a
multi-purpose Grid cluster serving a community of twenty e-Science projects.
The observed job statistics, resource utilisation and user behaviour is
discussed in the context of management approaches and models most suitable for
supporting a probabilistic and autonomous scheduling architecture
Task Runtime Prediction in Scientific Workflows Using an Online Incremental Learning Approach
Many algorithms in workflow scheduling and resource provisioning rely on the
performance estimation of tasks to produce a scheduling plan. A profiler that
is capable of modeling the execution of tasks and predicting their runtime
accurately, therefore, becomes an essential part of any Workflow Management
System (WMS). With the emergence of multi-tenant Workflow as a Service (WaaS)
platforms that use clouds for deploying scientific workflows, task runtime
prediction becomes more challenging because it requires the processing of a
significant amount of data in a near real-time scenario while dealing with the
performance variability of cloud resources. Hence, relying on methods such as
profiling tasks' execution data using basic statistical description (e.g.,
mean, standard deviation) or batch offline regression techniques to estimate
the runtime may not be suitable for such environments. In this paper, we
propose an online incremental learning approach to predict the runtime of tasks
in scientific workflows in clouds. To improve the performance of the
predictions, we harness fine-grained resources monitoring data in the form of
time-series records of CPU utilization, memory usage, and I/O activities that
are reflecting the unique characteristics of a task's execution. We compare our
solution to a state-of-the-art approach that exploits the resources monitoring
data based on regression machine learning technique. From our experiments, the
proposed strategy improves the performance, in terms of the error, up to
29.89%, compared to the state-of-the-art solutions.Comment: Accepted for presentation at main conference track of 11th IEEE/ACM
International Conference on Utility and Cloud Computin
Recommended from our members
Project Controls and Management Systems : current practice and how it has changed over the past decade
Project Controls and Management System (PCMS) refers to an ecosystem of processes, tools and personnel required for the proper planning and execution of capital projects throughout the different phases of design, procurement, construction and startup. This can be divided into different focus areas (functions) that would include Estimating, Planning, Scheduling, Cost Control, Change Management, Progressing, and Forecasting. Various trends such as globalization, contractor specialization and information technology developments have impacted the way PCMS are implemented and made it the subject of extensive research over the past years to investigate how to best utilize those trends. Replicating the research methodology used in a 2011 report published by the Construction Research Institute (CII), this work aims to investigate the current status of PCMS implementation and how it has changed over the past decade. It was concluded that while the original PCMS principles are still valid, adoption has drastically changed in terms of efficiency for the majority of the functions. The research also identifies areas of potential concerns and provides recommendations for further improvement.Civil, Architectural, and Environmental Engineerin
- …