4,440 research outputs found
Predicting Intermediate Storage Performance for Workflow Applications
Configuring a storage system to better serve an application is a challenging
task complicated by a multidimensional, discrete configuration space and the
high cost of space exploration (e.g., by running the application with different
storage configurations). To enable selecting the best configuration in a
reasonable time, we design an end-to-end performance prediction mechanism that
estimates the turn-around time of an application using storage system under a
given configuration. This approach focuses on a generic object-based storage
system design, supports exploring the impact of optimizations targeting
workflow applications (e.g., various data placement schemes) in addition to
other, more traditional, configuration knobs (e.g., stripe size or replication
level), and models the system operation at data-chunk and control message
level.
This paper presents our experience to date with designing and using this
prediction mechanism. We evaluate this mechanism using micro- as well as
synthetic benchmarks mimicking real workflow applications, and a real
application.. A preliminary evaluation shows that we are on a good track to
meet our objectives: it can scale to model a workflow application run on an
entire cluster while offering an over 200x speedup factor (normalized by
resource) compared to running the actual application, and can achieve, in the
limited number of scenarios we study, a prediction accuracy that enables
identifying the best storage system configuration
Multi-layered simulations at the heart of workflow enactment on clouds
Scientific workflow systems face new challenges when supporting Cloud computing, as the information on the state of the used infrastructures is much less detailed than before. Thus, organising virtual infrastructures in a way that not only supports the workflow execution but also optimises it for several service level objectives (e.g. maximum energy consumption limit, cost, reliability, availability) become reliant on good Cloud modelling and prediction information. While simulators were successfully aiding research on such workflow management systems, the currently available Cloud related simulation toolkits suffer from several issues (e.g. scalability and narrow scope) that hinder their applicability. To address these issues, this article introduces techniques for unifying two existing simulation toolkits by first analysing the problems with the current simulators, and then by illustrating the problems faced by workflow systems. We use for this purpose the example of the ASKALON environment, a scientific workflow composition and execution tool for cloud and grid environments. We illustrate the advantages of a workflow system with directly integrated simulation back-end and how the unification of the selected simulators does not affect the overall workflow execution simulation performance. Copyright © 2015 John Wiley & Sons, Ltd
MOLNs: A cloud platform for interactive, reproducible and scalable spatial stochastic computational experiments in systems biology using PyURDME
Computational experiments using spatial stochastic simulations have led to
important new biological insights, but they require specialized tools, a
complex software stack, as well as large and scalable compute and data analysis
resources due to the large computational cost associated with Monte Carlo
computational workflows. The complexity of setting up and managing a
large-scale distributed computation environment to support productive and
reproducible modeling can be prohibitive for practitioners in systems biology.
This results in a barrier to the adoption of spatial stochastic simulation
tools, effectively limiting the type of biological questions addressed by
quantitative modeling. In this paper, we present PyURDME, a new, user-friendly
spatial modeling and simulation package, and MOLNs, a cloud computing appliance
for distributed simulation of stochastic reaction-diffusion models. MOLNs is
based on IPython and provides an interactive programming platform for
development of sharable and reproducible distributed parallel computational
experiments
Simulating IoT Workflows in DISSECT-CF-Fog
The modelling of IoT applications utilising the resources of cloud and fog computing is not straightforward because they have to support various trigger-based events that make human life easier. The sequence of tasks, such as performing a service call, receiving a data packet in the form of a message sent by an IoT device, and managing actuators or executing a computational task on a virtual machine, are often associated with and composed of IoT workflows. The development and deployment of such IoT workflows and their management systems in real life, including communication and network operations, can be complicated due to high operation costs and access limitations. Therefore, simulation solutions are often applied for such purposes. In this paper, we introduce a novel simulator extension of the DISSECT-CF-Fog simulator that leverages the workflow scheduling and its execution capabilities to model real-life IoT use cases. We also show that state-of-the-art simulators typically omit the IoT factor in the case of the scientific workflow evaluation. Therefore, we present a scalability study focusing on scientific workflows and on the interoperability of scientific and IoT workflows in DISSECT-CF-Fog
Fostering energy-awareness in simulations behind scientific workflow management systems
© 2014 IEEE.Scientific workflow management systems face a new challenge in the era of cloud computing. The past availability of rich information regarding the state of the used infrastructures is gone. Thus, organising virtual infrastructures so that they not only support the workflow being executed, but also optimise for several service level objectives (e.g., Maximum energy consumption limit, cost, reliability, availability) become dependent on good infrastructure modelling and prediction techniques. While simulators have been successfully used in the past to aid research on such workflow management systems, the currently available cloud related simulation toolkits suffer form several issues (e.g., Scalability, narrow scope) that hinder their applicability. To address this need, this paper introduces techniques for unifying two existing simulation toolkits by first analysing the problems with the current simulators, and then by illustrating the problems faced by workflow systems through the example of the ASKALON environment. Finally, we show how the unification of the selected simulators improve on the the discussed problems
Modeling the Internet of Things: a simulation perspective
This paper deals with the problem of properly simulating the Internet of
Things (IoT). Simulating an IoT allows evaluating strategies that can be
employed to deploy smart services over different kinds of territories. However,
the heterogeneity of scenarios seriously complicates this task. This imposes
the use of sophisticated modeling and simulation techniques. We discuss novel
approaches for the provision of scalable simulation scenarios, that enable the
real-time execution of massively populated IoT environments. Attention is given
to novel hybrid and multi-level simulation techniques that, when combined with
agent-based, adaptive Parallel and Distributed Simulation (PADS) approaches,
can provide means to perform highly detailed simulations on demand. To support
this claim, we detail a use case concerned with the simulation of vehicular
transportation systems.Comment: Proceedings of the IEEE 2017 International Conference on High
Performance Computing and Simulation (HPCS 2017
Workflow scheduling for service oriented cloud computing
Service Orientation (SO) and grid computing are two computing paradigms that when put together using Internet technologies promise to provide a scalable yet flexible computing platform for a diverse set of distributed computing applications. This practice gives rise to the notion of a computing cloud that addresses some previous limitations of interoperability, resource sharing and utilization within distributed computing. In such a Service Oriented Computing Cloud (SOCC), applications are formed by composing a set of services together. In addition, hierarchical service layers are also possible where general purpose services at lower layers are composed to deliver more domain specific services at the higher layer. In general an SOCC is a horizontally scalable computing platform that offers its resources as services in a standardized fashion. Workflow based applications are a suitable target for SOCC where workflow tasks are executed via service calls within the cloud. One or more workflows can be deployed over an SOCC and their execution requires scheduling of services to workflow tasks as the task become ready following their interdependencies. In this thesis heuristics based scheduling policies are evaluated for scheduling workflows over a collection of services offered by the SOCC. Various execution scenarios and workflow characteristics are considered to understand the implication of the heuristic based workflow scheduling
On the Fly Orchestration of Unikernels: Tuning and Performance Evaluation of Virtual Infrastructure Managers
Network operators are facing significant challenges meeting the demand for
more bandwidth, agile infrastructures, innovative services, while keeping costs
low. Network Functions Virtualization (NFV) and Cloud Computing are emerging as
key trends of 5G network architectures, providing flexibility, fast
instantiation times, support of Commercial Off The Shelf hardware and
significant cost savings. NFV leverages Cloud Computing principles to move the
data-plane network functions from expensive, closed and proprietary hardware to
the so-called Virtual Network Functions (VNFs). In this paper we deal with the
management of virtual computing resources (Unikernels) for the execution of
VNFs. This functionality is performed by the Virtual Infrastructure Manager
(VIM) in the NFV MANagement and Orchestration (MANO) reference architecture. We
discuss the instantiation process of virtual resources and propose a generic
reference model, starting from the analysis of three open source VIMs, namely
OpenStack, Nomad and OpenVIM. We improve the aforementioned VIMs introducing
the support for special-purpose Unikernels and aiming at reducing the duration
of the instantiation process. We evaluate some performance aspects of the VIMs,
considering both stock and tuned versions. The VIM extensions and performance
evaluation tools are available under a liberal open source licence
- …