2,250 research outputs found
A Taxonomy of Workflow Management Systems for Grid Computing
With the advent of Grid and application technologies, scientists and
engineers are building more and more complex applications to manage and process
large data sets, and execute scientific experiments on distributed resources.
Such application scenarios require means for composing and executing complex
workflows. Therefore, many efforts have been made towards the development of
workflow management systems for Grid computing. In this paper, we propose a
taxonomy that characterizes and classifies various approaches for building and
executing workflows on Grids. We also survey several representative Grid
workflow systems developed by various projects world-wide to demonstrate the
comprehensiveness of the taxonomy. The taxonomy not only highlights the design
and engineering similarities and differences of state-of-the-art in Grid
workflow systems, but also identifies the areas that need further research.Comment: 29 pages, 15 figure
Service composition in stochastic settings
With the growth of the Internet-of-Things and online Web services, more services with more capabilities are available to us. The ability to generate new, more useful services from existing ones has been the focus of much research for over a decade. The goal is, given a specification of the behavior of the target service, to build a controller, known as an orchestrator, that uses existing services to satisfy the requirements of the target service. The model of services and requirements used in most work is that of a finite state machine. This implies that the specification can either be satisfied or not, with no middle ground. This is a major drawback, since often an exact solution cannot be obtained. In this paper we study a simple stochastic model for service composition: we annotate the tar- get service with probabilities describing the likelihood of requesting each action in a state, and rewards for being able to execute actions. We show how to solve the resulting problem by solving a certain Markov Decision Process (MDP) derived from the service and requirement specifications. The solution to this MDP induces an orchestrator that coincides with the exact solution if a composition exists. Otherwise it provides an approximate solution that maximizes the expected sum of values of user requests that can be serviced. The model studied although simple shades light on composition in stochastic settings and indeed we discuss several possible extensions
Quality measures for ETL processes: from goals to implementation
Extraction transformation loading (ETL) processes play an increasingly important role for the support of modern business operations. These business processes are centred around artifacts with high variability and diverse lifecycles, which correspond to key business entities. The apparent complexity of these activities has been examined through the prism of business process management, mainly focusing on functional requirements and performance optimization. However, the quality dimension has not yet been thoroughly investigated, and there is a need for a more human-centric approach to bring them closer to business-users requirements. In this paper, we take a first step towards this direction by defining a sound model for ETL process quality characteristics and quantitative measures for each characteristic, based on existing literature. Our model shows dependencies among quality characteristics and can provide the basis for subsequent analysis using goal modeling techniques. We showcase the use of goal modeling for ETL process design through a use case, where we employ the use of a goal model that includes quantitative components (i.e., indicators) for evaluation and analysis of alternative design decisions.Peer ReviewedPostprint (author's final draft
Modeling the controlled delivery power grid
Competitive energy markets, stricter regulation, and the integration of distributed renewable energy sources are forcing companies to reengineer energy production and distribution. The Controlled Delivery Power Grid is proposed as a novel approach to transport energy from generators to consumers. In this approach, energy distribution is performed in an asynchronous and distributed fashion. Much like the Internet, energy is delivered as addressable packets, which allow a controlled delivery of energy.
As a proof-of-concept of the controllable delivery grid, two experimental test beds, one with integrated energy storage and another with no energy storage, were designed and built to evaluate the efficiency of a power distribution and scheduling scheme. Both test beds use a request-grant protocol where energy is supplied in discrete quantities. The performance of the system is measured in terms of the ability to satisfy requests from consumers. The results show high satisfaction ratios for distribution capacities that are smaller than the maximum demand.
The distribution of energy is modelled with graph theory and as an Integer Linear Programming problem to minimize transmission losses and determine routes for energy flows in a network with distributed sources and consumers. The obtained results are compared with a heuristic approach based on the Dijkstra\u27s shortest path algorithm, which is proposed as a feasible approach to routing the transmission of packetized energy
Towards Power- and Energy-Efficient Datacenters
As the Internet evolves, cloud computing is now a dominant form of computation in modern lives. Warehouse-scale computers (WSCs), or datacenters, comprising the foundation of this cloud-centric web have been able to deliver satisfactory performance to both the Internet companies and the customers. With the increased focus and popularity of the cloud, however, datacenter loads rise and grow rapidly, and Internet companies are in need of boosted computing capacity to serve such demand. Unfortunately, power and energy are often the major limiting factors prohibiting datacenter growth: it is often the case that no more servers can be added to datacenters without surpassing the capacity of the existing power infrastructure.
This dissertation aims to investigate the issues of power and energy usage in a modern datacenter environment. We identify the source of power and energy inefficiency at three levels in a modern datacenter environment and provides insights and solutions to address each of these problems, aiming to prepare datacenters for critical future growth. We start at the datacenter-level and find that the peak provisioning and improper service placement in multi-level power delivery infrastructures fragment the power budget inside production datacenters, degrading the compute capacity the existing infrastructure can support. We find that the heterogeneity among datacenter workloads is key to address this issue and design systematic methods to reduce the fragmentation and improve the utilization of the power budget. This dissertation then narrow the focus to examine the energy usage of individual servers running cloud workloads. Especially, we examine the power management mechanisms employed in these servers and find that the coarse time granularity of these mechanisms is one critical factor that leads to excessive energy consumption. We propose an intelligent and low overhead solution on top of the emerging finer granularity voltage/frequency boosting circuit to effectively pinpoints and boosts queries that are likely to increase the tail distribution and can reap more benefit from the voltage/frequency boost, improving energy efficiency without sacrificing the quality of services. The final focus of this dissertation takes a further step to investigate how using a fundamentally more efficient computing substrate, field programmable gate arrays (FPGAs), benefit datacenter power and energy efficiency. Different from other types of hardware accelerations, FPGAs can be reconfigured on-the-fly to provide fine-grain control over hardware resource allocation and presents a unique set of challenges for optimal workload scheduling and resource allocation. We aim to design a set coordinated algorithms to manage these two key factors simultaneously and fully explore the benefit of deploying FPGAs in the highly varying cloud environment.PHDComputer Science & EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/144043/1/hsuch_1.pd
Dynamic Assembly for System Adaptability, Dependability, and Assurance
(DASASA) ProjectAuthor-contributed print ite
Mitigating the Effects of Partial Resource Failures for Cloud Providers
Competition for users on a global market is fierce, forcing enterprises to provide for better, faster services while offering the same more cheaply. At the same time, users choose to remain oblivious of the infrastructure behind the service – only demanding that it works. Cloud service failures and inefficient management of such failures can result in significant financial cost, loss of reputation for providers, and drive key customers away. At the same time failure situations can never be completely avoided. To mitigate their effects we present a decision model for providers to help them decide which jobs to keep running and which to cancel in order to minimize loss of revenue and key customers during partial resource failures. The results of the evaluation of the model and its extension show its ability to significantly improve revenue. Furthermore the model can also help to reduce the number of cancelled jobs
Time-bounded distributed QoS-aware service configuration in heterogeneous cooperative environments
The scarcity and diversity of resources among the devices of heterogeneous computing
environments may affect their ability to perform services with specific Quality
of Service constraints, particularly in dynamic distributed environments where the
characteristics of the computational load cannot always be predicted in advance.
Our work addresses this problem by allowing resource constrained devices to cooperate
with more powerful neighbour nodes, opportunistically taking advantage
of global distributed resources and processing power. Rather than assuming that
the dynamic configuration of this cooperative service executes until it computes
its optimal output, the paper proposes an anytime approach that has the ability
to tradeoff deliberation time for the quality of the solution. Extensive simulations
demonstrate that the proposed anytime algorithms are able to quickly find a good
initial solution and effectively optimise the rate at which the quality of the current
solution improves at each iteration, with an overhead that can be considered
negligible
Recommended from our members
QoS - Aware content oriented flow routing in optical computer network
This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.In this thesis, one of the most important issues in the field of networks communication is tackled and addressed. This issue is represented by QoS, where the increasing demand on highquality
applications together with the fast increase in the rates of Internet users have led to
massive traffic being transmitted on the Internet. This thesis proposes new ideas to manage the flow of this huge traffic in a manner that contributes in improving the communication QoS. This can be achieved by replacing the conventional application-insensitive routing schemes by others
which take into account the type of applications when making the routing decision. As a first contribution, the effect on the potential development in the quality of experience on the loading of
Basra optical network has been investigated. Furthermore, the traffic due to each application was dealt with in different ways according to their delay and loss sensitivities. Load rate distributions
over the various links due to the different applications were deployed to investigate the places of possible congestions in the network and the dominant applications that cause such congestions. In addition, OpenFlow and Optica Burst Switching (OBS) techniques were used to provide a wider range of network controllability and management. A centralised routing protocol
that takes into account the available bandwidth, delay, and security as three important QoS parameters, when forwarding traffics of different types, was proposed and implemented using OMNeT++ networks simulator. As a novel idea, security has been incorporated in our QoS requirements by incorporating Oyster Optics Technology (OOT) to secure some of the optical links aiming to supply the network with some secure paths for those applications that have high
privacy requirements. A particular type of traffic is to be routed according to the importance of these three QoS parameters for such a traffic type. The link utilisation, end to end delays and securities due to the different applications were recorded to prove the feasibility of our proposed
system. In order to decrease the amount of traffic overhead, the same QoS constraints were implemented on a distributed Ant colony based routing. The traditional Ant routing protocol was improved by adopting the idea of Red-Green-Blue (RGB) pheromones routing to incorporate these QoS constraints. Improvements of 11% load balancing, and 9% security for private data was achieved compared to the conventional Ant routing techniques. In addition, this Ant based
routing was utilised to propose an improved solution for the routing and wavelength assignment problem in the WDM optical computer networks
Reputation-guided Evolutionary Scheduling Algorithm for Independent Tasks in inter-Clouds Environments
Self-adaptation provides software with flexibility to different behaviours (configurations) it incorporates and the (semi-) autonomous ability to switch between these behaviours in response to changes. To empower clouds with the ability to capture and respond to quality feedback provided by users at runtime, we propose a reputation guided genetic scheduling algorithm for independent tasks. Current resource management services consider evolutionary strategies to improve the performance on resource allocation procedures or tasks scheduling algorithms, but they fail to consider the user as part of the scheduling process. Evolutionary computing offers different methods to find a near-optimal solution. In this paper we extended previous work with new optimisation heuristics for the problem of scheduling. We show how reputation is considered as an optimisation metric, and analyse how our metrics can be considered as upper bounds for others in the optimisation algorithm. By experimental comparison, we show our techniques can lead to optimised results.Peer Reviewe
- …