1,111 research outputs found
Datacenter Traffic Control: Understanding Techniques and Trade-offs
Datacenters provide cost-effective and flexible access to scalable compute
and storage resources necessary for today's cloud computing needs. A typical
datacenter is made up of thousands of servers connected with a large network
and usually managed by one operator. To provide quality access to the variety
of applications and services hosted on datacenters and maximize performance, it
deems necessary to use datacenter networks effectively and efficiently.
Datacenter traffic is often a mix of several classes with different priorities
and requirements. This includes user-generated interactive traffic, traffic
with deadlines, and long-running traffic. To this end, custom transport
protocols and traffic management techniques have been developed to improve
datacenter network performance.
In this tutorial paper, we review the general architecture of datacenter
networks, various topologies proposed for them, their traffic properties,
general traffic control challenges in datacenters and general traffic control
objectives. The purpose of this paper is to bring out the important
characteristics of traffic control in datacenters and not to survey all
existing solutions (as it is virtually impossible due to massive body of
existing research). We hope to provide readers with a wide range of options and
factors while considering a variety of traffic control mechanisms. We discuss
various characteristics of datacenter traffic control including management
schemes, transmission control, traffic shaping, prioritization, load balancing,
multipathing, and traffic scheduling. Next, we point to several open challenges
as well as new and interesting networking paradigms. At the end of this paper,
we briefly review inter-datacenter networks that connect geographically
dispersed datacenters which have been receiving increasing attention recently
and pose interesting and novel research problems.Comment: Accepted for Publication in IEEE Communications Surveys and Tutorial
Dominant Resource Fairness in Cloud Computing Systems with Heterogeneous Servers
We study the multi-resource allocation problem in cloud computing systems
where the resource pool is constructed from a large number of heterogeneous
servers, representing different points in the configuration space of resources
such as processing, memory, and storage. We design a multi-resource allocation
mechanism, called DRFH, that generalizes the notion of Dominant Resource
Fairness (DRF) from a single server to multiple heterogeneous servers. DRFH
provides a number of highly desirable properties. With DRFH, no user prefers
the allocation of another user; no one can improve its allocation without
decreasing that of the others; and more importantly, no user has an incentive
to lie about its resource demand. As a direct application, we design a simple
heuristic that implements DRFH in real-world systems. Large-scale simulations
driven by Google cluster traces show that DRFH significantly outperforms the
traditional slot-based scheduler, leading to much higher resource utilization
with substantially shorter job completion times
Autonomic Cloud Computing: Open Challenges and Architectural Elements
As Clouds are complex, large-scale, and heterogeneous distributed systems,
management of their resources is a challenging task. They need automated and
integrated intelligent strategies for provisioning of resources to offer
services that are secure, reliable, and cost-efficient. Hence, effective
management of services becomes fundamental in software platforms that
constitute the fabric of computing Clouds. In this direction, this paper
identifies open issues in autonomic resource provisioning and presents
innovative management techniques for supporting SaaS applications hosted on
Clouds. We present a conceptual architecture and early results evidencing the
benefits of autonomic management of Clouds.Comment: 8 pages, 6 figures, conference keynote pape
Choreo: network-aware task placement for cloud applications
Cloud computing infrastructures are increasingly being used by network-intensive applications that transfer significant amounts of data between the nodes on which they run. This paper shows that tenants can do a better job placing applications by understanding the underlying cloud network as well as the demands of the applications. To do so, tenants must be able to quickly and accurately measure the cloud network and profile their applications, and then use a network-aware placement method to place applications. This paper describes Choreo, a system that solves these problems. Our experiments measure Amazon's EC2 and Rackspace networks and use three weeks of network data from applications running on the HP Cloud network. We find that Choreo reduces application completion time by an average of 8%-14% (max improvement: 61%) when applications are placed all at once, and 22%-43% (max improvement: 79%) when they arrive in real-time, compared to alternative placement schemes.National Science Foundation (U.S.) (Grant 0645960)National Science Foundation (U.S.) (Grant 1065219)National Science Foundation (U.S.) (Grant 1040072
Supporting Big Data at the Vehicular Edge
Vehicular networks are commonplace, and many applications have been developed to utilize their sensor and computing resources. This is a great utilization of these resources as long as they are mobile. The question to ask is whether these resources could be put to use when the vehicle is not mobile. If the vehicle is parked, the resources are simply dormant and waiting for use. If the vehicle has a connection to a larger computing infrastructure, then it can put its resources towards that infrastructure. With enough vehicles interconnected, there exists a computing environment that could handle many cloud-based application services. If these vehicles were electric, then they could in return receive electrical charging services.
This Thesis will develop a simple vehicle datacenter solution based upon Smart Vehicles in a parking lot. While previous work has developed similar models based upon the idea of migration of jobs due to residency of the vehicles, this model will assume that residency times cannot be predicted and therefore no migration is utilized. In order to offset the migration of jobs, a divide-and-conquer approach is created. This uses a MapReduce process to divide the job into numerous sub-jobs and process the subtask in parallel. Finally, a checkpoint will be used between the Map and Reduce phase to avoid loss of intermediate data. This will serve as a means to test the practicality of the model and create a baseline for comparison with future research
- …