11,258 research outputs found
DRS: Dynamic Resource Scheduling for Real-Time Analytics over Fast Streams
In a data stream management system (DSMS), users register continuous queries,
and receive result updates as data arrive and expire. We focus on applications
with real-time constraints, in which the user must receive each result update
within a given period after the update occurs. To handle fast data, the DSMS is
commonly placed on top of a cloud infrastructure. Because stream properties
such as arrival rates can fluctuate unpredictably, cloud resources must be
dynamically provisioned and scheduled accordingly to ensure real-time response.
It is quite essential, for the existing systems or future developments, to
possess the ability of scheduling resources dynamically according to the
current workload, in order to avoid wasting resources, or failing in delivering
correct results on time. Motivated by this, we propose DRS, a novel dynamic
resource scheduler for cloud-based DSMSs. DRS overcomes three fundamental
challenges: (a) how to model the relationship between the provisioned resources
and query response time (b) where to best place resources; and (c) how to
measure system load with minimal overhead. In particular, DRS includes an
accurate performance model based on the theory of \emph{Jackson open queueing
networks} and is capable of handling \emph{arbitrary} operator topologies,
possibly with loops, splits and joins. Extensive experiments with real data
confirm that DRS achieves real-time response with close to optimal resource
consumption.Comment: This is the our latest version with certain modificatio
Recommended from our members
On Optimal and Fair Service Allocation in Mobile Cloud Computing
This paper studies the optimal and fair service allocation for a variety of
mobile applications (single or group and collaborative mobile applications) in
mobile cloud computing. We exploit the observation that using tiered clouds,
i.e. clouds at multiple levels (local and public) can increase the performance
and scalability of mobile applications. We proposed a novel framework to model
mobile applications as a location-time workflows (LTW) of tasks; here users
mobility patterns are translated to mobile service usage patterns. We show that
an optimal mapping of LTWs to tiered cloud resources considering multiple QoS
goals such application delay, device power consumption and user cost/price is
an NP-hard problem for both single and group-based applications. We propose an
efficient heuristic algorithm called MuSIC that is able to perform well (73% of
optimal, 30% better than simple strategies), and scale well to a large number
of users while ensuring high mobile application QoS. We evaluate MuSIC and the
2-tier mobile cloud approach via implementation (on real world clouds) and
extensive simulations using rich mobile applications like intensive signal
processing, video streaming and multimedia file sharing applications. Our
experimental and simulation results indicate that MuSIC supports scalable
operation (100+ concurrent users executing complex workflows) while improving
QoS. We observe about 25% lower delays and power (under fixed price
constraints) and about 35% decrease in price (considering fixed delay) in
comparison to only using the public cloud. Our studies also show that MuSIC
performs quite well under different mobility patterns, e.g. random waypoint and
Manhattan models
Dynamic, Latency-Optimal vNF Placement at the Network Edge
Future networks are expected to support low-latency, context-aware and user-specific services in a highly flexible and efficient manner. One approach to support emerging use cases such as, e.g., virtual reality and in-network image processing is to introduce virtualized network functions (vNF)s at the edge of the network, placed in close proximity to the end users to reduce end-to-end latency, time-to-response, and unnecessary utilisation in the core network. While placement of vNFs has been studied before, it has so far mostly focused on reducing the utilisation of server resources (i.e., minimising the number of servers required in the network to run a specific set of vNFs), and not taking network conditions into consideration such as, e.g., end-to-end latency, the constantly changing network dynamics, or user mobility patterns. In this paper, we formulate the Edge vNF placement problem to allocate vNFs to a distributed edge infrastructure, minimising end-to-end latency from all users to their associated vNFs. We present a way to dynamically re-schedule the optimal placement of vNFs based on temporal network-wide latency fluctuations using optimal stopping theory. We then evaluate our dynamic scheduler over a simulated nation-wide backbone network using real-world ISP latency characteristics. We show that our proposed dynamic placement scheduler minimises vNF migrations compared to other schedulers (e.g., periodic and always-on scheduling of a new placement), and offers Quality of Service guarantees by not exceeding a maximum number of latency violations that can be tolerated by certain applications
- …