Scheduling Storms and Streams in the Cloud

Motivated by emerging big streaming data processing paradigms (e.g., Twitter Storm, Streaming MapReduce), we investigate the problem of scheduling graphs over a large cluster of servers. Each graph is a job, where nodes represent compute tasks and edges indicate data-flows between these compute tasks. Jobs (graphs) arrive randomly over time, and upon completion, leave the system. When a job arrives, the scheduler needs to partition the graph and distribute it over the servers to satisfy load balancing and cost considerations. Specifically, neighboring compute tasks in the graph that are mapped to different servers incur load on the network; thus a mapping of the jobs among the servers incurs a cost that is proportional to the number of "broken edges". We propose a low complexity randomized scheduling algorithm that, without service preemptions, stabilizes the system with graph arrivals/departures; more importantly, it allows a smooth trade-off between minimizing average partitioning cost and average queue lengths. Interestingly, to avoid service preemptions, our approach does not rely on a Gibbs sampler; instead, we show that the corresponding limiting invariant measure has an interpretation stemming from a loss system.Comment: 14 page

Ordonnancement multi-objectifs de workflows dans le cloud : un modèle plus réaliste avec tâches de durée stochastique

National audienceLa souplesse en terme de disponibilité de ressources que permet le cloud rend possible une adaptation de l'ordonnancement des tâches qui y sont exécutées face à l'imprévisibilité de cer-tains paramètres. Cependant, les méthodes d'ordonnancement existantes utilisent des modèles trop simplifiés : les workflows sont totalement déterministes ou leur structure n'est pas consi-dérée, ou encore la modélisation du cloud ignore certains aspects capitaux de cette plateforme. Cet article propose un modèle prenant en compte le fait que le nombre d'instructions consti-tuant une tâche peut ne pas être déterministe sans pour autant sacrifier totalement la com-plexité de la plateforme ou la structure du workflow. Nous proposons en outre quelques pistes pour l'élaboration d'une méthode d'ordonnancement multi-objectifs reposant sur ce modèle

TAME: an Efficient Task Allocation Algorithm for Integrated Mobile Gaming

We consider an integrated mobile gaming platform, in which the mobile device (e.g., smartphone) of a player can offload some game tasks toward a server as well as some neighboring mobile devices. The advantages of such a platform are manyfold: it can lead to an improved game experience, to a better use of energy resources, and, while offloading tasks to other mobile users, to the exploitation of the unused computing and storage resources of the mobile equipments, thus reducing the bandwidth and computing costs of the overall system. In this context, we formulate the problem of offloading the game computational tasks as an optimization problem that minimizes the maximum energy consumption across a set of mobile devices, under the constraints of a maximum response time and a limited availability of computation, communication and storage resources. In light of the problem complexity, we then propose a heuristic, called TAME, which is shown to closely approximate the optimal solution in all scenarios we considered. TAME also outperforms state-of-the-art algorithms under both synthetic and real scenarios, which have been devised based on a realistic and detailed energy consumption model for computation and communication resources. Our results, although tailored to mobile gaming, could be extended to other applications where it may be beneficial to offload computational and storage tasks through device-to-device communications, as enabled by Wi-Fi, Bluetooth, or the upcoming 5G technology