Task Runtime Prediction in Scientific Workflows Using an Online Incremental Learning Approach

Many algorithms in workflow scheduling and resource provisioning rely on the performance estimation of tasks to produce a scheduling plan. A profiler that is capable of modeling the execution of tasks and predicting their runtime accurately, therefore, becomes an essential part of any Workflow Management System (WMS). With the emergence of multi-tenant Workflow as a Service (WaaS) platforms that use clouds for deploying scientific workflows, task runtime prediction becomes more challenging because it requires the processing of a significant amount of data in a near real-time scenario while dealing with the performance variability of cloud resources. Hence, relying on methods such as profiling tasks' execution data using basic statistical description (e.g., mean, standard deviation) or batch offline regression techniques to estimate the runtime may not be suitable for such environments. In this paper, we propose an online incremental learning approach to predict the runtime of tasks in scientific workflows in clouds. To improve the performance of the predictions, we harness fine-grained resources monitoring data in the form of time-series records of CPU utilization, memory usage, and I/O activities that are reflecting the unique characteristics of a task's execution. We compare our solution to a state-of-the-art approach that exploits the resources monitoring data based on regression machine learning technique. From our experiments, the proposed strategy improves the performance, in terms of the error, up to 29.89%, compared to the state-of-the-art solutions.Comment: Accepted for presentation at main conference track of 11th IEEE/ACM International Conference on Utility and Cloud Computin

Software-Defined Cloud Computing: Architectural Elements and Open Challenges

The variety of existing cloud services creates a challenge for service providers to enforce reasonable Software Level Agreements (SLA) stating the Quality of Service (QoS) and penalties in case QoS is not achieved. To avoid such penalties at the same time that the infrastructure operates with minimum energy and resource wastage, constant monitoring and adaptation of the infrastructure is needed. We refer to Software-Defined Cloud Computing, or simply Software-Defined Clouds (SDC), as an approach for automating the process of optimal cloud configuration by extending virtualization concept to all resources in a data center. An SDC enables easy reconfiguration and adaptation of physical resources in a cloud infrastructure, to better accommodate the demand on QoS through a software that can describe and manage various aspects comprising the cloud environment. In this paper, we present an architecture for SDCs on data centers with emphasis on mobile cloud applications. We present an evaluation, showcasing the potential of SDC in two use cases-QoS-aware bandwidth allocation and bandwidth-aware, energy-efficient VM placement-and discuss the research challenges and opportunities in this emerging area.Comment: Keynote Paper, 3rd International Conference on Advances in Computing, Communications and Informatics (ICACCI 2014), September 24-27, 2014, Delhi, Indi

HPC Cloud for Scientific and Business Applications: Taxonomy, Vision, and Research Challenges

High Performance Computing (HPC) clouds are becoming an alternative to on-premise clusters for executing scientific applications and business analytics services. Most research efforts in HPC cloud aim to understand the cost-benefit of moving resource-intensive applications from on-premise environments to public cloud platforms. Industry trends show hybrid environments are the natural path to get the best of the on-premise and cloud resources---steady (and sensitive) workloads can run on on-premise resources and peak demand can leverage remote resources in a pay-as-you-go manner. Nevertheless, there are plenty of questions to be answered in HPC cloud, which range from how to extract the best performance of an unknown underlying platform to what services are essential to make its usage easier. Moreover, the discussion on the right pricing and contractual models to fit small and large users is relevant for the sustainability of HPC clouds. This paper brings a survey and taxonomy of efforts in HPC cloud and a vision on what we believe is ahead of us, including a set of research challenges that, once tackled, can help advance businesses and scientific discoveries. This becomes particularly relevant due to the fast increasing wave of new HPC applications coming from big data and artificial intelligence.Comment: 29 pages, 5 figures, Published in ACM Computing Surveys (CSUR

A Framework for Energy-efficient Mobile Cloud Offloading

Esilekerkivad nutitelefonide tehnoloogiad on kogenud geomeetrilist kasvu ja on praegu veel tõusuteel. Inimesed kasutavad nutitelefone oma igapäevastes tegevustes nagu e-maili saatmine, fotode ja videode jagamine läbi erinevate peer-to-peersotsiaalvõrgustiku jaoturite ja nii edasi. Viimastel aastatel on nutitelefonid kogenud suuri tehnoloogilisi edusamme ja innovatsiooni seoses töötlusvõimekusega ja saab nüüd kasutada keerukate ja ressursimahukate ülesannete täitmiseks rakendustes, näiteks videode monteerimine ja töötlemine ning objekti äratundmine. Kuigi enamus nutitelefone on oluliselt täiustatud, et hakkama saada suurendatud rakendustega, millel on keerukad arvutusvajadused, piiravad neid ikkagi nende energiavarud, näiteks aku kestvus. Akutehnoloogia ei ole arenenud nii kiirelt kui teised nutitelefoni valdkonnad ja seega arvutusintensiivsete ülesannete läbiviimine põhjustaks selle kiire kahanemise; tõestuseks vajadus pidevalt laadida seadme akut. Mitmeid meetodeid on pakutud välja energiasäästu maksimeerimiseks mobiilsetel seadmetel. Mõned neist aeglustavad keskprotsessor või lülitavad ekraani välja, kui on tegevusetud. Nende hulgast kõige märkimisväärsem tehnika nutitelefoni energia säästmiseks on arvutusvõimsuse koormuse jaotamine. See hõlmab teatud ülesannete töötluse üleviimist piiratud ressurssidega nutitelefonist kaugesse ressursirikkasse seadmesse hõlbustades seega nutitelefoni energia tarbimist. See on küllaltki lai uurimisvaldkond ja on hulganisti panustatud selle ala arendamiseks. Sellele vaatamata on veel palju tööd vaja teha seoses energia säästmisega läbi arvutusvõimsuse koormuse jaotamise korduva ressursimahuka töötlemise ajal. Selles teadusuuringus on me eesmärk vähendada energia tarbimist korduva energiamahuka töötlemise ajal. Me arvestame konteksti teadlikkust pakkudes välja plaanuri mudelit, mis saaks vähendada mobiilse seadme energia kiiret vähenemist seega saavutades meie eesmärgi. Pakume teenusele orienteeritud raamistikku eesmärgiga võimaldada energiatõhusa ülesande täitmist mobiilsel seadmel plaanuri käitumisalgoritmi abil. Me arendame kontseptsiooni tõestuse prototüüpi Android seadmel, et demonstreerida ja hinnata raamistiku energiasäästu võimekust.Emerging smartphone technologies has experienced a geometric increase and is currently still on the rise. People use the smartphone for their day-to-day activities such as sending emails, sharing photos and videos through various peer-to-peer social network hubs and so on. In the last few years, the smartphone has experienced massive technological advancements and innovation with respect to its processing capabilities and can now be used to perform complex, resource-intensive tasks in advanced applications like video editing and processing, and object recognition. Although most smartphones have been greatly augmented to handle advanced applications with complex computational needs, they are still limited in terms of their energy resources i.e. battery life. Battery technology has not evolved as rapidly as other areas of the smartphone and so the execution of computational-intensive tasks would cause its rapid depletion; evidenced by the need to constantly charge the device battery. Many techniques have been proffered to maximize energy conservation on mobile devices. Some of which are slowing down the CPU, or shutting off the screen when idle. Among these, the most notable technique for conserving smartphone energy is computation offloading. This basically involves the transfer of the processing of certain tasks from a resource-constrained smartphone to a remote, resource-rich device thereby facilitating energy conservation on the smartphone. This is a fairly large research area and numerous contributions have been made towards advancement in this field. However, much work is yet to be done with regards to energy conservation through offloading during recurrent resource-intensive processing. In this research study we aim to reduce energy consumption during continuous, energy-intensive processing. We consider context-awareness in proposing a scheduling model that could potentially minimize the speedy depletion of mobile device energy thus achieving our aim. We propose a service-oriented framework towards enabling energy-optimal task execution through a task scheduling offload algorithm. We develop a proof-of-concept prototype on an Android device to demonstrate and evaluate the framework’s energy conserving capabilities

Towards a Practical Pedestrian Distraction Detection Framework using Wearables

Pedestrian safety continues to be a significant concern in urban communities and pedestrian distraction is emerging as one of the main causes of grave and fatal accidents involving pedestrians. The advent of sophisticated mobile and wearable devices, equipped with high-precision on-board sensors capable of measuring fine-grained user movements and context, provides a tremendous opportunity for designing effective pedestrian safety systems and applications. Accurate and efficient recognition of pedestrian distractions in real-time given the memory, computation and communication limitations of these devices, however, remains the key technical challenge in the design of such systems. Earlier research efforts in pedestrian distraction detection using data available from mobile and wearable devices have primarily focused only on achieving high detection accuracy, resulting in designs that are either resource intensive and unsuitable for implementation on mainstream mobile devices, or computationally slow and not useful for real-time pedestrian safety applications, or require specialized hardware and less likely to be adopted by most users. In the quest for a pedestrian safety system that achieves a favorable balance between computational efficiency, detection accuracy, and energy consumption, this paper makes the following main contributions: (i) design of a novel complex activity recognition framework which employs motion data available from users' mobile and wearable devices and a lightweight frequency matching approach to accurately and efficiently recognize complex distraction related activities, and (ii) a comprehensive comparative evaluation of the proposed framework with well-known complex activity recognition techniques in the literature with the help of data collected from human subject pedestrians and prototype implementations on commercially-available mobile and wearable devices

CoolCloud: improving energy efficiency in virtualized data centers

In recent years, cloud computing services continue to grow and has become more pervasive and indispensable in people\u27s lives. The energy consumption continues to rise as more and more data centers are being built. How to provide a more energy efficient data center infrastructure that can support today\u27s cloud computing services has become one of the most important issues in the field of cloud computing research. In this thesis, we mainly tackle three research problems: 1. how to achieve energy savings in a virtualized data center environment; 2. how to maintain service level agreements; 3. how to make our design practical for actual implementation in enterprise data centers. Combining all the studies above, we propose an optimization framework named CoolCloud to minimize energy consumption in virtualized data centers with the service level agreement taken into consideration. The proposed framework minimizes energy at two different layers: (1) minimize local server energy using dynamic voltage \& frequency scaling (DVFS) exploiting runtime program phases. (2) minimize global cluster energy using dynamic mapping between virtual machines (VMs) and servers based on each VM\u27s resource requirement. Such optimization leads to the most economical way to operate an enterprise data center. On each local server, we develop a voltage and frequency scheduler that can provide CPU energy savings under applications\u27 or virtual machines\u27 specified SLA requirements by exploiting applications\u27 run-time program phases. At the cluster level, we propose a practical solution for managing the mappings of VMs to physical servers. This framework solves the problem of finding the most energy efficient way (least resource wastage and least power consumption) of placing the VMs considering their resource requirements