QoS-aware mechanisms for improving cost-efficiency of datacenters

Warehouse Scale Computers (WSCs) promise high cost-efficiency by amortizing power, cooling, and management overheads. WSCs today host a large variety of jobs with two broad performance requirements categories: latency-critical (LC) and best-effort (BE). Ideally, to fully utilize all hardware resources, WSC operators can simply fill all the nodes with computing jobs. Unfortunately, because colocated jobs contend for shared resources, systems with high loads often experience performance degradation, which negatively impacts the Quality of Service (QoS) for LC jobs. In fact, service providers usually over-provision resources to avoid any interference with LC jobs, leading to significant resource inefficiencies. In this dissertation, I explore opportunities across different system-abstraction layers to improve the cost-efficiency of dataceters by increasing resource utilization of WSCs with little or no impact on the performance of LC jobs. The dissertation has three main components. First, I explore opportunities to improve the throughput of multicore systems by reducing the performance variation of LC jobs. The main insight is that by reshaping the latency distribution curve, performance headroom of LC jobs can be effectively converted to improved BE throughput. I develop, implement, and evaluate a runtime system that achieves this goal with existing hardware. I leverage the cache partitioning, per-core frequency scaling, and thread masking of server processors. Evaluation results show the proposed solution enables 30% higher system throughput compared to solutions proposed in prior works while maintaining at least as good QoS for LC jobs. Second, I study resource contention in near-future heterogeneous memory architectures (HMA). This study is motivated by recent developments in non-volatile memory (NVM) technologies, which enable higher storage density at the cost of same performance. To understand the performance and QoS impact of HMAs, I design and implement a performance emulator in the Linux kernel that runs unmodified workloads with high accuracy, low overhead, and complete transparency. I further propose and evaluate multiple data and resource management QoS mechanisms, such as locality-aware page admission, occupancy management, and write buffer jailing. Third, I focus on accelerated machine learning (ML) systems. By profiling the performance of production workloads and accelerators, I show that accelerated ML tasks are highly sensitive to main memory interference due to fine-grained interaction between CPU and accelerator tasks. As a result, memory resource contention can significantly decreases the performance and efficiency gains of accelerators. I propose a runtime system that leverages existing hardware capabilities and show 17% higher system efficiency compared to previous approaches. This study further exposes opportunities for future processor architecturesElectrical and Computer Engineerin

Design Space Exploration and Resource Management of Multi/Many-Core Systems

The increasing demand of processing a higher number of applications and related data on computing platforms has resulted in reliance on multi-/many-core chips as they facilitate parallel processing. However, there is a desire for these platforms to be energy-efficient and reliable, and they need to perform secure computations for the interest of the whole community. This book provides perspectives on the aforementioned aspects from leading researchers in terms of state-of-the-art contributions and upcoming trends

Automatic generation of workload profiles using unsupervised learning pipelines

The complexity of resource usage and power consumption on cloud-based applications makes the understanding of application behavior through expert examination difficult. The difficulty increases when applications are seen as “black boxes”, where only external monitoring can be retrieved. Furthermore, given the different amount of scenarios and applications, automation is required. Here we examine and model application behavior by finding behavior phases. We use Conditional Restricted Boltzmann Machines (CRBM) to model time-series containing resources traces measurements like CPU, Memory and IO. CRBMs can be used to map a given given historic window of trace behaviour into a single vector. This low dimensional and time-aware vector can be passed through clustering methods, from simplistic ones like k-means to more complex ones like those based on Hidden Markov Models (HMM). We use these methods to find phases of similar behaviour in the workloads. Our experimental evaluation shows that the proposed method is able to identify different phases of resource consumption across different workloads. We show that the distinct phases contain specific resource patterns that distinguish them.Peer ReviewedPostprint (published version

A survey on architectures and energy efficiency in Data Center Networks

Data Center Networks (DCNs) are attracting growing interest from both academia and industry to keep pace with the exponential growth in cloud computing and enterprise networks. Modern DCNs are facing two main challenges of scalability and cost-effectiveness. The architecture of a DCN directly impacts on its scalability, while its cost is largely driven by its power consumption. In this paper, we conduct a detailed survey of the most recent advances and research activities in DCNs, with a special focus on the architectural evolution of DCNs and their energy efficiency. The paper provides a qualitative categorization of existing DCN architectures into switch-centric and server-centric topologies as well as their design technologies. Energy efficiency in data centers is discussed in details with survey of existing techniques in energy savings, green data centers and renewable energy approaches. Finally, we outline potential future research directions in DCNs

Fog Orchestration and Simulation for IoT Services

The Internet of Things (IoT) interconnects physical objects including sensors, vehicles, and buildings into a virtual circumstance, resulting in the increasing integration of Cyber-physical objects. The Fog computing paradigm extends both computation and storage services in Cloud computing environment to the network edge. Typically, IoT services comprise of a set of software components running over different locations connected through datacenter or wireless sensor networks. It is significantly important and cost-effective to orchestrate and deploy a group of microservices onto Fog appliances such as edge devices or Cloud servers for the formation of such IoT services. In this chapter, we discuss the challenges of realizing Fog orchestration for IoT services, and present a software-defined orchestration architecture and simulation solutions to intelligently compose and orchestrate thousands of heterogeneous Fog appliances. The resource provisioning, component placement and runtime QoS control in the orchestration procedure can harness workload dynamicity, network uncertainty and security demands whilst considering different applications’ requirement and appliances’ capabilities. Our practical experiences show that the proposed parallelized orchestrator can reduce the execution time by 50% with at least 30% higher orchestration quality. We believe that our solution plays an important role in the current Fog ecosystem

Management of Cloud systems applied to eHealth

This thesis explores techniques, models and algorithms for an efficient management of Cloud systems and how to apply them to the healthcare sector in order to improve current treatments. It presents two Cloud-based eHealth applications to telemonitor and control smoke-quitting and hypertensive patients. Different Cloud-based models were obtained and used to develop a Cloudbased infrastructure where these applications are deployed. The results show that these applications improve current treatments and that can be scaled as computing requirements grow. Multiple Cloud architectures and models were analyzed and then implemented using different techniques and scenarios. The Smoking Patient Control (S-PC) tool was deployed and tested in a real environment, showing a 28.4% increase in long-term abstinence. The Hypertension Patient Control (H-PC) tool, was successfully designed and implemented, and the computing boundaries were measuredAquesta tesi explora tèniques, models i algorismes per una gestió eficient en sistemes al Núvol i com aplicar-ho en el sector de la salut per tal de millorar els tractaments actuals. Presenta dues aplicacions de salut electrònica basades en el Núvol per telemonitoritzar i controlar pacients fumadors i hipertensos. S'ha obtingut diferents models basats en el Núvol i s'han utilitzat per a desenvolupar una infraestructura on desplegar aquestes aplicacions. Els resultats mostren que aquestes aplicacions milloren els tractaments actuals així com escalen a mesura que els requeriments computacionals augmenten. Múltiples arquitectures i models han estat analitzats i implementats utilitzant diferents tècniques i escenaris. L'aplicació Smoking Patient Control (S-PC) ha estat desplegada i provada en un entorn real, aconseguint un augment del 28,4% en l'absistinència a llarg termini de pacients fumadors. L'aplicació Hypertension Patient Control (H-PC) ha estat dissenyada i implementada amb èxit, i els seus límits computacionals han estat mesurats.Esta tesis explora ténicas, modelos y algoritmos para una gestión eficiente de sistemas en la Nube y como aplicarlo en el sector de la salud con el fin de mejorar los tratamientos actuales. Presenta dos aplicaciones de salud electrónica basadas en la Nube para telemonitorizar y controlar pacientes fumadores e hipertensos. Se han obtenido diferentes modelos basados en la Nube y se han utilizado para desarrollar una infraestructura donde desplegar estas aplicaciones. Los resultados muestran que estas aplicaciones mejoran los tratamientos actuales así como escalan a medida que los requerimientos computacionales aumentan. Múltiples arquitecturas y modelos han sido analizados e implementados utilizando diferentes técnicas y escenarios. La aplicación Smoking Patient Control (S-PC) se ha desplegado y provado en un entorno real, consiguiendo un aumento del 28,4% en la abstinencia a largo plazo de pacientes fumadores. La aplicación Hypertension Patient Control (H-PC) ha sido diseñada e implementada con éxito, y sus límites computacionales han sido medidos

Proceedings of the Second International Workshop on Sustainable Ultrascale Computing Systems (NESUS 2015) Krakow, Poland

Proceedings of: Second International Workshop on Sustainable Ultrascale Computing Systems (NESUS 2015). Krakow (Poland), September 10-11, 2015

Modeling and simulation of data-driven applications in SDN-aware environments

PhD ThesisThe rising popularity of Software-Defined Networking (SDN) is increasing as it promises to offer a window of opportunity and new features in terms of network performance, configuration, and management. As such, SDN is exploited by several emerging applications and environments, such as cloud computing, edge computing, IoT, and data- driven applications. Although SDN has demonstrated significant improvements in industry, still little research has explored the embracing of SDN in the area of cross-layer optimization in different SDN-aware environments. Each application and computing environment require different functionalities and Quality of Service (QoS) requirements. For example, a typical MapReduce application would require data transmission at three different times while the data transmission of stream-based applications would be unknown due to uncertainty about the number of required tasks and dependencies among stream tasks. As such, the deployment of SDN with different applications are not identical, which require different deployment strategies and algorithms to meet different QoS requirements (e.g., high bandwidth, deadline). Further, each application and environment has unique architectures, which impose different form of complexity in terms of computing, storage, and network. Due to such complexities, finding optimal solutions for SDN-aware applications and environments become very challenging. Therefore, this thesis presents multilateral research towards optimization, modeling, and simulation of cross-layer optimization of SDN-aware applications and environments. Several tools and algorithms have been proposed, implemented, and evaluated, considering various environments and applications[1–4]. The main contributions of this thesis are as follows: • Proposing and modeling a new holistic framework that simulates MapReduce ap- plications, big data management systems (BDMS), and SDN-aware networks in cloud-based environments. Theoretical and mathematical models of MapReduce in SDN-aware cloud datacenters are also proposedThe government of Saudi Arabia represented by Saudi Electronic University (SEU) and the Royal Embassy of Saudi Arabia Cultural Burea