Effective Resource and Workload Management in Data Centers

The increasing demand for storage, computation, and business continuity has driven the growth of data centers. Managing data centers efficiently is a difficult task because of the wide variety of datacenter applications, their ever-changing intensities, and the fact that application performance targets may differ widely. Server virtualization has been a game-changing technology for IT, providing the possibility to support multiple virtual machines (VMs) simultaneously. This dissertation focuses on how virtualization technologies can be utilized to develop new tools for maintaining high resource utilization, for achieving high application performance, and for reducing the cost of data center management.;For multi-tiered applications, bursty workload traffic can significantly deteriorate performance. This dissertation proposes an admission control algorithm AWAIT, for handling overloading conditions in multi-tier web services. AWAIT places on hold requests of accepted sessions and refuses to admit new sessions when the system is in a sudden workload surge. to meet the service-level objective, AWAIT serves the requests in the blocking queue with high priority. The size of the queue is dynamically determined according to the workload burstiness.;Many admission control policies are triggered by instantaneous measurements of system resource usage, e.g., CPU utilization. This dissertation first demonstrates that directly measuring virtual machine resource utilizations with standard tools cannot always lead to accurate estimates. A directed factor graph (DFG) model is defined to model the dependencies among multiple types of resources across physical and virtual layers.;Virtualized data centers always enable sharing of resources among hosted applications for achieving high resource utilization. However, it is difficult to satisfy application SLOs on a shared infrastructure, as application workloads patterns change over time. AppRM, an automated management system not only allocates right amount of resources to applications for their performance target but also adjusts to dynamic workloads using an adaptive model.;Server consolidation is one of the key applications of server virtualization. This dissertation proposes a VM consolidation mechanism, first by extending the fair load balancing scheme for multi-dimensional vector scheduling, and then by using a queueing network model to capture the service contentions for a particular virtual machine placement

Autonomic management of virtualized resources in cloud computing

The last five years have witnessed a rapid growth of cloud computing in business, governmental and educational IT deployment. The success of cloud services depends critically on the effective management of virtualized resources. A key requirement of cloud management is the ability to dynamically match resource allocations to actual demands, To this end, we aim to design and implement a cloud resource management mechanism that manages underlying complexity, automates resource provisioning and controls client-perceived quality of service (QoS) while still achieving resource efficiency. The design of an automatic resource management centers on two questions: when to adjust resource allocations and how much to adjust. In a cloud, applications have different definitions on capacity and cloud dynamics makes it difficult to determine a static resource to performance relationship. In this dissertation, we have proposed a generic metric that measures application capacity, designed model-independent and adaptive approaches to manage resources and built a cloud management system scalable to a cluster of machines. To understand web system capacity, we propose to use a metric of productivity index (PI), which is defined as the ratio of yield to cost, to measure the system processing capability online. PI is a generic concept that can be applied to different levels to monitor system progress in order to identify if more capacity is needed. We applied the concept of PI to the problem of overload prevention in multi-tier websites. The overload predictor built on the PI metric shows more accurate and responsive overload prevention compared to conventional approaches. To address the issue of the lack of accurate server model, we propose a model-independent fuzzy control based approach for CPU allocation. For adaptive and stable control performance, we embed the controller with self-tuning output amplification and flexible rule selection. Finally, we build a QoS provisioning framework that supports multi-objective QoS control and service differentiation. Experiments on a virtual cluster with two service classes show the effectiveness of our approach in both performance and power control. To address the problems of complex interplay between resources and process delays in fine-grained multi-resource allocation, we consider capacity management as a decision-making problem and employ reinforcement learning (RL) to optimize the process. The optimization depends on the trial-and-error interactions with the cloud system. In order to improve the initial management performance, we propose a model-based RL algorithm. The neural network based environment model, which is learned from previous management history, generates simulated resource allocations for the RL agent. Experiment results on heterogeneous applications show that our approach makes efficient use of limited interactions and find near optimal resource configurations within 7 steps. Finally, we present a distributed reinforcement learning approach to the cluster-wide cloud resource management. We decompose the cluster-wide resource allocation problem into sub-problems concerning individual VM resource configurations. The cluster-wide allocation is optimized if individual VMs meet their SLA with a high resource utilization. For scalability, we develop an efficient reinforcement learning approach with continuous state space. For adaptability, we use VM low-level runtime statistics to accommodate workload dynamics. Prototyped in a iBalloon system, the distributed learning approach successfully manages 128 VMs on a 16-node close correlated cluster

Revenue maximization problems in commercial data centers

PhD ThesisAs IT systems are becoming more important everyday, one of the main concerns is that users may face major problems and eventually incur major costs if computing systems do not meet the expected performance requirements: customers expect reliability and performance guarantees, while underperforming systems loose revenues. Even with the adoption of data centers as the hub of IT organizations and provider of business efficiencies the problems are not over because it is extremely difficult for service providers to meet the promised performance guarantees in the face of unpredictable demand. One possible approach is the adoption of Service Level Agreements (SLAs), contracts that specify a level of performance that must be met and compensations in case of failure. In this thesis I will address some of the performance problems arising when IT companies sell the service of running ‘jobs’ subject to Quality of Service (QoS) constraints. In particular, the aim is to improve the efficiency of service provisioning systems by allowing them to adapt to changing demand conditions. First, I will define the problem in terms of an utility function to maximize. Two different models are analyzed, one for single jobs and the other useful to deal with session-based traffic. Then, I will introduce an autonomic model for service provision. The architecture consists of a set of hosted applications that share a certain number of servers. The system collects demand and performance statistics and estimates traffic parameters. These estimates are used by management policies which implement dynamic resource allocation and admission algorithms. Results from a number of experiments show that the performance of these heuristics is close to optimal.QoSP (Quality of Service Provisioning)British Teleco

Supporting Quality-Of-Service of Mobile Commerce Transactions

With the deployment of 3G and 4G mobile networks, a sizable proportion of e-commerce traffic is expected to move to these networks. These transactions are likely to be diverse. Mobile transactions can include unique requirements such as atomicity (all or none steps), push or pull, security, and privacy. Because users are mobile, unpredictable link characteristics, and other problems associated with wireless networks, some mobile commerce transactions may not be completed causing significant annoyance to the users affected. Therefore, the probability of completing mobile transactions is an important parameter for measuring quality-of-service of a network supporting mobile commerce. The transaction completion probability measures the ability of networks to support completion of transactions. This research focuses on improving the support of mobile commerce transactions by the underlying wireless networks. Mobile commerce traffic is classified as messaging, information connectivity, and transactions in order to provide efficient quality-of-service to various applications. This paper introduces the use of priority, sociability and delegation to improve the transaction completion in wireless networks

Performance Improvement of Multithreaded Java Applications Execution on Multiprocessor Systems

El disseny del llenguatge Java, que inclou aspectes importants com són la seva portabilitat i neutralitat envers l'arquitectura, les seves capacitats multithreading, la seva familiaritat (degut a la seva semblança amb C/C++), la seva robustesa, les seves capacitats en seguretat i la seva naturalesa distribuïda, fan que sigui un llenguatge potencialment interessant per ser utilitzat en entorns paral·lels com són els entorns de computació d'altes prestacions (HPC), on les aplicacions poden treure profit del suport que ofereix Java a l'execució multithreaded per realitzar càlculs en paral·lel, o en entorns e-business, on els servidors Java multithreaded (que segueixen l'especificació J2EE) poden treure profit de les capacitats multithreading de Java per atendre de manera concurrent un gran nombre de peticions.No obstant, l'ús de Java per la programació paral·lela ha d'enfrontar-se a una sèrie de problemes que fàcilment poden neutralitzar el guany obtingut amb l'execució en paral·lel. El primer problema és el gran overhead provocat pel suport de threads de la JVM quan s'utilitzen threads per executar feina de gra fi, quan es crea un gran nombre de threads per suportar l'execució d'una aplicació o quan els threads interaccionen estretament mitjançant mecanismes de sincronització. El segon problema és la degradació en el rendiment produïda quan aquestes aplicacions multithreaded s'executen en sistemes paral·lels multiprogramats. La principal causa d'aquest problemes és la manca de comunicació entre l'entorn d'execució i les aplicacions, la qual pot induir a les aplicacions a fer un ús descoordinat dels recursos disponibles.Aquesta tesi contribueix amb la definició d'un entorn per analitzar i comprendre el comportament de les aplicacions Java multithreaded. La contribució principal d'aquest entorn és que la informació de tots els nivells involucrats en l'execució (aplicació, servidor d'aplicacions, JVM i sistema operatiu) està correlada. Aquest fet és molt important per entendre com aquest tipus d'aplicacions es comporten quan s'executen en entorns que inclouen servidors i màquines virtuals, donat que l'origen dels problemes de rendiment es pot trobar en qualsevol d'aquests nivells o en la seva interacció.Addicionalment, i basat en el coneixement adquirit mitjançant l'entorn d'anàlisis proposat, aquesta tesi contribueix amb mecanismes i polítiques de planificació orientats cap a l'execució eficient d'aplicacions Java multithreaded en sistemes multiprocessador considerant les interaccions i la coordinació dels mecanismes i les polítiques de planificació en els diferents nivells involucrats en l'execució. La idea bàsica consisteix en permetre la cooperació entre les aplicacions i l'entorn d'execució en la gestió de recursos establint una comunicació bi-direccional entre les aplicacions i el sistema. Per una banda, les aplicacions demanen a l'entorn d'execució la quantitat de recursos que necessiten. Per altra banda, l'entorn d'execució pot ser inquirit en qualsevol moment per les aplicacions ser informades sobre la seva assignació de recursos. Aquesta tesi proposa que les aplicacions utilitzin la informació proporcionada per l'entorn d'execució per adaptar el seu comportament a la quantitat de recursos que tenen assignats (aplicacions auto-adaptables). Aquesta adaptació s'assoleix en aquesta tesi per entorns HPC per mitjà de la mal·leabilitat de les aplicacions, i per entorns e-business amb una proposta de control de congestió que fa control d'admissió basat en la diferenciació de connexions SSL per prevenir la degradació del rendiment i mantenir la Qualitat de Servei (QoS).Els resultats de l'avaluació demostren que subministrar recursos de manera dinàmica a les aplicacions auto-adaptables en funció de la seva demanda millora el rendiment de les aplicacions Java multithreaded tant en entorns HPC com en entorns e-business. Mentre disposar d'aplicacions auto-adaptables evita la degradació del rendiment, el subministrament dinàmic de recursos permet satisfer els requeriments de les aplicacions en funció de la seva demanda i adaptar-se a la variabilitat de les seves necessitats de recursos. D'aquesta manera s'aconsegueix una millor utilització dels recursos donat que els recursos que no utilitza una aplicació determinada poden ser distribuïts entre les altres aplicacions.The design of the Java language, which includes important aspects such as its portability and architecture neutrality, its multithreading facilities, its familiarity (due to its resemblance with C/C++), its robustness, its security capabilities and its distributed nature, makes it a potentially interesting language to be used in parallel environments such as high performance computing (HPC) environments, where applications can benefit from the Java multithreading support for performing parallel calculations, or e-business environments, where multithreaded Java application servers (i.e. following the J2EE specification) can take profit of Java multithreading facilities to handle concurrently a large number of requests.However, the use of Java for parallel programming has to face a number of problems that can easily offset the gain due to parallel execution. The first problem is the large overhead incurred by the threading support available in the JVM when threads are used to execute fine-grained work, when a large number of threads are created to support the execution of the application or when threads closely interact through synchronization mechanisms. The second problem is the performance degradation occurred when these multithreaded applications are executed in multiprogrammed parallel systems. The main issue that causes these problems is the lack of communication between the execution environment and the applications, which can cause these applications to make an uncoordinated use of the available resources.This thesis contributes with the definition of an environment to analyze and understand the behavior of multithreaded Java applications. The main contribution of this environment is that all levels in the execution (application, application server, JVM and operating system) are correlated. This is very important to understand how this kind of applications behaves when executed on environments that include servers and virtual machines, because the origin of performance problems can reside in any of these levels or in their interaction.In addition, and based on the understanding gathered using the proposed analysis environment, this thesis contributes with scheduling mechanisms and policies oriented towards the efficient execution of multithreaded Java applications on multiprocessor systems considering the interactions and coordination between scheduling mechanisms and policies at the different levels involved in the execution. The basis idea consists of allowing the cooperation between the applications and the execution environment in the resource management by establishing a bi-directional communication path between the applications and the underlying system. On one side, the applications request to the execution environment the amount of resources they need. On the other side, the execution environment can be requested at any time by the applications to inform them about their resource assignments. This thesis proposes that applications use the information provided by the execution environment to adapt their behavior to the amount of resources allocated to them (self-adaptive applications). This adaptation is accomplished in this thesis for HPC environments through the malleability of the applications, and for e-business environments with an overload control approach that performs admission control based on SSL connections differentiation for preventing throughput degradation and maintaining Quality of Service (QoS).The evaluation results demonstrate that providing resources dynamically to self-adaptive applications on demand improves the performance of multithreaded Java applications as in HPC environments as in e-business environments. While having self-adaptive applications avoids performance degradation, dynamic provision of resources allows meeting the requirements of the applications on demand and adapting to their changing resource needs. In this way, better resource utilization is achieved because the resources not used by some application may be distributed among other applications

Performance modelling and analysis of e-commerce systems using class based priority scheduling. An investigation into the development of new class based priority scheduling mechanisms for e-commerce system combining different techniques.

Recently, technological developments have affected most lifestyles, especially with the growth in Internet usage. Internet applications highlight the E-commerce capabilities and applications which are now available everywhere; they receive a great number of users on a 24-7 basis because online services are easy to use, faster and cheaper to acquire. Thus E-commerce web sites have become crucial for companies to increase their revenues. This importance has identified certain effective requirements needed from the performance of these applications. In particular, if the web server is overloaded, poor performance can result, due to either a huge rate of requests being generated which are beyond the server¿s capacity, or due to saturation of the communication links capacity which connects the web server to the network. Recent researches consider the overload issue and explore different mechanisms for managing the performance of E-commerce applications under overload condition. This thesis proposes a formal approach in order to investigate the effects of the extreme load and the number of dropped requests on the performance of E- III commerce web servers. The proposed approach is based on the class-based priority scheme that classifies E-commerce requests into different classes. Because no single technique can solve all aspects of overload problems, this research combines several techniques including: admission control mechanism, session-based admission control, service differentiation, request scheduling and queuing model-based approach. Request classification is based on the premise that some requests (e.g. buy) are generally considered more important than others (e.g. browse or search). Moreover, this research considers the extended models from Priority Scheduling Mechanism (PSM). These models add a new parameter, such as a review model or modify the basic PSM to low priority fair model, after the discovery of ineffectiveness with low priority customers or to add new features such as portal models. The proposed model is formally specified using the ¿ -calculus in early stage of models design and a multi-actor simulation was developed to reflect the target models as accurately as possible and is implemented as a Java-based prototype system. A formal specification that captures the essential PSM features while keeping the performance model sufficiently simple is presented. Furthermore, the simplicity of the UML bridges the gap between ¿-calculus and Java programming language. IV There are many metrics for measuring the performance of E-commerce web servers. This research focuses on the performance of E-commerce web servers that refer to the throughput, utilisation, average response time, dropped requests and arrival rate. A number of experiments are conducted in order to test the performance management of the proposed approaches

Flow-oriented anomaly-based detection of denial of service attacks with flow-control-assisted mitigation

Flooding-based distributed denial-of-service (DDoS) attacks present a serious and major threat to the targeted enterprises and hosts. Current protection technologies are still largely inadequate in mitigating such attacks, especially if they are large-scale. In this doctoral dissertation, the Computer Network Management and Control System (CNMCS) is proposed and investigated; it consists of the Flow-based Network Intrusion Detection System (FNIDS), the Flow-based Congestion Control (FCC) System, and the Server Bandwidth Management System (SBMS). These components form a composite defense system intended to protect against DDoS flooding attacks. The system as a whole adopts a flow-oriented and anomaly-based approach to the detection of these attacks, as well as a control-theoretic approach to adjust the flow rate of every link to sustain the high priority flow-rates at their desired level. The results showed that the misclassification rates of FNIDS are low, less than 0.1%, for the investigated DDOS attacks, while the fine-grained service differentiation and resource isolation provided within the FCC comprise a novel and powerful built-in protection mechanism that helps mitigate DDoS attacks

Performance modelling with adaptive hidden Markov models and discriminatory processor sharing queues

In modern computer systems, workload varies at different times and locations. It is important to model the performance of such systems via workload models that are both representative and efficient. For example, model-generated workloads represent realistic system behaviour, especially during peak times, when it is crucial to predict and address performance bottlenecks. In this thesis, we model performance, namely throughput and delay, using adaptive models and discrete queues. Hidden Markov models (HMMs) parsimoniously capture the correlation and burstiness of workloads with spatiotemporal characteristics. By adapting the batch training of standard HMMs to incremental learning, online HMMs act as benchmarks on workloads obtained from live systems (i.e. storage systems and financial markets) and reduce time complexity of the Baum-Welch algorithm. Similarly, by extending HMM capabilities to train on multiple traces simultaneously it follows that workloads of different types are modelled in parallel by a multi-input HMM. Typically, the HMM-generated traces verify the throughput and burstiness of the real data. Applications of adaptive HMMs include predicting user behaviour in social networks and performance-energy measurements in smartphone applications. Equally important is measuring system delay through response times. For example, workloads such as Internet traffic arriving at routers are affected by queueing delays. To meet quality of service needs, queueing delays must be minimised and, hence, it is important to model and predict such queueing delays in an efficient and cost-effective manner. Therefore, we propose a class of discrete, processor-sharing queues for approximating queueing delay as response time distributions, which represent service level agreements at specific spatiotemporal levels. We adapt discrete queues to model job arrivals with distributions given by a Markov-modulated Poisson process (MMPP) and served under discriminatory processor-sharing scheduling. Further, we propose a dynamic strategy of service allocation to minimise delays in UDP traffic flows whilst maximising a utility function.Open Acces

Revenue maximization problems in commercial data centers

As IT systems are becoming more important everyday, one of the main concerns is that users may face major problems and eventually incur major costs if computing systems do not meet the expected performance requirements: customers expect reliability and performance guarantees, while underperforming systems loose revenues. Even with the adoption of data centers as the hub of IT organizations and provider of business efficiencies the problems are not over because it is extremely difficult for service providers to meet the promised performance guarantees in the face of unpredictable demand. One possible approach is the adoption of Service Level Agreements (SLAs), contracts that specify a level of performance that must be met and compensations in case of failure. In this thesis I will address some of the performance problems arising when IT companies sell the service of running ‘jobs’ subject to Quality of Service (QoS) constraints. In particular, the aim is to improve the efficiency of service provisioning systems by allowing them to adapt to changing demand conditions. First, I will define the problem in terms of an utility function to maximize. Two different models are analyzed, one for single jobs and the other useful to deal with session-based traffic. Then, I will introduce an autonomic model for service provision. The architecture consists of a set of hosted applications that share a certain number of servers. The system collects demand and performance statistics and estimates traffic parameters. These estimates are used by management policies which implement dynamic resource allocation and admission algorithms. Results from a number of experiments show that the performance of these heuristics is close to optimal.EThOS - Electronic Theses Online ServiceQoSP (Quality of Service Provisioning) : British TelecomGBUnited Kingdo