Libra: An Economy driven Job Scheduling System for Clusters

Clusters of computers have emerged as mainstream parallel and distributed platforms for high-performance, high-throughput and high-availability computing. To enable effective resource management on clusters, numerous cluster managements systems and schedulers have been designed. However, their focus has essentially been on maximizing CPU performance, but not on improving the value of utility delivered to the user and quality of services. This paper presents a new computational economy driven scheduling system called Libra, which has been designed to support allocation of resources based on the users? quality of service (QoS) requirements. It is intended to work as an add-on to the existing queuing and resource management system. The first version has been implemented as a plugin scheduler to the PBS (Portable Batch System) system. The scheduler offers market-based economy driven service for managing batch jobs on clusters by scheduling CPU time according to user utility as determined by their budget and deadline rather than system performance considerations. The Libra scheduler ensures that both these constraints are met within an O(n) run-time. The Libra scheduler has been simulated using the GridSim toolkit to carry out a detailed performance analysis. Results show that the deadline and budget based proportional resource allocation strategy improves the utility of the system and user satisfaction as compared to system-centric scheduling strategies.Comment: 13 page

Performance Modeling of Softwarized Network Services Based on Queuing Theory with Experimental Validation

Network Functions Virtualization facilitates the automation of the scaling of softwarized network services (SNSs). However, the realization of such a scenario requires a way to determine the needed amount of resources so that the SNSs performance requisites are met for a given workload. This problem is known as resource dimensioning, and it can be efficiently tackled by performance modeling. In this vein, this paper describes an analytical model based on an open queuing network of G/G/m queues to evaluate the response time of SNSs. We validate our model experimentally for a virtualized Mobility Management Entity (vMME) with a three-tiered architecture running on a testbed that resembles a typical data center virtualization environment. We detail the description of our experimental setup and procedures. We solve our resulting queueing network by using the Queueing Networks Analyzer (QNA), Jackson’s networks, and Mean Value Analysis methodologies, and compare them in terms of estimation error. Results show that, for medium and high workloads, the QNA method achieves less than half of error compared to the standard techniques. For low workloads, the three methods produce an error lower than 10%. Finally, we show the usefulness of the model for performing the dynamic provisioning of the vMME experimentally.This work has been partially funded by the H2020 research and innovation project 5G-CLARITY (Grant No. 871428)National research project 5G-City: TEC2016-76795-C6-4-RSpanish Ministry of Education, Culture and Sport (FPU Grant 13/04833). We would also like to thank the reviewers for their valuable feedback to enhance the quality and contribution of this wor

Management of Cloud systems applied to eHealth

This thesis explores techniques, models and algorithms for an efficient management of Cloud systems and how to apply them to the healthcare sector in order to improve current treatments. It presents two Cloud-based eHealth applications to telemonitor and control smoke-quitting and hypertensive patients. Different Cloud-based models were obtained and used to develop a Cloudbased infrastructure where these applications are deployed. The results show that these applications improve current treatments and that can be scaled as computing requirements grow. Multiple Cloud architectures and models were analyzed and then implemented using different techniques and scenarios. The Smoking Patient Control (S-PC) tool was deployed and tested in a real environment, showing a 28.4% increase in long-term abstinence. The Hypertension Patient Control (H-PC) tool, was successfully designed and implemented, and the computing boundaries were measuredAquesta tesi explora tèniques, models i algorismes per una gestió eficient en sistemes al Núvol i com aplicar-ho en el sector de la salut per tal de millorar els tractaments actuals. Presenta dues aplicacions de salut electrònica basades en el Núvol per telemonitoritzar i controlar pacients fumadors i hipertensos. S'ha obtingut diferents models basats en el Núvol i s'han utilitzat per a desenvolupar una infraestructura on desplegar aquestes aplicacions. Els resultats mostren que aquestes aplicacions milloren els tractaments actuals així com escalen a mesura que els requeriments computacionals augmenten. Múltiples arquitectures i models han estat analitzats i implementats utilitzant diferents tècniques i escenaris. L'aplicació Smoking Patient Control (S-PC) ha estat desplegada i provada en un entorn real, aconseguint un augment del 28,4% en l'absistinència a llarg termini de pacients fumadors. L'aplicació Hypertension Patient Control (H-PC) ha estat dissenyada i implementada amb èxit, i els seus límits computacionals han estat mesurats.Esta tesis explora ténicas, modelos y algoritmos para una gestión eficiente de sistemas en la Nube y como aplicarlo en el sector de la salud con el fin de mejorar los tratamientos actuales. Presenta dos aplicaciones de salud electrónica basadas en la Nube para telemonitorizar y controlar pacientes fumadores e hipertensos. Se han obtenido diferentes modelos basados en la Nube y se han utilizado para desarrollar una infraestructura donde desplegar estas aplicaciones. Los resultados muestran que estas aplicaciones mejoran los tratamientos actuales así como escalan a medida que los requerimientos computacionales aumentan. Múltiples arquitecturas y modelos han sido analizados e implementados utilizando diferentes técnicas y escenarios. La aplicación Smoking Patient Control (S-PC) se ha desplegado y provado en un entorno real, consiguiendo un aumento del 28,4% en la abstinencia a largo plazo de pacientes fumadores. La aplicación Hypertension Patient Control (H-PC) ha sido diseñada e implementada con éxito, y sus límites computacionales han sido medidos

Knowledge-based web services for context adaptation.

The need for higher value, reliable online services to promote new Internet-based business models is a requirement facing many technologists and business leaders. This need coupled with the trend towards greater mobility of networked devices and consumers creates significant challenges for current and future systems developers. The proliferation of mobile devices and the variability of their capabilities present an overwhelming number of options to systems designers and engineers who are tasked with the development of next generation context adaptive software services. Given the dynamic nature of this environment, implementing solutions for the current set of devices in the held makes an assumption that this deployment situation is somehow fixed this assumption does little to support the future and longer term needs within the marketplace. To add to the complexity, the timeframes necessary to develop robust and adaptive online software services can be long by comparison, so that the development projects and their resources are often behind on platform support before the first release is launched to the public. New approaches and methodologies for engineering dynamic and adaptive online services will be necessary and, as will be shown, are in fact mandated by the regulation imposed by service level guarantees. These new techniques and technology are commercially useless unless they can be used in engineering practice. New context adaptation processes and architectures must be capable of performing under strict service level agreements those that will undoubtedly govern future business relationships between online parties. This programme of engineering study and research investigates several key issues found in the emerging area of context adaptation services for online mobile networks. As a series of engineering investigations, the work described here involves a wider array of technical activity than found in traditional doctoral work and this is reflected throughout the dissertation. First, a clear definition of industrial motivation is stated to provide the engineering foundation. Next, the programme focuses on the nature of contextual adaptation through product development projects. The development process within these projects results in several issues with the commercial feasibility of the technology. From this point, the programme of study then progresses through the lifecycle of the engineering process, investigating at each stage the critical engineering challenges. Further analysis of the problems and possible solutions for deploying such adaptive solutions are reviewed and experiments are undertaken in the areas of systems component and performance analysis. System-wide architectural options are then evaluated with specific interest in using knowledge-base systems as one approach to solving some of the issues in context adaptation. The central hypothesis is that due to the dynamic nature of context parameters, the concept of a mobile device knowledge base as a necessary component of an architectural solution is presented and justified through prototyping efforts. The utility of web ontologies and other "soft computing" technologies on the nature of the solution are also examined through the review of relevant work and the engineering design of the demonstration system. These technology selections are supported directly by the industrial context and mission. In the final sections, the architecture is evaluated through the demonstration of promising techniques and methods in order to confirm understanding and to evaluate the use of knowledge-bases, AI and other technologies within the scope of the project. Through the implementation of a context adaptation architecture as a business process workflow, the impact of future trends of device reconfiguration are highlighted and discussed. To address the challenge of context adaptation in reconftgurable device architectures, an evolutionary computation approach is then presented as a means to provide an optimal baseline on which a service may execute. These last two techniques are discussed and new designs are proposed to specifically address the major issues uncovered in timely collection and evaluation of contextual parameters in a mobile service network. The programme summary and future work then brings together all the key results into a practitioner's reference guide for the creation of online context adaptive services with a greater degree of intelligence and maintainability while executing with the term of a service level agreement

3D analytical modelling and iterative solution for high performance computing clusters

Mobile Cloud Computing enables the migration of services to the edge of the Internet. Therefore, high-performance computing clusters are widely deployed to improve computational capabilities of such environments. However, they are prone to failures and need analytical models to predict their behaviour in order to deliver desired quality-of-service and quality-of-experience to mobile users. This paper proposes a 3D analytical model and a problem-solving approach for sustainability evaluation of high-performance computing clusters. The proposed solution uses an iterative approach to obtain performance measurements to overcome the state space explosion problem. The availability modelling and evaluation of master and computing nodes are performed using a multi-repairman approach. The optimum number of repairmen is also obtained to get realistic results and reduce the overall cost. The proposed model is validated using discrete event simulation. The analytical approach is much faster and in good agreement with the simulations. The analysis focuses on mean queue length, throughput, and mean response time outputs. The maximum differences between analytical and simulation results in the considered scenarios of up to a billion states are less than1.149%,3.82%, and3.76%respectively. These differences are well within the5%of confidence interval of the simulation and the proposed model

Model-driven Scheduling for Distributed Stream Processing Systems

Distributed Stream Processing frameworks are being commonly used with the evolution of Internet of Things(IoT). These frameworks are designed to adapt to the dynamic input message rate by scaling in/out.Apache Storm, originally developed by Twitter is a widely used stream processing engine while others includes Flink, Spark streaming. For running the streaming applications successfully there is need to know the optimal resource requirement, as over-estimation of resources adds extra cost.So we need some strategy to come up with the optimal resource requirement for a given streaming application. In this article, we propose a model-driven approach for scheduling streaming applications that effectively utilizes a priori knowledge of the applications to provide predictable scheduling behavior. Specifically, we use application performance models to offer reliable estimates of the resource allocation required. Further, this intuition also drives resource mapping, and helps narrow the estimated and actual dataflow performance and resource utilization. Together, this model-driven scheduling approach gives a predictable application performance and resource utilization behavior for executing a given DSPS application at a target input stream rate on distributed resources.Comment: 54 page