Transfer Learning for Improving Model Predictions in Highly Configurable Software

Modern software systems are built to be used in dynamic environments using configuration capabilities to adapt to changes and external uncertainties. In a self-adaptation context, we are often interested in reasoning about the performance of the systems under different configurations. Usually, we learn a black-box model based on real measurements to predict the performance of the system given a specific configuration. However, as modern systems become more complex, there are many configuration parameters that may interact and we end up learning an exponentially large configuration space. Naturally, this does not scale when relying on real measurements in the actual changing environment. We propose a different solution: Instead of taking the measurements from the real system, we learn the model using samples from other sources, such as simulators that approximate performance of the real system at low cost. We define a cost model that transform the traditional view of model learning into a multi-objective problem that not only takes into account model accuracy but also measurements effort as well. We evaluate our cost-aware transfer learning solution using real-world configurable software including (i) a robotic system, (ii) 3 different stream processing applications, and (iii) a NoSQL database system. The experimental results demonstrate that our approach can achieve (a) a high prediction accuracy, as well as (b) a high model reliability.Comment: To be published in the proceedings of the 12th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS'17

BestConfig: Tapping the Performance Potential of Systems via Automatic Configuration Tuning

An ever increasing number of configuration parameters are provided to system users. But many users have used one configuration setting across different workloads, leaving untapped the performance potential of systems. A good configuration setting can greatly improve the performance of a deployed system under certain workloads. But with tens or hundreds of parameters, it becomes a highly costly task to decide which configuration setting leads to the best performance. While such task requires the strong expertise in both the system and the application, users commonly lack such expertise. To help users tap the performance potential of systems, we present BestConfig, a system for automatically finding a best configuration setting within a resource limit for a deployed system under a given application workload. BestConfig is designed with an extensible architecture to automate the configuration tuning for general systems. To tune system configurations within a resource limit, we propose the divide-and-diverge sampling method and the recursive bound-and-search algorithm. BestConfig can improve the throughput of Tomcat by 75%, that of Cassandra by 63%, that of MySQL by 430%, and reduce the running time of Hive join job by about 50% and that of Spark join job by about 80%, solely by configuration adjustment

A Framework for the Automatic Physical Configuration and Tuning of a Mysql Community Server

Manual physical configuration and tuning of database servers, is a complicated task requiring a high level of expertise. Database administrators must consider numerous possibilities, to determine a candidate configuration for implementation. In recent times database vendors have responded to this problem, providing solutions which can automatically configure and tune their products. Poor configuration choices, resulting in performance degradation commonplace in manual configurations, have been significantly reduced in these solutions. However, no such solution exists for MySQL Community Server. This thesis, proposes a novel framework for automatically tuning a MySQL Community Server. A first iteration of the framework has been built and is presented in this paper together with its performance measurements

Optimizing System Configurations Quickly by Guessing at the Performance

The performance of a Web system can be greatly improved by tuning its configuration parameters. However, finding the optimal configuration has been a time-consuming task due to the long measurement time needed to evaluate the performance of a given configuration. We propose an algorithm, which we refer to as Quick Optimization via Guessing (QOG), that quickly selects one of nearly best configurations with high probability. The key ideas in QOG are (i) the measurement of a configuration is terminated as soon as the configuration is found to be suboptimal, and (ii) the performance of a configuration is guessed at based on the measured similar configurations, so that the better configurations are more likely to be measured before the others. If the performance of a good configuration has been measured, a poor configuration will be quickly found to be suboptimal with short measurement time. We apply QOG to optimizing the configuration of a real Web system, and find that QOG can drastically reduce the total measurement time needed to select the best configuration. Our experiments also illuminate several interesting properties of QOG specifically when it is applied to optimizing Web systems

Improving web server efficiency on commodity hardware

El ràpid creixement de la Web requereix una gran quantitat de recursos computacionals que han de ser utilitzats eficientment. Avui en dia, els servidors basats en hardware estendard son les plataformes preferides per executar els servidors web, ja que són les plataformes amb millor relació rendiment/cost. El treball presentat en aquesta tesi esta dirigit a millorar la eficàcia en la gestió de recursos dels servidors web actuals. Per assolir els objectius d'aquesta tesis s'ha caracteritzat el funcionament dels servidors web en diverses entorns representatius, per tal de identificar el problemes i coll d'ampolla que limiten el rendiment del servidor web. Amb l'estudi dels servidors web s'ha identificat dos problemes principals que disminueixen l'eficiència dels servidors web en la utilització dels recursos hardware disponibles. El primer problema identificat és la evolució del protocol HTTP per incorporar connexions persistents i seguretat, que disminueix el rendiment e incrementa la complexitat de configuració dels servidors web. El segon problema és la naturalesa de algunes aplicacions web, les quals estan limitades per la memòria física o l'ample de banda amb el disc, que impedeix la correcta utilització dels recursos presents en les maquines multiprocessadors. Per solucionar aquests dos problemes dels servidors web hem proposat dues tècniques. En primer lloc, l'arquitectura hibrida, una evolució de l'arquitectura multi-threaded que es pot implementar fàcilment el els servidor web actuals i que millora notablement la gestió de les connexions i redueix la complexitat de configuració de tot el sistema. En segon lloc, hem implementat en el kernel del sistema operatiu Linux un comprensió de memòria principal per millorar el rendiment de les aplicacions que tenen la memòria com ha coll d'ampolla, millorant així la utilització dels recursos disponibles. Els resultats d'aquesta tesis estan avalats per una avaluació experimental exhaustiva que ha provat la efectivitat i viabilitat de les nostres propostes. Cal destacar que l'arquitectura de servidor web hybrida proposada en aquesta tesis ha estat implementada recentment per coneguts servidors web com és el cas de Apache, Tomcat i Glassfish.The unstoppable growth of the World Wide Web requires a huge amount of computational resources that must be used efficiently. Nowadays, commodity hardware is the preferred platform to run web server systems because it is the most cost-effective solution. The work presented in this thesis aims to improve the efficiency of current web server systems, allowing the web servers to make the most of hardware resources. To this end, we first characterize current web server system and identify the problems that hinder web servers from providing an efficient utilization of resources. From the study of web servers in a wide range of situations and environments, we have identified two main issues that prevents web servers systems from efficiently using current hardware resources. The first is the extension of the HTTP protocol to include connection persistence and security, which dramatically impacts the performance and configuration complexity of traditional multi-threaded web servers. The second is the memory-bounded or disk-bounded nature of some web workloads that prevents the full utilization of the abundant CPU resources available on current commodity hardware. We propose two novel techniques to overcome the main problems with current web server systems. Firstly, we propose a Hybrid web serverarchitecture which can be easily implemented in any multi-threaded web server to improve CPU utilization so as to provide better management of client connections. And secondly, we describe a main memory compression technique implemented in the Linux operating system that makes optimum use of current multiprocessor's hardware, in order to improve the performance of memory bound web applications. The thesis is supported by an exhaustive experimental evaluation that proves the effectiveness and feasibility of our proposals for current systems. It is worth noting that the main concepts behind the Hybrid architecture have recently been implemented in popular web servers like Apache, Tomcat and Glassfish

Improving reliability and performance of telecommunications systems by using autonomic, self-learning and self-adaptive systems

Meine Dissertation beschaeftigt sich mit autonomen, selbst-lernenden und selbst-adaptiven Systemen. Prinzipiell muss ein autonomes und selbst-lernendes System seinen eigenen Status, sowie die externen Operationen kennen, muss Systemveraenderungen erkennen koennen und muss in der Lage sein sich selbst zu adaptieren. Verbesserung der Zuverlaessigkeit von Multimedia Kommunikation: Im Zuge des Testens eines kommerziellen VoIP Servers wurde deutlich, dass das SIP Protokoll, welches fuer die Initiierung von VoIP Telefonaten verwendet wird, in einem sehr offenen Standard definiert ist. Fuer eine korrekte SIP Nachricht sind nur einige wenige Informationen notwendig. Es gibt allerdings eine enorme Anzahl an optionalen Informationen, die ebenfalls innerhalb einer SIP Nachricht verwendet werden koennen. Diese Tatsache fuehrt dazu, dass VoIP Geraete eine enorme Anzahl an unterschiedlichen SIP-Dialekten verwenden, die aus der riesigen Anzahl an unterschiedlichen Parameterkombinationen entstehen. Dies kann zu dem Problem fuehren, dass Telefone die dasselbe Protokoll verwenden, trotzdem nicht in der Lage sind\ud miteinander zu kommunizieren. Deshalb wird ein autonomes, selbst-lernendes SIP-Uebersetzungstool praesentiert, welches die Rate der faelschlich vom Server abgewiesenen SIP Nachrichten drastisch reduziert, indem ankommende Nachrichten analysiert und eventuell veraendert werden. Autonome Adaption von Systemparametern, um die Systemperformance zu verbessern: Die Performance eines kommerziellen Systems, welches Daten von unterschiedlichen mobilen Geraeten sammelt und verarbeitet, ist aufgrund des hohen ankommenden Datenaufkommens extrem wichtig. Ankommende Datentickets wandern durch ein Warteschlangensystem, wo in jedem durchlaufenen Knoten unterschiedliche atomare Aktionen durchgefuehrt werden. Dieser Aufbau ermoeglicht es, die einzelnen Knoten zu parallelisieren, in dem mehrere Auspraegungen der Knoten auf unterschiedlichen CPU-Kernen gestartet werden. Mit Hilfe eines Systems, welches analytische Ansaetze, Messungen und Simulationen verwendet, wird die optimale Softwarekonfiguration fuer eine bestimmte Hardware automatisiert gefunden. Dadurch passt sich die Software immer exakt an die aktuelle Hardware und an das aktuelle Datenaufkommen an. Die Performance des Gesamtsystems kann so drastisch verbessert werden.My dissertation will be about autonomic, self-learning and self-adaptive sys- tems. Usually an autonomic and self-learning system must be able to know its own status and the external operations, must be able to monitor system changes and must be able to self-adapt to them. Within this area my disser- tation will present two case studies of autonomic and self-learning systems. Improving reliability of multimedia communication: While testing a commercial VoIP server it became obvious that the SIP proto- col, used to initiate VoIP calls, is defined in a very open standard. That fact results in a great number of different SIP dialects, leading to the problem that some VoIP devices (hard and soft phones) may not be able to communicate with each other, even though they use the same protocol. Therefore an au- tonomic, self-learning SIP translator will be presented, that will decrease the rate of rejected SIP messages. Automatic adaptation of system parameters to improve system performance: The performance of a commercial system that collects data from various mo- bile devices is critical, because of the high amount of incoming data. There- fore performance tests will be initiated and automatically evaluated. Through self-learning techniques the system will self-adapt to the environment and the hardware on which the system is currently running, with the goal to improve the systems performance