Full-Stack Performance Model Evaluation using Probabilistic Garbage Collection Simulation

ABSTRACT Performance models can represent the performance relevant aspects of an enterprise application. Corresponding simulation engines use such models for simulating performance metrics (e.g., response times, resource utilization, throughput) and allow for performance evaluations without load testing the actual system. Creating such models manually often outweighs their benefits. Therefore, recent research created performance model generators, which can generate such models out of Application Performance Management software. However, a full-stack evaluation containing all relevant resources of an enterprise application (Central Processing Unit, memory, network and Hard Disk Drive) has not been conducted to the best of our knowledge. This work closes this gap using a pre-release version of the next generation industry benchmark SPECjEnterpriseNEXT of the Standard Performance Evaluation Corporation as example enterprise application, the Palladio Component Model as performance model and the performance model generator of the RETIT Capacity Manager. Furthermore, this work extends the generated model with a probabilistic garbage collection model to simulate memory allocation and releases more accurately

Automated extraction of palladio component models from running enterprise Java applications

Nowadays, software systems have to fulfill increasingly strin-gent requirements for performance and scalability. To ensure that a system meets its performance requirements during op-eration, the ability to predict its performance under different configurations and workloads is essential. Most performance analysis tools currently used in industry focus on monitoring the current system state. They provide low-level monitoring data without any performance prediction capabilities. For performance prediction, performance models are normally required. However, building predictive performance models manually requires a lot of time and effort. In this paper, we present a method for automated extraction of perfor-mance models of Java EE applications, based on monitor-ing data collected during operation. We extract instances of the Palladio Component Model (PCM)- a performance meta-model targeted at component-based systems. We eval-uate the model extraction method in the context of a case study with a real-world enterprise application. Even though the extraction requires some manual intervention, the case study demonstrates that the existing gap between low-level monitoring data and high-level performance models can be closed. 1

Continuous Monitoring of Software Services: Design and Application of the Kieker Framework

In addition to studying the construction and evolution of software services, the software engineering discipline needs to address the operation of continuously running software services. A requirement for its robust operation are means for effective monitoring of software runtime behavior. In contrast to profiling for construction activities, monitoring of operational services should only impose a small performance overhead. Furthermore, instrumentation should be non-intrusive to the business logic, as far as possible. We present the Kieker framework for monitoring software runtime behavior, e.g., internal performance or (distributed) trace data. The flexible architecture allows to replace or add framework components, including monitoring probes, analysis components, and monitoring record types shared by logging and analysis. As a non-intrusive instrumentation technique, Kieker currently employs, but is not restricted to, aspect-oriented programming. An extensive lab study evaluates and quantifies the low overhead caused by the framework components. Qualitative evaluations provided by industrial case studies demonstrate the practicality of the approach with a telecommunication customer self service and a digital photo submission service. Kieker is available as open-source software, where both the academic and industrial partners contribute to the code. Our experiment data is publicly available, allowing interested researchers to repeat and extend our lab experiments

Adapting a Stress Testing Framework to a Multi-module Security-oriented Spring Application

Programmeeritakse mitmekomponendilist süsteemi. Kolm põhikomponenti on järgmised: põhiserver (Spring rakendus), mobiilirakendused (iOS, Android), klienditeeninduse veebiportaalid. Kõige tähtsam süsteemi töös on põhiserver, kuna see on enamuse veebiportaalide ning mobiilirakenduste päringute sihtpunkt. See on mitmemooduliline projekt, kus kõik moodulid suhtlevad omavahel. Potentsiaalselt hakkab süsteemi kasutama sadu tuhandeid inimesi – kümneid tuhandeid paralleelseid sessioone. Seetõttu tuleb läbi viia süsteemi ulatuslik koormustestimine. Kahjuks on nii, et koormustestimise raamistikud oma originaalseisus ei sobi antud süsteemi testimiseks. Seega, koormustestimise raamistiku tuleb seadistada ning laiendada selleks, et see toetaks antud süsteemi spetsiifilisi protokolle ja võimaldaks testida kõiki komponente üheskoos. Hetkel on saadaval palju koormustestimise raamistikke. Mõned nendest on: Locust, Apache JMeter, Gatling Project. Need raamistikud erinevad üksteisest programmeerimiskeele, eriomaduste ning põhiloogika järgi. Kuna tegu on kommertsprojektiga, peab valitud koormustestimise raamistik vastama kliendi funktsionaalsete ja mittefunktsionaalsete nõuetele. Kuna koormustestimist viiakse läbi ainult põhiserveril, peab seadistama ja laiendama valitud raamistikku, et simuleerida teisi süsteemi komponente ja serveri protokolle. See töö annab kiire ülevaate varem mainitud koormustestimise raamistikest eriomaduste järgi, valib raamistiku, mida kohandatakse antud projekti raames koormustestimise läbi viimiseks ning kirjeldab kohandamise protsessi. Samuti toob see töö välja mõned koormustestimise raamistike piirangud ning kirjeldab meetodeid nende ületamiseks. Viimaks, süsteemi testitakse valitud raamistiku abil ning esitatakse ja valideeritakse tulemusi.A multi-component system is being build. Three main components are: backend server (Spring application), mobile applications (iOS, Android), customer service web portals. Our main concern is the backend server, because it is the destination of the majority of requests from customer service web portals and mobile applications. It is a multi-module project where all modules communicate to each other. The system is going to be used potentially by hundreds thousands of users with tens thousands of simultaneous usages. Therefore, extensive stress-testing must be conducted. Unfortunately, stress-testing frameworks in the original state are not suitable for the given system. Thus a stress-testing framework must be configured and extended to the point it supports the system’s specific protocols and can test all the system’s components together. There are numerous of stress-testing frameworks available. Some examples are: Locust, Apache JMeter, Gatling Project. These frameworks differ in terms of coding language, features and core logic. As it is a commercial project, the chosen stress-testing framework must also comply with client’s functional and non-functional requirements. Due to stress-testing being conducted only on the backend server component, the selected stress-testing framework must be configured/extended to simulate other components and the required server protocols. The thesis provides a brief comparison of the available stress-testing frameworks based on their features and written code language and define the one which is going to be adapted to conduct the stress-testing within the project and how the adaptation is done. The thesis also points out some of stress-testing frameworks’ limitations with techniques to overcome them. Finally, the system is tested using the selected testing framework and the results are presented and validated

A software development framework for secure microservices

Abstract: The software development community has seen the proliferation of a new style of building applications based on small and specialized autonomous units of computation logic called microservices. Microservices collaborate by sending light-weight messages to automate a business task. These microservices are independently deployable with arbitrary schedules, allowing enterprises to quickly create new sets of business capabilities in response to changing business requirements. It is expected that the use of microservices will become the default style of building software applications by the year 2023, with the microservices’ market projected to reach thirtytwo billion United States of American dollars. The adoption of microservices presents new security challenges due to the way the units of computation logic are designed, deployed and maintained. The decomposition of an application into small independent units increases the attack surface, and makes it a challenge to secure and control network traffic for each unit. These new security challenges cannot be addressed by traditional security strategies. Software engineers developing microservices are facing growing pressure to build secure microservices to ensure the security of business information assets and guarantee business continuity. The research conducted in this thesis proposes a software development framework that software engineers can use to build secure microservices. The framework defines artefacts, development and maintenance activities together with methods and techniques that software engineers can use to ensure that microservices are developed from the ground up to be secure. The goal of the framework is to ensure that microservices are designed and built to be able to detect, react, respond and recover from attacks during day-to-day operations. To prove the capability of the framework, a microservices-based application is developed using the proposed software development framework as part of an experiment to determine its effectiveness. These results, together with a comparative and quality review of the framework indicate that the software development framework can be effectively used to develop secure microservices.Ph.D. (Computer Science

Deriving Goal-oriented Performance Models by Systematic Experimentation

Performance modelling can require substantial effort when creating and maintaining performance models for software systems that are based on existing software. Therefore, this thesis addresses the challenge of performance prediction in such scenarios. It proposes a novel goal-oriented method for experimental, measurement-based performance modelling. We validated the approach in a number of case studies including standard industry benchmarks as well as a real development scenario at SAP

Self-Adaptive Performance Monitoring for Component-Based Software Systems

Effective monitoring of a software system’s runtime behavior is necessary to evaluate the compliance of performance objectives. This thesis has emerged in the context of the Kieker framework addressing application performance monitoring. The contribution includes a self-adaptive performance monitoring approach allowing for dynamic adaptation of the monitoring coverage at runtime. The monitoring data includes performance measures such as throughput and response time statistics, the utilization of system resources, as well as the inter- and intra-component control flow. Based on this data, performance anomaly scores are computed using time series analysis and clustering methods. The self-adaptive performance monitoring approach reduces the business-critical failure diagnosis time, as it saves time-consuming manual debugging activities. The approach and its underlying anomaly scores are extensively evaluated in lab experiments

Performance Problem Diagnostics by Systematic Experimentation

In this book, we introduce an automatic, experiment-based approach for performance problem diagnostics in enterprise software systems. The proposed approach systematically searches for root causes of detected performance problems by executing series of systematic performance tests. The presented approach is evaluated by various case studies showing that the presented approach is applicable to a wide range of contexts

Performance Problem Diagnostics by Systematic Experimentation

Diagnostics of performance problems requires deep expertise in performance engineering and entails a high manual effort. As a consequence, performance evaluations are postponed to the last minute of the development process. In this thesis, we introduce an automatic, experiment-based approach for performance problem diagnostics in enterprise software systems. With this approach, performance engineers can concentrate on their core competences instead of conducting repeating tasks