Design-time performance analysis of component-based real-time systems

In current real-time systems, performance metrics are one of the most challenging properties to specify, predict and measure. Performance properties depend on various factors, like environmental context, load profile, middleware, operating system, hardware platform and sharing of internal resources. Performance failures and not satisfying related requirements cause delays, cost overruns, and even abandonment of projects. In order to avoid these performancerelated project failures, the performance properties should be obtained and analyzed already at the early design phase of a project. In this thesis we employ principles of component-based software engineering (CBSE), which enable building software systems from individual components. The advantage of CBSE is that individual components can be modeled, reused and traded. The main objective of this thesis is to develop a method that enables to predict the performance properties of a system, based on the performance properties of the involved individual components. The prediction method serves rapid prototyping and performance analysis of the architecture or related alternatives, without performing the usual testing and implementation stages. The involved research questions are as follows. How should the behaviour and performance properties of individual components be specified in order to enable automated composition of these properties into an analyzable model of a complete system? How to synthesize the models of individual components into a model of a complete system in an automated way, such that the resulting system model can be analyzed against the performance properties? The thesis presents a new framework called DeepCompass, which realizes the concept of predictable assembly throughout all phases of the system design. The cornerstones of the framework are the composable models of individual software components and hardware blocks. The models are specified at the component development time and shipped in a component package. At the component composition phase, the models of the constituent components are synthesized into an executable system model. Since the thesis focuses on performance properties, we introduce performance-related types of component models, such as behaviour, performance and resource models. The dynamics of the system execution are captured in scenario models. The essential advantage of the introduced models is that, through the behaviour of individual components and scenario models, the behaviour of the complete system is synthesized in the executable system model. Further simulation-based analysis of the obtained executable system model provides application-specific and system-specific performance property values. To support the performance analysis, we have developed a CARAT software toolkit that provides and automates the algorithms for model synthesis and simulation. Besides this, the toolkit provides graphical tools for designing alternative architectures and visualization of obtained performance properties. We have conducted an empirical case study on the use of scenarios in the industry to analyze the system performance at the early design phase. It was found that industrial architects make extensive use of scenarios for performance evaluation. Based on the inputs of the architects, we have provided a set of guidelines for identification and use of performance-critical scenarios. At the end of this thesis, we have validated the DeepCompass framework by performing three case studies on performance prediction of real-time systems: an MPEG-4 video decoder, a Car Radio Navigation system and a JPEG application. For each case study, we have constructed models of the individual components, defined the SW/HW architecture, and used the CARAT toolkit to synthesize and simulate the executable system model. The simulation provided the predicted performance properties, which we later compared with the actual performance properties of the realized systems. With respect to resource usage properties and average task latencies, the variation of the prediction error showed to be within 30% of the actual performance. Concerning the pick loads on the processor nodes, the actual values were sometimes three times larger than the predicted values. As a conclusion, the framework has proven to be effective in rapid architecture prototyping and performance analysis of a complete system. This is valid, as in the case studies we have spent not more than 4-5 days on the average for the complete iteration cycle, including the design of several architecture alternatives. The framework can handle different architectural styles, which makes it widely applicable. A conceptual limitation of the framework is that it assumes that the models of individual components are already available at the design phase

ExTrA: Explaining architectural design tradeoff spaces via dimensionality reduction

In software design, guaranteeing the correctness of run-time system behavior while achieving an acceptable balance among multiple quality attributes remains a challenging problem. Moreover, providing guarantees about the satisfaction of those requirements when systems are subject to uncertain environments is even more challenging. While recent developments in architectural analysis techniques can assist architects in exploring the satisfaction of quantitative guarantees across the design space, existing approaches are still limited because they do not explicitly link design decisions to satisfaction of quality requirements. Furthermore, the amount of information they yield can be overwhelming to a human designer, making it difficult to see the forest for the trees. In this paper we present ExTrA (Explaining Tradeoffs of software Architecture design spaces), an approach to analyzing architectural design spaces that addresses these limitations and provides a basis for explaining design tradeoffs. Our approach employs dimensionality reduction techniques employed in machine learning pipelines like Principal Component Analysis (PCA) and Decision Tree Learning (DTL) to enable architects to understand how design decisions contribute to the satisfaction of extra-functional properties across the design space. Our results show feasibility of the approach in two case studies and evidence that combining complementary techniques like PCA and DTL is a viable approach to facilitate comprehension of tradeoffs in poorly-understood design spaces

An Approach for Guiding Developers to Performance and Scalability Solutions

This thesis proposes an approach that enables developers who are novices in software performance engineering to solve software performance and scalability problems without the assistance of a software performance expert. The contribution of this thesis is the explicit consideration of the implementation level to recommend solutions for software performance and scalability problems. This includes a set of description languages for data representation and human computer interaction and a workflow

Exploring performance trade-offs of a JPEG decoder using the deepcompass framework

Designing embedded systems for multiprocessor platforms requires early prediction and balancing of multiple system quality attributes. We present a design space exploration framework for component-based software systems that allows an architect to get insight into a space of possible design alternatives with further evaluation and comparison of these alternatives. The framework provides (a) tool-guided design of multiple alternatives of software and hardware architectures, (b) early design-time predictions of performance properties and identification of bottlenecks for each architectural alternative, and (c) evaluation of each alternative with respect to multi-objective trade-offs. The performance prediction technique employs modeling of individual components and composition of the models into a system model representing the system behaviour and resource usage. We illustrate the framework by a case study of a JPEG decoder application. For this system, we consider architectural alternatives, show their specification, and explore their trade-offs with respect to task latencies, resource utilization and system cost

Exploring performance trade-offs of a JPEG decoder using the deepcompass framework

Designing embedded systems for multiprocessor platforms requires early prediction and balancing of multiple system quality attributes. We present a design space exploration framework for component-based software systems that allows an architect to get insight into a space of possible design alternatives with further evaluation and comparison of these alternatives. The framework provides (a) tool-guided design of multiple alternatives of software and hardware architectures, (b) early design-time predictions of performance properties and identification of bottlenecks for each architectural alternative, and (c) evaluation of each alternative with respect to multi-objective trade-offs. The performance prediction technique employs modeling of individual components and composition of the models into a system model representing the system behaviour and resource usage. We illustrate the framework by a case study of a JPEG decoder application. For this system, we consider architectural alternatives, show their specification, and explore their trade-offs with respect to task latencies, resource utilization and system cost

Parameter dependencies for reusable performance specifications of software components

To avoid design-related per­for­mance problems, model-driven performance prediction methods analyse the response times, throughputs, and re­source utilizations of software architectures before and during implementation. This thesis proposes new modeling languages and according model transformations, which allow a reusable description of usage profile dependencies to the performance of software components. Predictions based on this new methods can support performance-related design decisions

Automated Improvement of Software Architecture Models for Performance and Other Quality Attributes

Quality attributes, such as performance or reliability, are crucial for the success of a software system and largely influenced by the software architecture. Their quantitative prediction supports systematic, goal-oriented software design and forms a base of an engineering approach to software design. This thesis proposes a method and tool to automatically improve component-based software architecture (CBA) models based on such quantitative quality prediction techniques

An Approach for Guiding Developers to Performance and Scalability Solutions

The quality of enterprise software applications plays a crucial role for the satisfaction of the users and the economic success of the enterprises. Software applications with unsatisfying performance and scalability are perceived by its users as low in quality, as less interesting and less attractive, and cause frustration when preventing the users from attaining their goals. This book proposes an approach for a recommendation system that enables developers who are novices in software perfor