Continuous Monitoring of Software Services: Design and Application of the Kieker Framework

In addition to studying the construction and evolution of software services, the software engineering discipline needs to address the operation of continuously running software services. A requirement for its robust operation are means for effective monitoring of software runtime behavior. In contrast to profiling for construction activities, monitoring of operational services should only impose a small performance overhead. Furthermore, instrumentation should be non-intrusive to the business logic, as far as possible. We present the Kieker framework for monitoring software runtime behavior, e.g., internal performance or (distributed) trace data. The flexible architecture allows to replace or add framework components, including monitoring probes, analysis components, and monitoring record types shared by logging and analysis. As a non-intrusive instrumentation technique, Kieker currently employs, but is not restricted to, aspect-oriented programming. An extensive lab study evaluates and quantifies the low overhead caused by the framework components. Qualitative evaluations provided by industrial case studies demonstrate the practicality of the approach with a telecommunication customer self service and a digital photo submission service. Kieker is available as open-source software, where both the academic and industrial partners contribute to the code. Our experiment data is publicly available, allowing interested researchers to repeat and extend our lab experiments

Run-time Architecture Models for Dynamic Adaptation and Evolution of Cloud Applications

Cloud applications are subject to continuous change due to modifications of the software application itself and, in particular, its environment. To manage changes, cloud-based systems provide diverse self-adaptation mechanisms based on run-time models. Observed run-time models are means for leveraging self- adaption, however, are hard to apply during software evolution as they are usually too detailed for comprehension by humans.In this paper, we propose iObserve, an approach to cloud-based system adaptation and evolution through run-time observation and continuous quality analysis. With iObserve, run-time adaptation and evolution are two mutual, interwoven activities that influence each other. Central to iObserve is (a) the specification of the correspondence between observation results and design models, and (b) their use in both adaptation and evolution. Run-time observation data is promoted to meaningful values mapped to design models, thereby continuously updating and calibrating those design models during run-time while keeping the models comprehendible by humans. This engineering approach allows for automated adaptation at run-time and simultaneously supports software evolution. Model-driven software engineering is employed for various purposes such as monitoring instrumentation and model transformation. We report on the experimental evaluation of this approach in lab experiments using the CoCoME benchmark deployed on an OpenStack cloud

Architecture-based Evolution of Dependable Software-intensive Systems

This cumulative habilitation thesis, proposes concepts for (i) modelling and analysing dependability based on architectural models of software-intensive systems early in development, (ii) decomposition and composition of modelling languages and analysis techniques to enable more flexibility in evolution, and (iii) bridging the divergent levels of abstraction between data of the operation phase, architectural models and source code of the development phase

AutoTaSC : Model driven development for autonomic software engineering

Whilst much research progress has been achieved towards the development of autonomic software engineering tools and techniques including: policy-based management, modelbased development, service-oriented architecture and model driven architecture. They have often focused on and started from chosen object-oriented models of required software behaviour, rather than domain model including user intentions and/or software goals. Such an approach is often reported to lead to "misalignment" between business process layer and their associated computational enabling systems. This is specifically noticeable in adaptive and evolving business systems and/or processes settings. To address this long-standing problem research has over the years investigated many avenues to close the gap between business process modelling and the generation of enactment (computation) layer, which is responsive to business changes. Within this problem domain, this research sets out to study the extension of the Model Driven Development (MOD) paradigm to business/domain model, that is, how to raise the abstraction level of model-driven software development to the domain level and provide model synchronisation to trace and analyse the impact of a given model change. The main contribution of this research is the development of a MOD-based design method for autonomic systems referred to as AutoTaSC. The latter consists of a series of related models, where each of which represents the system under development at a given stage. The first and highest level model represents the abstract model referred to as the Platform Independent Model (PIM). The next model encapsulates the PIM model for the autonomic system where the autonomic capabilities and required components (such as monitor, sensor, actuator, analyser, policy, etc.) are added via some appropriate transformation rules. Targeting a specific technology involves adding, also via transformation rules, specific information related to that platform from which the Platform Specific Model (PSM) for the autonomic system is extracted. In the last stage, code can be generated for the specific platform or technology targeted in the previous stage, web services for instance. In addition, the AutoTaSC method provides a situated model synchronisation mechanism, which is designed following the autonomic systems principles. For instance, to guarantee model synchronisation each model from each AutoTaSC stage has an associated policy-based feedback control loop, which regulates its reaction to detected model change. Thus, AutaTase method model transformation approach to drive model query, view and synchronisation. The Auto'Iast? method was evaluated using a number of benchmark case-studies to test this research hypothesis including the effectiveness and generality of AutaTaSe design method

Certifications of Critical Systems – The CECRIS Experience

In recent years, a considerable amount of effort has been devoted, both in industry and academia, to the development, validation and verification of critical systems, i.e. those systems whose malfunctions or failures reach a critical level both in terms of risks to human life as well as having a large economic impact.Certifications of Critical Systems – The CECRIS Experience documents the main insights on Cost Effective Verification and Validation processes that were gained during work in the European Research Project CECRIS (acronym for Certification of Critical Systems). The objective of the research was to tackle the challenges of certification by focusing on those aspects that turn out to be more difficult/important for current and future critical systems industry: the effective use of methodologies, processes and tools.The CECRIS project took a step forward in the growing field of development, verification and validation and certification of critical systems. It focused on the more difficult/important aspects of critical system development, verification and validation and certification process. Starting from both the scientific and industrial state of the art methodologies for system development and the impact of their usage on the verification and validation and certification of critical systems, the project aimed at developing strategies and techniques supported by automatic or semi-automatic tools and methods for these activities, setting guidelines to support engineers during the planning of the verification and validation phases

Certifications of Critical Systems – The CECRIS Experience

In recent years, a considerable amount of effort has been devoted, both in industry and academia, to the development, validation and verification of critical systems, i.e. those systems whose malfunctions or failures reach a critical level both in terms of risks to human life as well as having a large economic impact.Certifications of Critical Systems – The CECRIS Experience documents the main insights on Cost Effective Verification and Validation processes that were gained during work in the European Research Project CECRIS (acronym for Certification of Critical Systems). The objective of the research was to tackle the challenges of certification by focusing on those aspects that turn out to be more difficult/important for current and future critical systems industry: the effective use of methodologies, processes and tools.The CECRIS project took a step forward in the growing field of development, verification and validation and certification of critical systems. It focused on the more difficult/important aspects of critical system development, verification and validation and certification process. Starting from both the scientific and industrial state of the art methodologies for system development and the impact of their usage on the verification and validation and certification of critical systems, the project aimed at developing strategies and techniques supported by automatic or semi-automatic tools and methods for these activities, setting guidelines to support engineers during the planning of the verification and validation phases

Performance Benchmarking of Application Monitoring Frameworks

Application-level monitoring of continuously operating software systems provides insights into their dynamic behavior, helping to maintain their performance and availability during runtime. Such monitoring may cause a significant runtime overhead to the monitored system, depending on the number and location of used instrumentation probes. In order to improve a system’s instrumentation and to reduce the caused monitoring overhead, it is necessary to know the performance impact of each probe. While many monitoring frameworks are claiming to have minimal impact on the performance, these claims are often not backed up with a detailed performance evaluation determining the actual cost of monitoring. Benchmarks can be used as an effective and affordable way for these evaluations. However, no benchmark specifically targeting the overhead of monitoring itself exists. Furthermore, no established benchmark engineering methodology exists that provides guidelines for the design, execution, and analysis of benchmarks. This thesis introduces a benchmark approach to measure the performance overhead of application-level monitoring frameworks. The core contributions of this approach are 1) a definition of common causes of monitoring overhead, 2) a general benchmark engineering methodology, 3) the MooBench micro-benchmark to measure and quantify causes of monitoring overhead, and 4) detailed performance evaluations of three different application-level monitoring frameworks. Extensive experiments demonstrate the feasibility and practicality of the approach and validate the benchmark results. The developed benchmark is available as open source software and the results of all experiments are available for download to facilitate further validation and replication of the results

A Model-Driven Approach for the Design, Implementation, and Execution of Software Development Methods

[EN] Software development projects are diverse in nature. For this reason, software companies are often forced to define their methods in-house. In order to define methods efficiently and effectively, software companies require systematic solutions that are built upon sound methodical foundations. Providing these solutions is the main goal of the Method Engineering discipline. Method Engineering is the discipline to design, construct, and adapt methods, techniques, and tools for the development of information systems. Over the last two decades, a lot of research work has been performed in this area. However, despite its potential benefits, Method Engineering is not widely used in industrial settings. Some of the causes of this reality are the high theoretical complexity of Method Engineering and the lack of adequate software support. In this thesis, we aim to mitigate some of the problems that affect Method Engineering by providing a novel methodological approach that is built upon Model-Driven Engineering (MDE) foundations. The use of MDE enables a rise in abstraction, automation, and reuse that allows us to alleviate the complexity of our Method Engineering approach. Furthermore, by leveraging MDE techniques (such as metamodeling, model transformations, and models at runtime), our approach supports three phases of the Method Engineering lifecycle: design, implementation, and execution. This is unlike traditional Method Engineering approaches, which, in general, only support one of these phases. In order to provide software support for our proposal, we developed a Computer-Aided Method Engineering (CAME) environment that is called MOSKitt4ME. To ensure that MOSKitt4ME offered the necessary functionality, we identified a set of functional requirements prior to developing the tool. Then, after these requirements were identified, we defined the architecture of our CAME environment, and, finally, we implemented the architecture in the context of Eclipse. The thesis work was evaluated by means of a study that involved the participation of end users. In this study, MOSKitt4ME was assessed by means of the Technology Acceptance Model (TAM) and the Think Aloud method. While the TAM allowed us to measure usefulness and ease of use in a subjective manner, the Think Aloud method allowed us to analyze these measures objectively. Overall, the results were favorable. MOSKitt4ME was highly rated in perceived usefulness and ease of use; we also obtained positive results with respect to the users' actual performance and the difficulty experienced.[ES] Los proyectos de desarrollo de software son diversos por naturaleza. Por este motivo, las compañías de software se ven forzadas frecuentemente a definir sus métodos de manera interna. Para poder definir métodos de forma efectiva y eficiente, las compañías necesitan soluciones sistemáticas que estén definidas sobre unos fundamentos metodológicos sólidos. Proporcionar estas soluciones es el principal objetivo de la Ingeniería de Métodos. La Ingeniería de Métodos es la disciplina que aborda el diseño, la construcción y la adaptación de métodos, técnicas y herramientas para el desarrollo de sistemas de información. Durante las dos últimas décadas, se ha llevado a cabo mucho trabajo de investigación en esta área. Sin embargo, pese a sus potenciales beneficios, la Ingeniería de Métodos no se aplica ampliamente en contextos industriales. Algunas de las principales causas de esta situación son la alta complejidad teórica de la Ingeniería de Métodos y la falta de un apropiado soporte software. En esta tesis, pretendemos mitigar algunos de los problemas que afectan a la Ingeniería de Métodos proporcionando una propuesta metodológica innovadora que está basada en la Ingeniería Dirigida por Modelos (MDE). El uso de MDE permite elevar el nivel de abstracción, automatización y reuso, lo que posibilita una reducción de la complejidad de nuestra propuesta. Además, aprovechando técnicas de MDE (como por ejemplo el metamodelado, las transformaciones de modelos y los modelos en tiempo de ejecución), nuestra aproximación da soporte a tres fases del ciclo de vida de la Ingeniería de Métodos: diseño, implementación y ejecución. Esto es a diferencia de las propuestas existentes, las cuales, por lo general, sólo dan soporte a una de estas fases. Con el objetivo de proporcionar soporte software para nuestra propuesta, implementamos una herramienta CAME (Computer-Aided Method Engineering) llamada MOSKitt4ME. Para garantizar que MOSKitt4ME proporcionaba la funcionalidad necesaria, definimos un conjunto de requisitos funcionales como paso previo al desarrollo de la herramienta. Tras la definción de estos requisitos, definimos la arquitectura de la herramienta CAME y, finalmente, implementamos la arquitectura en el contexto de Eclipse. El trabajo desarrollado en esta tesis se evaluó por medio de un estudio donde participaron usuarios finales. En este estudio, MOSKitt4ME se evaluó por medio del Technology Acceptance Model (TAM) y del método Think Aloud. Mientras que el TAM permitió medir utilidad y facilidad de uso de forma subjetiva, el método Think Aloud permitió analizar estas medidas objetivamente. En general, los resultados obtenidos fueron favorables. MOSKitt4ME fue valorado de forma positiva en cuanto a utilidad y facilidad de uso percibida; además, obtuvimos resultados positivos en cuanto al rendimiento objetivo de los usuarios y la dificultad experimentada.[CA] Els projectes de desenvolupament de programari són diversos per naturalesa. Per aquest motiu, les companyies es veuen forçades freqüenment a definir els seus mètodes de manera interna. Per poder definir mètodes de forma efectiva i eficient, les companyies necessiten solucions sistemàtiques que estiguin definides sobre uns fundaments metodològics sòlids. Proporcionar aquestes solucions és el principal objectiu de l'Enginyeria de Mètodes. L'Enginyeria de Mètodes és la disciplina que aborda el diseny, la construcció i l'adaptació de mètodes, tècniques i eines per al desenvolupament de sistemes d'informació. Durant les dues últimes dècades, s'ha dut a terme molt de treball de recerca en aquesta àrea. No obstant, malgrat els seus potencials beneficis, l'Enginyeria de Mètodes no s'aplica àmpliament en contextes industrials. Algunes de les principals causes d'aquesta situació són l'alta complexitat teòrica de l'Enginyeria de Mètodes i la falta d'un apropiat suport de programari. En aquesta tesi, pretenem mitigar alguns dels problemes que afecten a l'Enginyeria de Mètodes proporcionant una proposta metodològica innovadora que està basada en l'Enginyeria Dirigida per Models (MDE). L'ús de MDE ens permet elevar el nivell d'abstracció, automatització i reutilització, possibilitant una reducció de la complexitat de la nostra proposta. A més a més, aprofitant tècniques de MDE (com per exemple el metamodelat, les transformacions de models i els models en temps d'execució), la nostra aproximació suporta tres fases del cicle de vida de l'Enginyeria de Mètodes: diseny, implementació i execució. Açò és a diferència de les propostes existents, les quals, en general, només suporten una d'aquestes fases. Amb l'objectiu de proporcionar suport de programari per a la nostra proposta, implementàrem una eina CAME (Computer-Aided Method Engineering) anomenada MOSKitt4ME. Per garantir que MOSKitt4ME oferia la funcionalitat necessària, definírem un conjunt de requisits funcionals com a pas previ al desenvolupament de l'eina. Després de la definició d'aquests requisits, definírem la arquitectura de l'eina CAME i, finalment, implementàrem l'arquitectura en el contexte d'Eclipse. El treball desenvolupat en aquesta tesi es va avaluar per mitjà d'un estudi on van participar usuaris finals. En aquest estudi, MOSKitt4ME es va avaluar per mitjà del Technology Acceptance Model (TAM) i el mètode Think Aloud. Mentre que el TAM va permetre mesurar utilitat i facilitat d'ús de manera subjectiva, el mètode Think Aloud va permetre analitzar aquestes mesures objectivament. En general, els resultats obtinguts van ser favorables. MOSKitt4ME va ser valorat de forma positiva pel que fa a utilitat i facilitat d'ús percebuda; a més a més, vam obtenir resultats positius pel que fa al rendiment objectiu dels usuaris i a la dificultat experimentada.Cervera Úbeda, M. (2015). A Model-Driven Approach for the Design, Implementation, and Execution of Software Development Methods [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/53931TESI

Automatic generation of software interfaces for supporting decisionmaking processes. An application of domain engineering & machine learning

[EN] Data analysis is a key process to foster knowledge generation in particular domains or fields of study. With a strong informative foundation derived from the analysis of collected data, decision-makers can make strategic choices with the aim of obtaining valuable benefits in their specific areas of action. However, given the steady growth of data volumes, data analysis needs to rely on powerful tools to enable knowledge extraction. Information dashboards offer a software solution to analyze large volumes of data visually to identify patterns and relations and make decisions according to the presented information. But decision-makers may have different goals and, consequently, different necessities regarding their dashboards. Moreover, the variety of data sources, structures, and domains can hamper the design and implementation of these tools. This Ph.D. Thesis tackles the challenge of improving the development process of information dashboards and data visualizations while enhancing their quality and features in terms of personalization, usability, and flexibility, among others. Several research activities have been carried out to support this thesis. First, a systematic literature mapping and review was performed to analyze different methodologies and solutions related to the automatic generation of tailored information dashboards. The outcomes of the review led to the selection of a modeldriven approach in combination with the software product line paradigm to deal with the automatic generation of information dashboards. In this context, a meta-model was developed following a domain engineering approach. This meta-model represents the skeleton of information dashboards and data visualizations through the abstraction of their components and features and has been the backbone of the subsequent generative pipeline of these tools. The meta-model and generative pipeline have been tested through their integration in different scenarios, both theoretical and practical. Regarding the theoretical dimension of the research, the meta-model has been successfully integrated with other meta-model to support knowledge generation in learning ecosystems, and as a framework to conceptualize and instantiate information dashboards in different domains. In terms of the practical applications, the focus has been put on how to transform the meta-model into an instance adapted to a specific context, and how to finally transform this later model into code, i.e., the final, functional product. These practical scenarios involved the automatic generation of dashboards in the context of a Ph.D. Programme, the application of Artificial Intelligence algorithms in the process, and the development of a graphical instantiation platform that combines the meta-model and the generative pipeline into a visual generation system. Finally, different case studies have been conducted in the employment and employability, health, and education domains. The number of applications of the meta-model in theoretical and practical dimensions and domains is also a result itself. Every outcome associated to this thesis is driven by the dashboard meta-model, which also proves its versatility and flexibility when it comes to conceptualize, generate, and capture knowledge related to dashboards and data visualizations

Security Analysis of System Behaviour - From "Security by Design" to "Security at Runtime" -

The Internet today provides the environment for novel applications and processes which may evolve way beyond pre-planned scope and purpose. Security analysis is growing in complexity with the increase in functionality, connectivity, and dynamics of current electronic business processes. Technical processes within critical infrastructures also have to cope with these developments. To tackle the complexity of the security analysis, the application of models is becoming standard practice. However, model-based support for security analysis is not only needed in pre-operational phases but also during process execution, in order to provide situational security awareness at runtime. This cumulative thesis provides three major contributions to modelling methodology. Firstly, this thesis provides an approach for model-based analysis and verification of security and safety properties in order to support fault prevention and fault removal in system design or redesign. Furthermore, some construction principles for the design of well-behaved scalable systems are given. The second topic is the analysis of the exposition of vulnerabilities in the software components of networked systems to exploitation by internal or external threats. This kind of fault forecasting allows the security assessment of alternative system configurations and security policies. Validation and deployment of security policies that minimise the attack surface can now improve fault tolerance and mitigate the impact of successful attacks. Thirdly, the approach is extended to runtime applicability. An observing system monitors an event stream from the observed system with the aim to detect faults - deviations from the specified behaviour or security compliance violations - at runtime. Furthermore, knowledge about the expected behaviour given by an operational model is used to predict faults in the near future. Building on this, a holistic security management strategy is proposed. The architecture of the observing system is described and the applicability of model-based security analysis at runtime is demonstrated utilising processes from several industrial scenarios. The results of this cumulative thesis are provided by 19 selected peer-reviewed papers