Colored model based testing for software product lines (CMBT-SWPL)

Over the last decade, the software product line domain has emerged as one of the mostpromising software development paradigms. The main benefits of a software product lineapproach are improvements in productivity, time to market, product quality, and customersatisfaction.Therefore, one topic that needs greater emphasis is testing of software product lines toachieve the required software quality assurance. Our concern is how to test a softwareproduct line as early as possible in order to detect errors, because the cost of error detectedIn early phases is much less compared to the cost of errors when detected later.The method suggested in this thesis is a model-based, reuse-oriented test technique calledColored Model Based Testing for Software Product Lines (CMBT-SWPL). CMBT-SWPLis a requirements-based approach for efficiently generating tests for products in a soft-ware product line. This testing approach is used for validation and verification of productlines. It is a novel approach to test product lines using a Colored State Chart (CSC), whichconsiders variability early in the product line development process. More precisely, the vari-ability will be introduced in the main components of the CSC. Accordingly, the variabilityis preserved in test cases, as they are generated from colored test models automatically.During domain engineering, the CSC is derived from the feature model. By coloring theState Chart, the behavior of several product line variants can be modeled simultaneouslyin a single diagram and thus address product line variability early. The CSC representsthe test model, from which test cases using statistical testing are derived.During application engineering, these colored test models are customized for a specificapplication of the product line. At the end of this test process, the test cases are generatedagain using statistical testing, executed and the test results are ready for evaluation. Inxaddition, the CSC will be transformed to a Colored Petri Net (CPN) for verification andsimulation purposes.The main gains of applying the CMBT-SWPL method are early detection of defects inrequirements, such as ambiguities incompleteness and redundancy which is then reflectedin saving the test effort, time, development and maintenance costs

Localizing State-Dependent Faults Using Associated Sequence Mining

In this thesis we developed a new fault localization process to localize faults in object oriented software. The process is built upon the Encapsulation\u27\u27 principle and aims to locate state-dependent discrepancies in the software\u27s behavior. We experimented with the proposed process on 50 seeded faults in 8 subject programs, and were able to locate the faulty class in 100% of the cases when objects with constant states were taken into consideration, while we missed 24% percent of the faults when these objects were not considered. We also developed a customized data mining technique Associated sequence mining\u27\u27 to be used in the localization process; experiments showed that it only provided slight enhancement to the result of the process. The customization provided at least 17% enhancement in the time performance and it is generic enough to be applicable in other domains. In addition to that we have developed an extensive taxonomy for object-oriented software faults based on UML models. We used the taxonomy to make decisions regarding the localization process. It provides an aid for understanding the nature of software faults, and will help enhance the different tasks related to software quality assurance. The main contributions of the thesis were based on preliminary experimentation on the usability of the classification algorithms implemented in WEKA in software fault localization, which resulted in the conclusion that both the fault type and the mechanism implemented in the analysis algorithm were significant to affect the results of the localization

Dependability Assessment of Android OS

In this brave new world of smartphone-dependent society, dependability is a strong requirement and needs to be addressed properly. Assessing the dependability of these mobile system is still an open issue, and companies should have the tools to improve their devices and beat the competition against other vendors. The main objective of this dissertation is to provide the methods to assess the dependability of mobile OS, fundamental for further improvements. Mobile OS are threatened mainly by traditional residual faults (when errors spread across components as failures), aging-related faults (when errors accumulate over time), and misuses by users and applications. This thesis faces these three aspects. First, it presents a qualitative method to define the fault model of a mobile OS, and an exhaustive fault model for Android. I designed and developed AndroFIT, a novel fault injection tool for Android smartphone, and performed an extensive fault injection campaign on three Android devices from different vendors to analyze the impact of component failure on the mobile OS. Second, it presents an experimental methodology to analyze the software aging phenomenon in mobile OS. I performed a software aging analysis campaign on Android devices to identify the impacting factors on performance degradation and resource consumption. Third, it presents the design and implementation of a novel fuzzing tool, namely Chizpurfle, able to automatically test Android vendor customizations by leveraging code coverage information at run-time

Certifying Software Component Performance Specifications

In component-based software engineering, performance prediction approaches support the design of business information systems on the architectural level. They are based on behavior specifications of components. This work presents a round-trip approach for using, assessing, and certifying the accuracy of parameterized, probabilistic, deterministic, and concurrent performance specifications. Its applicability and effectiveness are demonstrated using the CoCoME benchmark

Certifications of Critical Systems – The CECRIS Experience

In recent years, a considerable amount of effort has been devoted, both in industry and academia, to the development, validation and verification of critical systems, i.e. those systems whose malfunctions or failures reach a critical level both in terms of risks to human life as well as having a large economic impact.Certifications of Critical Systems – The CECRIS Experience documents the main insights on Cost Effective Verification and Validation processes that were gained during work in the European Research Project CECRIS (acronym for Certification of Critical Systems). The objective of the research was to tackle the challenges of certification by focusing on those aspects that turn out to be more difficult/important for current and future critical systems industry: the effective use of methodologies, processes and tools.The CECRIS project took a step forward in the growing field of development, verification and validation and certification of critical systems. It focused on the more difficult/important aspects of critical system development, verification and validation and certification process. Starting from both the scientific and industrial state of the art methodologies for system development and the impact of their usage on the verification and validation and certification of critical systems, the project aimed at developing strategies and techniques supported by automatic or semi-automatic tools and methods for these activities, setting guidelines to support engineers during the planning of the verification and validation phases

Black-Box Testfall-Selektion und -Priorisierung für Software-Varianten und -Versionen

Software testing is a fundamental task in software quality assurance. Especially when dealing with several product variants or software versions under test, testing everything for each variant and version is infeasible due to limited testing resources. To cope with increasing complexity both in time (i.e., versions) and space (i.e., variants), new techniques have to be developed to focus on the most important parts for testing. In the past, regression testing techniques such as test case selection and prioritization have emerged to tackle these issues for single-software systems. However, testing of variants and versions is still a challenging task, especially when no source code is available. Most existing regression testing techniques analyze source code to identify important changes to be retested, i.e., they are likely to reveal a failure. To this end, this thesis contributes different techniques for both, variants and versions, to allow more efficient and effective testing in difficult black-box scenarios by identifying important test cases to be re-executed. Four major contributions in software testing are made. (1) We propose a test case prioritization framework for software product lines based on delta-oriented test models to reduce the redundancy in testing between different product variants.(2) We introduce a risk-based testing technique for software product lines. Our semi-automatic test case prioritization approach is able to compute risk values for test model elements and scales with large numbers of product variants. (3) For black-box software versions, we provide a test case selection technique based on genetic algorithms. In particular, seven different black-box selection objectives are defined, thus, we perform a multi-objective test case selection finding Pareto optimal test sets to reduce the testing effort. (4) We propose a novel test case prioritization technique based on supervised machine learning. It is able to imitate decisions made by experts based on different features, such as natural language test case descriptions and black-box meta-data. All of these techniques have been evaluated using the Body Comfort System case study. For testing of software versions, we also assesses our testing techniques using an industrial system. Our evaluation results indicate that our black-box testing approaches for software variants and versions are able to successfully reduce testing effort compared to existing techniques.Testen ist eine fundamentale Aufgabe zur Qualitätssicherung von modernen Softwaresystemen. Mangels limitierter Ressourcen ist das Testen von vielen Produktvarianten oder Versionen sehr herausfordernd und das wiederholte Ausführen aller Testfälle nicht wirtschaftlich. Um mit der Raum- (Varianten) und Zeitdimension (Versionen) in der Entwicklung umzugehen, wurden in der Vergangenheit verschiedene Testansätze entwickelt. Es existieren jedoch nach wie vor große Herausforderungen, welche es zu lösen gilt. Dies ist vor allem der Fall, wenn der Quellcode der getesteten Softwaresysteme unbekannt ist. Das Testen von Black-Box-Systemen erschwert die Identifikation von zu testenden Unterschieden zu vorher getesteten Varianten oder Versionen. In der Literatur finden sich wenige Ansätze, welche versuchen diese Herausforderungen zu lösen. Daher werden in dieser Dissertation neue Ansätze entwickelt und vorgestellt, welche beim Black-Box Testen von Software-Varianten und -Versionen helfen, wichtige Testfälle zur erneuten Ausführung zu identifizieren. Dies erspart die Ausführung von Testfällen, welche weder neues Verhalten testen noch mit hoher Wahrscheinlichkeit neue Fehler zu finden. Insgesamt leistet diese Dissertation die folgenden vier wissenschaftlichen Beiträge: (1) Ein modell-basiertes Framework zur Definition von Testfallpriorisierungsfunktionen für variantenreiche Systeme. Das Framework ermöglicht eine flexible Priorisierung von Testfällen für individuelle Produktvarianten. (2) Einen risiko-basierten Testfallpriorisierungsansatz für variantenreiche Systeme. Das Verfahren ermöglicht eine semi-automatisierte Berechnung von Risikowerten für Elemente von Produktvarianten und skaliert mit großen Produktzahlen. (3) Ein multi-kriterielles Testfallselektionsverfahren für den Regressionstest von Black-Box Software-Versionen. Es werden Black-Box Testkriterien aufgestellt und mittels eines genetischen Algorithmus optimiert um Pareto-optimale Testsets zu berechnen. (4) Ein Testfallpriorisierungsverfahren für Black-Box Regressionstests mit Hilfe von Machine Learning. Der verwendete Algorithmus imitiert Entscheidungen von Testexperten um wichtige Testfälle zu identifizieren. Diese Ansätze wurden alle mit Hilfe von Fallstudien evaluiert. Die resultierenden Ergebnisse zeigen, dass die Ansätze die gewünschten Ziele erreichen und helfen, wichtige Testfälle effektiv zu identifizieren. Insgesamt wird der Testaufwand im Vergleich zu existierenden Techniken verringert

New Fault Detection, Mitigation and Injection Strategies for Current and Forthcoming Challenges of HW Embedded Designs

Tesis por compendio[EN] Relevance of electronics towards safety of common devices has only been growing, as an ever growing stake of the functionality is assigned to them. But of course, this comes along the constant need for higher performances to fulfill such functionality requirements, while keeping power and budget low. In this scenario, industry is struggling to provide a technology which meets all the performance, power and price specifications, at the cost of an increased vulnerability to several types of known faults or the appearance of new ones. To provide a solution for the new and growing faults in the systems, designers have been using traditional techniques from safety-critical applications, which offer in general suboptimal results. In fact, modern embedded architectures offer the possibility of optimizing the dependability properties by enabling the interaction of hardware, firmware and software levels in the process. However, that point is not yet successfully achieved. Advances in every level towards that direction are much needed if flexible, robust, resilient and cost effective fault tolerance is desired. The work presented here focuses on the hardware level, with the background consideration of a potential integration into a holistic approach. The efforts in this thesis have focused several issues: (i) to introduce additional fault models as required for adequate representativity of physical effects blooming in modern manufacturing technologies, (ii) to provide tools and methods to efficiently inject both the proposed models and classical ones, (iii) to analyze the optimum method for assessing the robustness of the systems by using extensive fault injection and later correlation with higher level layers in an effort to cut development time and cost, (iv) to provide new detection methodologies to cope with challenges modeled by proposed fault models, (v) to propose mitigation strategies focused towards tackling such new threat scenarios and (vi) to devise an automated methodology for the deployment of many fault tolerance mechanisms in a systematic robust way. The outcomes of the thesis constitute a suite of tools and methods to help the designer of critical systems in his task to develop robust, validated, and on-time designs tailored to his application.[ES] La relevancia que la electrónica adquiere en la seguridad de los productos ha crecido inexorablemente, puesto que cada vez ésta copa una mayor influencia en la funcionalidad de los mismos. Pero, por supuesto, este hecho viene acompañado de una necesidad constante de mayores prestaciones para cumplir con los requerimientos funcionales, al tiempo que se mantienen los costes y el consumo en unos niveles reducidos. En este escenario, la industria está realizando esfuerzos para proveer una tecnología que cumpla con todas las especificaciones de potencia, consumo y precio, a costa de un incremento en la vulnerabilidad a múltiples tipos de fallos conocidos o la introducción de nuevos. Para ofrecer una solución a los fallos nuevos y crecientes en los sistemas, los diseñadores han recurrido a técnicas tradicionalmente asociadas a sistemas críticos para la seguridad, que ofrecen en general resultados sub-óptimos. De hecho, las arquitecturas empotradas modernas ofrecen la posibilidad de optimizar las propiedades de confiabilidad al habilitar la interacción de los niveles de hardware, firmware y software en el proceso. No obstante, ese punto no está resulto todavía. Se necesitan avances en todos los niveles en la mencionada dirección para poder alcanzar los objetivos de una tolerancia a fallos flexible, robusta, resiliente y a bajo coste. El trabajo presentado aquí se centra en el nivel de hardware, con la consideración de fondo de una potencial integración en una estrategia holística. Los esfuerzos de esta tesis se han centrado en los siguientes aspectos: (i) la introducción de modelos de fallo adicionales requeridos para la representación adecuada de efectos físicos surgentes en las tecnologías de manufactura actuales, (ii) la provisión de herramientas y métodos para la inyección eficiente de los modelos propuestos y de los clásicos, (iii) el análisis del método óptimo para estudiar la robustez de sistemas mediante el uso de inyección de fallos extensiva, y la posterior correlación con capas de más alto nivel en un esfuerzo por recortar el tiempo y coste de desarrollo, (iv) la provisión de nuevos métodos de detección para cubrir los retos planteados por los modelos de fallo propuestos, (v) la propuesta de estrategias de mitigación enfocadas hacia el tratamiento de dichos escenarios de amenaza y (vi) la introducción de una metodología automatizada de despliegue de diversos mecanismos de tolerancia a fallos de forma robusta y sistemática. Los resultados de la presente tesis constituyen un conjunto de herramientas y métodos para ayudar al diseñador de sistemas críticos en su tarea de desarrollo de diseños robustos, validados y en tiempo adaptados a su aplicación.[CA] La rellevància que l'electrònica adquireix en la seguretat dels productes ha crescut inexorablement, puix cada volta més aquesta abasta una major influència en la funcionalitat dels mateixos. Però, per descomptat, aquest fet ve acompanyat d'un constant necessitat de majors prestacions per acomplir els requeriments funcionals, mentre es mantenen els costos i consums en uns nivells reduïts. Donat aquest escenari, la indústria està fent esforços per proveir una tecnologia que complisca amb totes les especificacions de potència, consum i preu, tot a costa d'un increment en la vulnerabilitat a diversos tipus de fallades conegudes, i a la introducció de nous tipus. Per oferir una solució a les noves i creixents fallades als sistemes, els dissenyadors han recorregut a tècniques tradicionalment associades a sistemes crítics per a la seguretat, que en general oferixen resultats sub-òptims. De fet, les arquitectures empotrades modernes oferixen la possibilitat d'optimitzar les propietats de confiabilitat en habilitar la interacció dels nivells de hardware, firmware i software en el procés. Tot i això eixe punt no està resolt encara. Es necessiten avanços a tots els nivells en l'esmentada direcció per poder assolir els objectius d'una tolerància a fallades flexible, robusta, resilient i a baix cost. El treball ací presentat se centra en el nivell de hardware, amb la consideració de fons d'una potencial integració en una estratègia holística. Els esforços d'esta tesi s'han centrat en els següents aspectes: (i) la introducció de models de fallada addicionals requerits per a la representació adequada d'efectes físics que apareixen en les tecnologies de fabricació actuals, (ii) la provisió de ferramentes i mètodes per a la injecció eficient del models proposats i dels clàssics, (iii) l'anàlisi del mètode òptim per estudiar la robustesa de sistemes mitjançant l'ús d'injecció de fallades extensiva, i la posterior correlació amb capes de més alt nivell en un esforç per retallar el temps i cost de desenvolupament, (iv) la provisió de nous mètodes de detecció per cobrir els reptes plantejats pels models de fallades proposats, (v) la proposta d'estratègies de mitigació enfocades cap al tractament dels esmentats escenaris d'amenaça i (vi) la introducció d'una metodologia automatitzada de desplegament de diversos mecanismes de tolerància a fallades de forma robusta i sistemàtica. Els resultats de la present tesi constitueixen un conjunt de ferramentes i mètodes per ajudar el dissenyador de sistemes crítics en la seua tasca de desenvolupament de dissenys robustos, validats i a temps adaptats a la seua aplicació.Espinosa García, J. (2016). New Fault Detection, Mitigation and Injection Strategies for Current and Forthcoming Challenges of HW Embedded Designs [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/73146TESISCompendi

An aesthetic for sustainable interactions in product-service systems?

Copyright @ 2012 Greenleaf PublishingEco-efficient Product-Service System (PSS) innovations represent a promising approach to sustainability. However the application of this concept is still very limited because its implementation and diffusion is hindered by several barriers (cultural, corporate and regulative ones). The paper investigates the barriers that affect the attractiveness and acceptation of eco-efficient PSS alternatives, and opens the debate on the aesthetic of eco-efficient PSS, and the way in which aesthetic could enhance some specific inner qualities of this kinds of innovations. Integrating insights from semiotics, the paper outlines some first research hypothesis on how the aesthetic elements of an eco-efficient PSS could facilitate user attraction, acceptation and satisfaction

Certifications of Critical Systems – The CECRIS Experience

In recent years, a considerable amount of effort has been devoted, both in industry and academia, to the development, validation and verification of critical systems, i.e. those systems whose malfunctions or failures reach a critical level both in terms of risks to human life as well as having a large economic impact.Certifications of Critical Systems – The CECRIS Experience documents the main insights on Cost Effective Verification and Validation processes that were gained during work in the European Research Project CECRIS (acronym for Certification of Critical Systems). The objective of the research was to tackle the challenges of certification by focusing on those aspects that turn out to be more difficult/important for current and future critical systems industry: the effective use of methodologies, processes and tools.The CECRIS project took a step forward in the growing field of development, verification and validation and certification of critical systems. It focused on the more difficult/important aspects of critical system development, verification and validation and certification process. Starting from both the scientific and industrial state of the art methodologies for system development and the impact of their usage on the verification and validation and certification of critical systems, the project aimed at developing strategies and techniques supported by automatic or semi-automatic tools and methods for these activities, setting guidelines to support engineers during the planning of the verification and validation phases

Techniques for Efficient and Effective Mobile Testing

The booming mobile app market attracts a large number of developers. As a result, the competition is extremely tough. This fierce competition leads to high standards required for mobile apps, which mandates efficient and effective testing. Efficient testing requires little effort to use, while effective testing checks that the app under test behaves as expected. Manual testing is highly effective, but it is costly. Automatic testing should come to the rescue, but current automatic methods are either ineffective or inefficient. Methods using implicit specifications – for instance, “an app should not crash” for catching fail-stop errors – are ineffective because they cannot find semantic problems. Methods using explicit specifications such as test scripts are inefficient because they require huge developer effort to create and maintain specifications. In this thesis, we present our two approaches for solving these challenges. We first built the AppDoctor system which efficiently tests mobile apps. It quickly explores an app then slowly but accurately verifies the potential problems to identify bugs without introducing false positives. It uses dependencies discovered between actions to simplify its reports. Our second approach, implemented in the AppFlow system, leverages the ample opportunity of reusing test cases between apps to gain efficiency without losing effectiveness. It allows common UI elements to be used in test scripts then recognizes these UI elements in real apps using a machine learning approach. The system also allows tests to be specified in reusable pieces, and provides a system to synthesize complete test cases from these reusable pieces. It enables robust tests to be created and reused across apps in the same category. The combination of these two approaches enables a developer to quickly test an app on a great number of combinations of actions for fail-stop problems, and effortlessly and efficiently test the app on most common scenarios for semantic problems. This combination covers most of her test requirements and greatly reduces her burden in testing the app