Modeling and Testing of Aspect-Oriented Systems

Aspect-Oriented programming modularizes crosscutting concerns into Aspects, which are automatically weaved to the specified points of a program. Although Aspect-Oriented programming improves program maintainability and the encapsulation of crosscutting concerns, it also breaks some traditional programming rules due to the weaving mechanism. Therefore, a new software testing approach has to be developed to rigorously test Aspect-Oriented programs. In this thesis, we introduce the concept of Aspect modeling and Aspect testing and then proceed to our investigation of a model-based incremental approach for testing Aspect-Oriented programs. First, a state machine model in UML is created for each Aspect and each base class, which is the class to be weaved with the aspect. Then each individual aspect or base class is tested using the test cases generated from state machine models. A combined state machine model is established by weaving the aspect model into the base class model. Finally, we perform a test on the woven program using test cases generated from the combined state machine model. Because the number of scenarios for weaving aspects and base classes could be very large, it may require a huge number of test cases to effectively test the program. To speed up the process, we propose a prioritizing strategy for selecting test cases in order to find errors sooner since different test cases have different capacity for tracking errors. We demonstrate that the test cases generated from the state machine model have to satisfy the adequacy of the transition coverage, the round-trip coverage, and the state coverage in the state machine model. Furthermore, the prioritizing strategy is developed based on the number of changes brought by weaving of an aspect and its base classes. The test case including more changes will have a higher priority. The effectiveness of the investigated strategy is evaluated through the case study and the mutation testing. The result of case study shows that the model-based incremental approach integrated with prioritizing test case selection provides an effective tool for testing large-scale Aspect-Oriented systems.  M.S

Scalable Model-based Robustness Testing: Novel Methodologies and Industrial Application

Embedded systems, as for example communication and control systems, are being increasingly used in our daily lives and hence require thorough and systematic testing before their actual use. Many of these systems interact with their environment and, therefore, their functionality is largely dependent on this environment whose behavior can be unpredictable. Robustness testing aims at testing the behavior of a system in the presence of faulty situations in its operating environment (e.g., sensors and actuators). In such situations, the system should gracefully degrade its performance instead of abruptly stopping execution. To systematically perform robustness testing, one option is to resort to Model-Based Robustness Testing (MBRT), which is a systematic, rigorous, and automated way of conducting robustness testing. However, to successfully apply MBRT in industrial contexts, new technologies need to be developed to scale to the complexity of real industrial systems. This thesis presents a solution for MBRT on industrial systems, including scalable robustness modeling and executable test case generation. One important contribution of this thesis is a scalable RobUstness Modeling Methodology (RUMM), which is achieved using Aspect-Oriented Modeling (AOM). It is a complete, automated, and practical methodology that covers all features of state machines and aspect concepts necessary for MBRT. Such methodology, relying on a standard (Unified Modeling Language or UML) and using the target notation as the basis to model the aspects themselves, is expected to make the practical adoption of robustness modeling easier in industrial contexts. The applicability of the methodology is demonstrated using an industrial case study. Results showed that the approach significantly reduced modeling effort (98% on average), improved separation of concerns, and eased model evolution. The approach is further empirically evaluated using two controlled experiments involving human subjects and results showed that the proposed methodology significantly improves the readability of models as compared to modeling using standard UML notations. Another important contribution of this thesis is an efficient approach for solving constraints (written in Objects Constraint Language (OCL)) on the operating environment of a system, which is mandatory for emulating faulty situation in the environment for the purpose of MBRT. A set of novel heuristics is devised for various OCL constructs, which are required for the application of search algorithms. The heuristics have been empirically evaluated on an industrial case study for robustness testing and the results showed to be very promising and significantly better than the existing works in the literature on OCL constraint solvers. A final contribution of the thesis is robustness test case generation from the models developed using RUMM. Test case generation also includes scripts generation for environment emulation, which is mandatory for automated robustness testing again using an industrial case study. In preliminary experiments, the execution of test cases found one critical, robustness fault in a deployed industrial system

TOOL-ASSISTED VALIDATION AND VERIFICATION TECHNIQUES FOR STATE-BASED FORMAL METHODS

To tackle the growing complexity of developing modern software systems that usually have embedded and distributed nature, and more and more involve safety critical aspects, formal methods (FMs) have been affirmed as an efficient approach to ensure the quality and correctness of the design, that permits to discover errors yet at the early stages of the system development. Among the several FMs available, some of them can be described as state-based, since they describe systems by using the notions of state and transitions between states. State-based FMs are sometimes preferred since they produce specifications that are more intuitive, being the notions of state and transition close to the notions of program state and program execution that are familiar to any developer. Moreover, state-based FMs are usually executable and permit to be simulated, so having an abstraction of the execution of the system under development. The aim of the thesis is to provide tool-assisted techniques that help the adoption of state-based FMs. In particular we address four main goals: 1) identifying a process for the development of an integrated framework around a formal method. The adoption of a formal method is often prevented by the lack of tools to support the user in the different development activities, as model editing, validation, verification, etc. Moreover, also when tools are available, they have usually been developed to target only one aspect of the system development process. So, having a well-engineered process that helps in the development of concrete notations and tools for a FM can make FMs of practical application. 2) promoting the integration of different FMs. Indeed, having only one formal notation, for doing different formal activities during the development of the system, is preferable than having a different notation for each formal activity. Moreover such notation should be high-level: working with high level notations is definitely easier than working with low-level ones, and the produced specifications are usually more readable. This goal can be seen as a sub-goal of the first goal; indeed, in a framework around a formal method, it should also be possible to integrate other formal methods that better address some particular formal activities. 3) helping the user in writing correct specifications. The basic assumption of any formal technique is that the specification, representing the desired properties of the system or the model of the system, is correct. However, in case the specification is not correct, all the verification activities based on the specification produce results that are meaningless. So, validation techniques should assure that the specification reflects the intended requirements; besides traditional simulation (user-guided or scenario-based), also model review techniques, checking for common quality attributes that any specification should have, are a viable solution. 4) reducing the distance between the formal specification and the actual implementation of the system. Several FMs work on a formal description of the system which is assumed to reflect the actual implementation; however, in practice, the formal specification and the actual implementation could be not conformant. A solution is to obtain the implementation, through refinements steps, from the formal specification, and proving that the refinements steps are correct. A different viable solution is to link the implementation with its formal specification and check, during the program execution, if they are conformant

Towards Automatic Repair of XACML Policies

In a complex information system, controlling the access to resources is challenging. As a new generation of access control techniques, Attribute-Based Access Control (ABAC) can provide more flexible and fine-grained access control than Role-Based-Access Control (RBAC). XACML (eXtensible Access Control Markup Language) is an industrial standard for specifying ABAC policies. XACML policies tend to be complex because of the great variety of attribute types for fine-grained access control. This means that XACML policies are prone to errors and difficult to debug. This paper presents a first attempt at automating the debugging process of XACML policies. Two techniques are used for this purpose: fault localization and mutation-based policy repair. Fault localization produces an ordered list of suspicious policy elements by correlating the test results and the test coverage information. Mutation-based policy repair searches for potential fixes by mutating suspicious policy elements with predefined mutation operators. Empirical studies show that the proposed approach is able to repair various faulty XACML policies with one or two seeded faults. Among the scoring methods for fault localization that are studied in the experiment, Naish2 and CBI-Inc are the most efficient

Higher Order Mutation Testing

Mutation testing is a fault-based software testing technique that has been studied widely for over three decades. To date, work in this field has focused largely on first order mutants because it is believed that higher order mutation testing is too computationally expensive to be practical. This thesis argues that some higher order mutants are potentially better able to simulate real world faults and to reveal insights into programming bugs than the restricted class of first order mutants. This thesis proposes a higher order mutation testing paradigm which combines valuable higher order mutants and non-trivial first order mutants together for mutation testing. To overcome the exponential increase in the number of higher order mutants a search process that seeks fit mutants (both first and higher order) from the space of all possible mutants is proposed. A fault-based higher order mutant classification scheme is introduced. Based on different types of fault interactions, this approach classifies higher order mutants into four categories: expected, worsening, fault masking and fault shifting. A search-based approach is then proposed for locating subsuming and strongly subsuming higher order mutants. These mutants are a subset of fault mask and fault shift classes of higher order mutants that are more difficult to kill than their constituent first order mutants. Finally, a hybrid test data generation approach is introduced, which combines the dynamic symbolic execution and search based software testing approaches to generate strongly adequate test data to kill first and higher order mutants