Scenarios for system requirements traceability and testing

Scenarios in current design methodologies, provide a natural way for the users to identify the inputs and outputs of the system revolving around a particular interaction process. A scenario typically consists of a sequence of steps which captures a particular run of the system and satisfies some aspect of the requirements. In this work we add additional structure to the scenarios used in the Prometheus agent development methodology. This additional structure then facilitates both traceability and automated testing. We describe our process for mapping the scenarios and their steps to the initial detailed design, where we then maintain the traceability as the design develops. The structured action lists that we define for both scenarios and their variations provides the basis for facilitating automated testing of system behavior. We describe how we use the newly defined structure within the scenarios to facilitate testing, describing how we automate test case generation, execution and analysis

Early detection of design faults relative to requirement specifications in agent-based models

Agent systems are used for a wide range of applications, and techniques to detect and avoid defects in such systems are valuable. In particular, it is desirable to detect issues as early as possible in the software development lifecycle. We describe a technique for checking the plan structures of a BDI agent design against the requirements models, specified in terms of scenarios and goals. This approach is applicable at design time, not requiring source code. A lightweight evaluation demonstrates that a range of defects can be found using this technique

A systematic approach for detecting faults in agent designs

This thesis proposes a mechanism, including automated tool support, for early-phase defect detection by comparing the plan structures of a belief-desire-intention (BDI) agent design against the following: (1) requirement models, specified in terms of scenarios and goals; and (2) agent communication models. The intuition of our approach is to extract sets of possible behaviour runs from the agent-behaviour models and to verify whether these runs conform to the specifications of the system-to-be. The proposed approach in this thesis is applicable at design time and does not require source code. Our approach is based on the Prometheus agent-design methodology but is applicable to other methodologies that support the same notions. We evaluate the proposed verification framework on designs, ranging from student projects to case studies of industry-level projects. Our evaluation demonstrates that even a simple specification developed by relatively experienced developers is prone to defects, and our approach is successful in uncovering most of these defects. In addition, we conduct a scalability analysis of our methods, and the outcomes reveal that our approach can scale when designs grow in size