The Value of Exact Analysis in Requirements Selection

Uncertainty is characterised by incomplete understanding. It is inevitable in the early phase of requirements engineering, and can lead to unsound requirement decisions. Inappropriate requirement choices may result in products that fail to satisfy stakeholders' needs, and might cause loss of revenue. To overcome uncertainty, requirements engineering decision support needs uncertainty management. In this research, we develop a decision support framework METRO for the Next Release Problem (NRP) to manage algorithmic uncertainty and requirements uncertainty. An exact NRP solver (NSGDP) lies at the heart of METRO. NSGDP's exactness eliminates interference caused by approximate existing NRP solvers. We apply NSGDP to three NRP instances, derived from a real world NRP instance, RALIC, and compare with NSGA-II, a widely-used approximate (inexact) technique. We find the randomness of NSGA-II results in decision makers missing up to 99.95% of the optimal solutions and obtaining up to 36:48% inexact requirement selection decisions. The chance of getting an inexact decision using existing approximate approaches is negatively correlated with the implementation cost of a requirement (Spearman ρ up to -0.72). Compared to the inexact existing approach, NSGDP saves 15.21% lost revenue, on average, for the RALIC dataset

Darwinian Data Structure Selection

Data structure selection and tuning is laborious but can vastly improve an application's performance and memory footprint. Some data structures share a common interface and enjoy multiple implementations. We call them Darwinian Data Structures (DDS), since we can subject their implementations to survival of the fittest. We introduce ARTEMIS a multi-objective, cloud-based search-based optimisation framework that automatically finds optimal, tuned DDS modulo a test suite, then changes an application to use that DDS. ARTEMIS achieves substantial performance improvements for \emph{every} project in $5$ Java projects from DaCapo benchmark, $8$ popular projects and $30$ uniformly sampled projects from GitHub. For execution time, CPU usage, and memory consumption, ARTEMIS finds at least one solution that improves \emph{all} measures for $86\%$ ($37/43$) of the projects. The median improvement across the best solutions is $4.8\%$, $10.1\%$, $5.1\%$ for runtime, memory and CPU usage. These aggregate results understate ARTEMIS's potential impact. Some of the benchmarks it improves are libraries or utility functions. Two examples are gson, a ubiquitous Java serialization framework, and xalan, Apache's XML transformation tool. ARTEMIS improves gson by $16.5$\%, $1\%$ and $2.2\%$ for memory, runtime, and CPU; ARTEMIS improves xalan's memory consumption by $23.5$\%. \emph{Every} client of these projects will benefit from these performance improvements.Comment: 11 page

Dependency-Aware Software Requirements Selection using Fuzzy Graphs and Integer Programming

Software requirements selection aims to find an optimal subset of the requirements with the highest value while respecting the project constraints. But the value of a requirement may depend on the presence or absence of other requirements in the optimal subset. Such Value Dependencies, however, are imprecise and hard to capture. In this paper, we propose a method based on integer programming and fuzzy graphs to account for value dependencies and their imprecision in software requirements selection. The proposed method, referred to as Dependency-Aware Software Requirements Selection (DARS), is comprised of three components: (i) an automated technique for the identification of value dependencies from user preferences, (ii) a modeling technique based on fuzzy graphs that allows for capturing the imprecision of value dependencies, and (iii) an Integer Linear Programming (ILP) model that takes into account user preferences and value dependencies identified from those preferences to reduce the risk of value loss in software projects. Our work is verified by studying a real-world software project. The results show that our proposed method reduces the value loss in software projects and is scalable to large requirement sets.Comment: arXiv admin note: text overlap with arXiv:2003.0480

Modelling and Analysing Software Requirements and Architecture Decisions under Uncertainty

Early requirements engineering and software architectural decisions are critical to the success of software development projects. However, such decisions are confronted with complexities resulting from uncertainty about the possible impacts of decision choices on objectives; conflicting stakeholder objectives; and a huge space of alternative designs. Quantitative decision modelling is a promising approach to tackling the increasing complexity of requirements and architectural decisions. It allows one to use quantitative techniques, such as stochastic simulation and multi-objective optimisation, to model and analyse the impact of alternative decisions on stakeholders' objectives. Existing requirements and architecture methods that use quantitative decision models are limited by the difficulty of elaborating quantitative decision models and/or lack of integrated tool support for automated decision analysis under uncertainty. This thesis addresses these problems by presenting a novel modelling language and automated decision analysis technique, implemented in a tool called RADAR, intended to facilitate requirements and architecture decisions under uncertainty. RADAR's modelling language has relations to quantitative AND/OR goal models used in requirements engineering and feature models used in software product lines. The language enables modelling requirements and architectural decision problems characterised by (i) single option selection similar to mutually exclusive option selection (XOR-nodes) of feature diagrams; (ii) multiple options selection similar to non-mutually exclusive options selections (OR-nodes) of feature diagrams; and (iii) constraints dependency relationships, e.g., excludes, requires and coupling, between options of decisions. RADAR's analysis technique uses multi-objective simulation optimisation technique in evaluating and shortlisting alternatives that produces the best trade-off between stakeholders' objectives. Additionally, the analysis technique employs information value analysis to estimate the financial value of reducing uncertainty before making a decision. We evaluate RADAR's applicability, usefulness and scalability on a set of real-world systems from different application domains and characterised by design space size between 6 and 2E50. Our evaluation results show that RADAR's modelling language and analysis technique is applicable on a range of real-world requirements and architecture decision problems, and that in few seconds, RADAR can analyse decision problems characterised by large design space using highly performant optimisation method through the use of evolutionary search-based optimisation algorithms