Evolutionary Search Techniques with Strong Heuristics for Multi-Objective Feature Selection in Software Product Lines

Software design is a process of trading off competing objectives. If the user objective space is rich, then we should use optimizers that can fully exploit that richness. For example, this study configures software product lines (expressed as feature models) using various search-based software engineering methods. Our main result is that as we increase the number of optimization objectives, the methods in widespread use (e.g. NSGA-II, SPEA2) perform much worse than IBEA (Indicator-Based Evolutionary Algorithm). IBEA works best since it makes most use of user preference knowledge. Hence it does better on the standard measures (hypervolume and spread) but it also generates far more products with 0 violations of domain constraints. We also present significant improvements to IBEA\u27s performance by employing three strong heuristic techniques that we call PUSH, PULL, and seeding. The PUSH technique forces the evolutionary search to respect certain rules and dependencies defined by the feature models, while the PULL technique gives higher weight to constraint satisfaction as an optimization objective and thus achieves a higher percentage of fully-compliant configurations within shorter runtimes. The seeding technique helps in guiding very large feature models to correct configurations very early in the optimization process. Our conclusion is that the methods we apply in search-based software engineering need to be carefully chosen, particularly when studying complex decision spaces with many optimization objectives. Also, we conclude that search methods must be customized to fit the problem at hand. Specifically, the evolutionary search must respect domain constraints

Single and Multi Objective Genetic Programming for Software Development Effort Estimation

The idea of exploiting Genetic Programming (GP) to estimate software development effort is based on the observation that the effort estimation problem can be formulated as an optimization problem. Indeed, among the possible models, we have to identify the one providing the most accurate estimates. To this end a suitable measure to evaluate and compare different models is needed. However, in the context of effort estimation there does not exist a unique measure that allows us to compare different models but several different criteria (e.g., MMRE, Pred(25), MdMRE) have been proposed. Aiming at getting an insight on the effects of using different measures as fitness function, in this paper we analyzed the performance of GP using each of the five most used evaluation criteria. Moreover, we designed a Multi-Objective Genetic Programming (MOGP) based on Pareto optimality to simultaneously optimize the five evaluation measures and analyzed whether MOGP is able to build estimation models more accurate than those obtained using GP. The results of the empirical analysis, carried out using three publicly available datasets, showed that the choice of the fitness function significantly affects the estimation accuracy of the models built with GP and the use of some fitness functions allowed GP to get estimation accuracy comparable with the ones provided by MOGP

Search-based approaches for software development effort estimation

2011 - 2012Effort estimation is a critical activity for planning and monitoring software project development and for delivering the product on time and within budget. Significant over or under-estimates expose a software project to several risks. As a matter of fact under-estimates could lead to addition of manpower to a late software project, making the project later (Brooks’s Law), or to the cancellation of activities, such as documentation and testing, negatively impacting on software quality and maintainability. Thus, the competitiveness of a software company heavily depends on the ability of its project managers to accurately predict in advance the effort required to develop software system. However, several challenges exists in making accurate estimates, e.g., the estimation is needed early in the software lifecycle, when few information about the project are available, or several factors can impact on project effort and these factor are usually specific for different production contexts. Several techniques have been proposed in the literature to support project manager in estimating software project development effort. In the last years the use of Search-Based (SB) approaches has been suggested to be employed as an effort estimation technique. These approaches include a variety of meta-heuristics, such as local search techniques (e.g., Hill Climbing, Tabu Search, Simulated Annealing) or Evolutionary Algorithms (e.g., Genetic Algorithms, Genetic Programming). The idea underlying the use of such techniques is based on the reformulation of software engineering problems as search or optimization problems whose goal is to find the most appropriate solutions which conform to some adequacy criteria (i.e., problem goals). In particular, the use of SB approaches in the context of effort estimation is twofold: they can be exploited to build effort estimation models or to enhance the use of existing effort estimation techniques. The usage reported in the literature of SB approaches for effort estimation have provided promising results that encourage further investigations. However, they can be considered preliminary studies. As a matter of fact, the capabilities of these approaches were not fully exploited, either the employed empirical analyses did not consider the more recent recommendations on how to carry out this kind of empirical assessment in the effort estimation and in the SBSE contexts. The main aim of the PhD dissertation is to provide an insight on the use of SB techniques for the effort estimation trying to highlight strengths and weaknesses of these approaches for both the uses above mentioned. [edited by Author]XI n.s

Explanatory and Causality Analysis in Software Engineering

Software fault proneness and software development efforts are two key areas of software engineering. Improving them will significantly reduce the cost and promote good planning and practice in developing and managing software projects. Traditionally, studies of software fault proneness and software development efforts were focused on analysis and prediction, which can help to answer questions like `when’ and `where’. The focus of this dissertation is on explanatory and causality studies that address questions like `why’ and `how’. First, we applied a case-control study to explain software fault proneness. We found that Bugfixes (Prerelease bugs), Developers, Code Churn, and Age of a file are the main contributors to the Postrelease bugs in some of the open-source projects. In terms of the interactions, we found that Bugfixes and Developers reduced the risk of post release software faults. The explanatory models were tested for prediction and their performance was either comparable or better than the top-performing classifiers used in related studies. Our results indicate that software project practitioners should pay more attention to the prerelease bug fixing process and the number of Developers assigned, as well as their interaction. Also, they need to pay more attention to the new files (less than one year old) which contributed significantly more to Postrelease bugs more than old files. Second, we built a model that explains and predicts multiple levels of software development effort and measured the effects of several metrics and their interactions using categorical regression models. The final models for the three data sets used were statistically fit, and performance was comparable to related studies. We found that project size, duration, the existence of any type of faults, the use of first- or second generation of programming languages, and team size significantly increased the software development effort. On the other side, the interactions between duration and defective project, and between duration and team size reduced the software development effort. These results suggest that software practitioners should pay extra attention to the time of the project and the team size assigned for every task because when they increased from a low to a higher level, they significantly increased the software development effort. Third, a structural equation modeling method was applied for causality analysis of software fault proneness. The method combined statistical and regression analysis to find the direct and indirect causes for software faults using partial least square path modeling method. We found direct and indirect paths from measurement models that led to software postrelease bugs. Specifically, the highest direct effect came from the change request, while changing the code had a minor impact on software faults. The highest impact of the code change resulted from the change requests (either for bug fixing or refactoring). Interestingly, the indirect impact from code characteristics to software fault proneness was higher than the direct impact. We found a similar level of direct and indirect impact from code characteristics to code change

Applying Genetic Algorithms for Software Design and Project Planning

Today's software systems are growing in size and complexity. This means not only increased complexity in developing software systems, but also increase in the budget and completion time. This trend will lead to a situation where traditional manual software engineering practices are not sufficient to develop and evolve software systems in an economic and timely manner. Automated support can aid software engineers in reducing the time-to-market and improving the quality of the software. This thesis work explores the application of genetic algorithms for automated software architecture design and project planning.Software architecture design and project planning are non-trivial and challenging tasks. This thesis applies genetic algorithms to introduce automation into these tasks. The proposed genetic algorithm exploits reusable solutions, such as design patterns, architecture styles and application specific solutions for transforming a given initial rudimentary model into detailed design. The architectures are evaluated using multiple quality attributes, such as modifiability, efficiency and complexity. The fitness function encompasses the knowledge required for evaluating the architectures according to multiple quality attributes. The output from the genetic algorithm is an architecture proposal optimized with respect to multiple quality attributes.A genetic algorithm has also been devised for assigning work across teams located in distributed sites. The genetic algorithm takes information about the target system and the development organization as input and produces a set of work distribution and schedule plans optimized with respect to cost and duration objectives. The fitness function considers the differences in teams and barriers created by global dispersion into account in evaluating the work assignment. In addition, the genetic algorithm also takes solutions that ease or hamper distributed development into account in allocating the work. The genetic algorithm has been further extended with Pareto optimality to find a set of suitable work distribution proposals in a tradeoff between project cost and duration. In the experiments, an electronic home control system was developed by a set of different organizations structures. The results demonstrate that the proposed genetic algorithm can create reasonable work distribution proposals that conform to the general assumptions about the nature of cost and project completion time, i.e., cost of the project can be reduced at the expense of project completion time and vice-versa.In addition, variations have been made to the genetic algorithm approach to software architecture design. To accelerate the genetic algorithm towards multi-objective solutions, a quality farms approach has been developed. The approach uses the idea of cross breeding, where different individuals that are good with respect to one quality objective are combined for producing software architecture proposals that are good in multiple objectives. Also, to explore the suitability of other methods for software architecture synthesis, a constraint satisfaction approach has been developed. The approach models the software architecture design problem as a constraint satisfaction and optimization problem and solves it using constraint satisfaction techniques. This approach can provide rationale about why certain decisions are chosen in the proposed architecture proposals.Tool support for genetic algorithm-based architecture design and work planning approaches has been proposed. It facilitates an end user to give input, view and analyze the results of the developed genetic algorithm based approaches. The tool also provides support for semi-automated architecture design, where a human architect can guide the genetic algorithm towards optimal solutions. An empirical study has also been performed. It suggests that the quality of the proposals produced through semiautomated architecture design is roughly at the level of senior software engineering students. Furthermore, the project manager can interact with the tool and perform whatif analysis for choosing the suitable work distribution for the project at hand

Evolving Fault Tolerant Robotic Controllers

Fault tolerant control and evolutionary algorithms are two different research areas. However with the development of artificial intelligence, evolutionary algorithms have demonstrated competitive performance compared to traditional approaches for the optimisation task. For this reason, the combination of fault tolerant control and evolutionary algorithms has become a new research topic with the evolving of controllers so as to achieve different fault tolerant control schemes. However most of the controller evolution tasks are based on the optimisation of controller parameters so as to achieve the fault tolerant control, so structure optimisation based evolutionary algorithm approaches have not been investigated as the same level as parameter optimisation approaches. For this reason, this thesis investigates whether structure optimisation based evolutionary algorithm approaches could be implemented into a robot sensor fault tolerant control scheme based on the phototaxis task in addition to just parameter optimisation, and explores whether controller structure optimisation could demonstrate potential benefit in a greater degree than just controller parameter optimisation. This thesis presents a new multi-objective optimisation algorithm in the structure optimisation level called Multi-objective Cartesian Genetic Programming, which is created based on Cartesian Genetic Programming and Non-dominated Sorting Genetic Algorithm 2, in terms of NeuroEvolution based robotic controller optimisation. In order to solve two main problems during the algorithm development, this thesis investigates the benefit of genetic redundancy as well as preserving neutral genetic drift in order to solve the random neighbour pick problem during crowding fill for survival selection and investigates how hyper-volume indicator is employed to measure the multi-objective optimisation algorithm performance in order to assess the convergence for Multi-objective Cartesian Genetic Programming. Furthermore, this thesis compares Multi-objective Cartesian Genetic Programming with Non-dominated Sorting Genetic Algorithm 2 for their evolution performance and investigates how Multi-objective Cartesian Genetic Programming could be performing for a more difficult fault tolerant control scenario besides the basic one, which further demonstrates the benefit of utilising structure optimisation based evolutionary algorithm approach for robotic fault tolerant control