Software evolvability - empirically discovered evolvability issues and human evaluations

Evolution of a software system can take decades and can cost up to several billion Euros. Software evolvability refers to how easily software is understood, modified, adapted, corrected, and developed. It has been estimated that software evolvability can explain 25% to 38% of the costs of software evolution. Prior research has presented software evolvability criteria and quantified the criteria utilizing source code metrics. However, the empirical observations of software evolvability issues and human evaluations of them have largely been ignored. This dissertation empirically studies human evaluations and observations of software evolvability issues. This work utilizes both qualitative and quantitative research methods. Empirical data was collected from controlled experiments with student subjects, and by observing issues that were discovered in real industrial settings. This dissertation presents a new classification for software evolvability issues. The information provided by the classification is extended by the detailed analysis of evolvability issues that have been discovered in code reviews and their distributions to different issue types. Furthermore, this work studies human evaluations of software evolvability; more specifically, it focuses on the interrater agreement of the evaluations, the affect of demographics, the evolvability issues that humans find to be most significant, as well as the relationship between human evaluation and source code metrics based evaluations. The results show that code review that is performed after light functional testing reveals three times as many evolvability issues as functional defects. We also discovered a new evolvability issue called "solution approach", which indicates a need to rethink the current solution rather than reorganize it. For solution approach issues, we are not aware of any research that presents or discusses such issues in the software engineering domain. We found weak evidence that software evolvability evaluations are more affected by a person's role in the organization and the relationship (authorship) to the code than by education and work experience. Comparison of code metrics and human evaluations revealed that metrics cannot detect all human found evolvability issues

Neutral Networks of Real-World Programs and their Application to Automated Software Evolution

The existing software development ecosystem is the product of evolutionary forces, and consequently real-world software is amenable to improvement through automated evolutionary techniques. This dissertation presents empirical evidence that software is inherently robust to small randomized program transformations, or \u27mutations. Simple and general mutation operations are demonstrated that can be applied to software source code, compiled assembler code, or directly to binary executables. These mutations often generate variants of working programs that differ significantly from the original, yet remain fully functional. Applying successive mutations to the same software program uncovers large \u27neutral networks\u27 of fully functional variants of real-world software projects. These properties of \u27mutational robustness\u27 and the corresponding \u27neutral networks\u27 have been studied extensively in biology and are believed to be related to the capacity for unsupervised evolution and adaptation. As in biological systems, mutational robustness and neutral networks in software systems enable automated evolution. The dissertation presents several applications that leverage software neutral networks to automate common software development and maintenance tasks. Neutral networks are explored to generate diverse implementations of software for improving runtime security and for proactively repairing latent bugs. Next, a technique is introduced for automatically repairing bugs in the assembler and executables compiled from off-the-shelf software. As demonstration, a proprietary executable is manipulated to patch security vulnerabilities without access to source code or any aid from the software vendor. Finally, software neutral networks are leveraged to optimize complex nonfunctional runtime properties. This optimization technique is used to reduce the energy consumption of the popular PARSEC benchmark applications by 20% as compared to the best available public domain compiler optimizations. The applications presented herein apply evolutionary computation techniques to existing software using common software engineering tools. By enabling evolutionary techniques within the existing software development toolchain, this work is more likely to be of practical benefit to the developers and maintainers of real-world software systems

Towards an Information Theoretic Framework for Evolutionary Learning

The vital essence of evolutionary learning consists of information flows between the environment and the entities differentially surviving and reproducing therein. Gain or loss of information in individuals and populations due to evolutionary steps should be considered in evolutionary algorithm theory and practice. Information theory has rarely been applied to evolutionary computation - a lacuna that this dissertation addresses, with an emphasis on objectively and explicitly evaluating the ensemble models implicit in evolutionary learning. Information theoretic functionals can provide objective, justifiable, general, computable, commensurate measures of fitness and diversity. We identify information transmission channels implicit in evolutionary learning. We define information distance metrics and indices for ensembles. We extend Price\u27s Theorem to non-random mating, give it an effective fitness interpretation and decompose it to show the key factors influencing heritability and evolvability. We argue that heritability and evolvability of our information theoretic indicators are high. We illustrate use of our indices for reproductive and survival selection. We develop algorithms to estimate information theoretic quantities on mixed continuous and discrete data via the empirical copula and information dimension. We extend statistical resampling. We present experimental and real world application results: chaotic time series prediction; parity; complex continuous functions; industrial process control; and small sample social science data. We formalize conjectures regarding evolutionary learning and information geometry

A pragmatic approach for identifying and managing design science research goals and evaluation criteria

International audienceThe effectiveness of a Design Science Research (DSR) project is judged both by the fitness of the designed artifact as a solution in the application environment and by the level of new research contributions. An important and understudied challenge is how to translate DSR project research goals into discrete and measurable evaluation criteria for use in the DSR processes. This position paper proposes an inclusive approach for articulating DSR goals and then identifying project evaluation criteria for these goals. The goals are organized hierarchically as utilitarian goals, safety goals, interaction and communication goals, cognitive and aesthetic goals, innovation goals, and evolution goals. Goals in a DSR project are identified pragmatically by considering the components of the context coupled with the hierarchy of goals. Based on the identified goals, the associated evaluation criteria are determined and organized along the same hierarchy. These criteria measure the ability of the artifact to meet its goals in itscontext (immediate fitness). Moreover, our approach also supports the innovation and research contributions of the project. The apex of the goal hierarchy addresses the identification of criteria measuring the fitness for evolution of the designed artifact, to accommodate for changes in goals or context

Essays on the co-evolution between strategies and technologies

Sensitivity, Innovation Attitudes, and Perseverance as the Strategic Foundations of Exaptation. Functions, Modular Architectures, and Technological Evolvability. A Generalized NK-Framework to Study the Co-Evolution Between Industry Dynamics and Artefact’s Architecture. Local Technological Evolution & University-Industry Collaboration

Sustainability evaluation of software architectures

Long-living software systems are sustainable if they can be cost-efficiently maintained and evolved over their entire life-cycle. The quality of software architectures determines sus-tainability to a large extent. Scenario-based software archi-tecture evaluation methods can support sustainability anal-ysis, but they are still reluctantly used in practice. They are also not integrated with architecture-level metrics when evaluating implemented systems, which limits their capabil-ities. Existing literature reviews for architecture evaluation focus on scenario-based methods, but do not provide a criti-cal reflection of the applicability of such methods for sustain-ability evaluation. Our goal is to measure the sustainabil-ity of a software architecture both during early design us-ing scenarios and during evolution using scenarios and met-rics, which is highly relevant in practice. We thus provide a systematic literature review assessing scenario-based meth-ods for sustainability support and categorize more than 40 architecture-level metrics according to several design prin-ciples. Our review identifies a need for further empirical research, for the integration of existing methods, and for the more efficient use of formal architectural models. 1

A Unified Metamodel for Assessing and Predicting Software Evolvability Quality

Software quality is a key assessment factor for organizations to determine the ability of software ecosystems to meet the constantly changing requirements. Many quality models exist that capture and assess the changing factors affecting the quality of a software product. Common to these models is that they, contrary to the software ecosystems they are assessing, are not evolvable or reusable. The thesis first defines what constitutes a unified, evolvable, and reusable quality metamodel. We then introduce SE-EQUAM, a novel, ontological, quality assessment metamodel that was designed from the ground up to support quality unification, reuse, and evolvability. We then validate the reus-ability of our metamodel through instantiating a domain specific quality assessment model called OntEQAM that assesses evolvability as a non-functional software quality based on product and com-munity dimensions. A fuzzy logic based assessment process that addresses uncertainties around score boundaries supports the evolvability quality assessment. The presented assessment process also uses the unified representation of the input knowledge artifacts, the metamodel, and the model to provide a fuzzy assessment score. Finally, we further interpret and predict the evolvability as-sessment scores using a novel, cross-disciplinary approach that re-applies financial technical analy-sis, which are indicators, and patterns typically used for price analysis and the forecasting of stocks in financial markets. We performed several case studies to illustrate and evaluate the applicability of our proposed evolvability score prediction approach

Evolved transistor array robot controllers

For the first time a field programmable transistor array (FPTA) was used to evolve robot control circuits directly in analog hardware. Controllers were successfully incrementally evolved for a physical robot engaged in a series of visually guided behaviours, including finding a target in a complex environment where the goal was hidden from most locations. Circuits for recognising spoken commands were also evolved and these were used in conjunction with the controllers to enable voice control of the robot, triggering behavioural switching. Poor quality visual sensors were deliberately used to test the ability of evolved analog circuits to deal with noisy uncertain data in realtime. Visual features were coevolved with the controllers to automatically achieve dimensionality reduction and feature extraction and selection in an integrated way. An efficient new method was developed for simulating the robot in its visual environment. This allowed controllers to be evaluated in a simulation connected to the FPTA. The controllers then transferred seamlessly to the real world. The circuit replication issue was also addressed in experiments where circuits were evolved to be able to function correctly in multiple areas of the FPTA. A methodology was developed to analyse the evolved circuits which provided insights into their operation. Comparative experiments demonstrated the superior evolvability of the transistor array medium

Animating the evolution of software

The use and development of open source software has increased significantly in the last decade. The high frequency of changes and releases across a distributed environment requires good project management tools in order to control the process adequately. However, even with these tools in place, the nature of the development and the fact that developers will often work on many other projects simultaneously, means that the developers are unlikely to have a clear picture of the current state of the project at any time. Furthermore, the poor documentation associated with many projects has a detrimental effect when encouraging new developers to contribute to the software. A typical version control repository contains a mine of information that is not always obvious and not easy to comprehend in its raw form. However, presenting this historical data in a suitable format by using software visualisation techniques allows the evolution of the software over a number of releases to be shown. This allows the changes that have been made to the software to be identified clearly, thus ensuring that the effect of those changes will also be emphasised. This then enables both managers and developers to gain a more detailed view of the current state of the project. The visualisation of evolving software introduces a number of new issues. This thesis investigates some of these issues in detail, and recommends a number of solutions in order to alleviate the problems that may otherwise arise. The solutions are then demonstrated in the definition of two new visualisations. These use historical data contained within version control repositories to show the evolution of the software at a number of levels of granularity. Additionally, animation is used as an integral part of both visualisations - not only to show the evolution by representing the progression of time, but also to highlight the changes that have occurred. Previously, the use of animation within software visualisation has been primarily restricted to small-scale, hand generated visualisations. However, this thesis shows the viability of using animation within software visualisation with automated visualisations on a large scale. In addition, evaluation of the visualisations has shown that they are suitable for showing the changes that have occurred in the software over a period of time, and subsequently how the software has evolved. These visualisations are therefore suitable for use by developers and managers involved with open source software. In addition, they also provide a basis for future research in evolutionary visualisations, software evolution and open source development

The Watchmaker's guide to Artificial Life: On the Role of Death, Modularity and Physicality in Evolutionary Robotics

Photograph used for a newspaper owned by the Oklahoma Publishing Company