A document based traceability model for test management

Software testing has became more complicated in the emergence of distributed network, real-time environment, third party software enablers and the need to test system at multiple integration levels. These scenarios have created more concern over the quality of software testing. The quality of software has been deteriorating due to inefficient and ineffective testing activities. One of the main flaws is due to ineffective use of test management to manage software documentations. In documentations, it is difficult to detect and trace bugs in some related documents of which traceability is the major concern. Currently, various studies have been conducted on test management, however very few have focused on document traceability in particular to support the error propagation with respect to documentation. The objective of this thesis is to develop a new traceability model that integrates software engineering documents to support test management. The artefacts refer to requirements, design, source code, test description and test result. The proposed model managed to tackle software traceability in both forward and backward propagations by implementing multi-bidirectional pointer. This platform enabled the test manager to navigate and capture a set of related artefacts to support test management process. A new prototype was developed to facilitate observation of software traceability on all related artefacts across the entire documentation lifecycle. The proposed model was then applied to a case study of a finished software development project with a complete set of software documents called the On-Board Automobile (OBA). The proposed model was evaluated qualitatively and quantitatively using the feature analysis, precision and recall, and expert validation. The evaluation results proved that the proposed model and its prototype were justified and significant to support test management

HydroShare – A Case Study of the Application of Modern Software Engineering to a Large Distributed Federally-Funded Scientific Software Development Project

HydroShare is an online collaborative system under development to support the open sharing of hydrologic data, analytical tools, and computer models. With HydroShare, scientists can easily discover, access, and analyze hydrologic data and thereby enhance the production and reproducibility of hydrologic scientific results. HydroShare also takes advantage of emerging social media functionality to enable users to enhance information about and collaboration around hydrologic data and models. HydroShare is being developed by an interdisciplinary collaborative team of domain scientists, university software developers, and professional software engineers from ten institutions located across the United States. While the combination of non–co-located, diverse stakeholders presents communication and management challenges, the interdisciplinary nature of the team is integral to the project’s goal of improving scientific software development and capabilities in academia. This chapter describes the challenges faced and lessons learned with the development of HydroShare, as well as the approach to software development that the HydroShare team adopted on the basis of the lessons learned. The chapter closes with recommendations for the application of modern software engineering techniques to large, collaborative, scientific software development projects, similar to the National Science Foundation (NSF)–funded HydroShare, in order to promote the successful application of the approach described herein by other teams for other projects

Higher Order Mutation Testing

Mutation testing is a fault-based software testing technique that has been studied widely for over three decades. To date, work in this field has focused largely on first order mutants because it is believed that higher order mutation testing is too computationally expensive to be practical. This thesis argues that some higher order mutants are potentially better able to simulate real world faults and to reveal insights into programming bugs than the restricted class of first order mutants. This thesis proposes a higher order mutation testing paradigm which combines valuable higher order mutants and non-trivial first order mutants together for mutation testing. To overcome the exponential increase in the number of higher order mutants a search process that seeks fit mutants (both first and higher order) from the space of all possible mutants is proposed. A fault-based higher order mutant classification scheme is introduced. Based on different types of fault interactions, this approach classifies higher order mutants into four categories: expected, worsening, fault masking and fault shifting. A search-based approach is then proposed for locating subsuming and strongly subsuming higher order mutants. These mutants are a subset of fault mask and fault shift classes of higher order mutants that are more difficult to kill than their constituent first order mutants. Finally, a hybrid test data generation approach is introduced, which combines the dynamic symbolic execution and search based software testing approaches to generate strongly adequate test data to kill first and higher order mutants

Bisnesmetriikan kerääminen ja visualisointi pilvipohjaisessa kehitysympäristössä

Monitoring cloud computing resources is a straightforward and common task for any cloud application developer. The problem with current monitoring solutions is that they only focus on infrastructure resources. Many companies on the other hand would need data about the business side of their applications. This thesis extends the current monitoring solutions to capture business metrics from within applications. The metrics are then visualized to quickly allow for better analysis of the data. The tool is composed of three main components. The metrics are captured with a Node.js library that is imported in the monitored application. The library sends the captured data to InfluxDB timeseries database. The data is visualized with Grafana which implements tables, graphs, and gauges. The provided command-line tool creates a file that can be imported in Grafana to create a new dashboard with graphs in it. The requirements for the tool were created through the needs of software developers and clients of web- and mobile-developer Codemate. An architectural design was made based on the requirements and then implemented on the AWS cloud platform on top of Kubernetes. The implementation was evaluated by testing it in a real production server. The tool is functional and it works as intended. The results from the evaluation prove that the tool created in this thesis can help companies gain better information about their products. Future work includes adding the metrics capture for other languages such as Go and Ruby as well as integrating the tool to Codemate’s new development environment. Further research can be done especially in improving performance of the solution in large systems.Pilviresurssien monitorointi on selkeä ja yleinen tehtävä jokaiselle pilvipalvelun kehittäjälle. Monitorointisovellukset keskittyvät vain infrastruktuuriresursseihin, vaikka monet nykyajan yritykset tarvitsisivat tarkempaa tietoa sovellusten bisnespuolesta. Tämä diplomityö laajentaa nykyisiä monitorointisovelluksia kattamaan bisnesmetriikan keräämisen applikaatioiden sisältä sekä visualisoi datan paremman analyysin mahdollistamiseksi. Diplomityössä kehitetty työkalu koostuu kolmesta osasta. Metriikat kerätään sovelluksista Node.js-kirjaston avulla, joka lisätään sovelluksen koodiin. Kirjasto lähettää dataa InfluxDB-tietokantaan, josta se visualisoidaan Grafanalla interaktiivisten kuvaajien sekä taulukoiden avulla. Grafanaan voidaan lisäksi luoda työpöytiä diplomityötä varten luodulla ohjelmalla. Bisnesmetriikan keräämiseen ja visualisointiin luotu työkalu määriteltiin ohjelmistokehittäjä Codematen ohjelmistoinsinöörien sekä asiakkaiden tarpeiden mukaan. Määrittelyä käytettiin työkalun arkkitehtuurin luomiseen, joka ohjasi käytännön toteutusta. Työkalu rakennettiin Amazonin AWS-palveluun Kuberneteksen päälle. Toteutetun työkalun toimivuus testattiin lopuksi asiakasympäristössä tuotantopalvelimella. Työkalun todettiin toimivan tarkoituksenmukaisesti. Testauksesta saadut tulokset osoittavat, että työkalu voi auttaa yrityksiä saamaan parempaa informaatiota ohjelmistotuotteistaan sekä niiden käytöstä. Työkalun kehitystä voidaan jatkaa laajentamalla sen toimintaa Go- ja Ruby-kielille sekä integroimalla se tiiviimmin Codematen uuteen kehitysympäristöön. Lisätutkimus erityisesti suorituskyvyn parantamiseen laajoissa järjestelmissä on tarpeen

Using formal methods to support testing

Formal methods and testing are two important approaches that assist in the development of high quality software. While traditionally these approaches have been seen as rivals, in recent years a new consensus has developed in which they are seen as complementary. This article reviews the state of the art regarding ways in which the presence of a formal specification can be used to assist testing

Automated Realistic Test Input Generation and Cost Reduction in Service-centric System Testing

Service-centric System Testing (ScST) is more challenging than testing traditional software due to the complexity of service technologies and the limitations that are imposed by the SOA environment. One of the most important problems in ScST is the problem of realistic test data generation. Realistic test data is often generated manually or using an existing source, thus it is hard to automate and laborious to generate. One of the limitations that makes ScST challenging is the cost associated with invoking services during testing process. This thesis aims to provide solutions to the aforementioned problems, automated realistic input generation and cost reduction in ScST. To address automation in realistic test data generation, the concept of Service-centric Test Data Generation (ScTDG) is presented, in which existing services used as realistic data sources. ScTDG minimises the need for tester input and dependence on existing data sources by automatically generating service compositions that can generate the required test data. In experimental analysis, our approach achieved between 93% and 100% success rates in generating realistic data while state-of-the-art automated test data generation achieved only between 2% and 34%. The thesis addresses cost concerns at test data generation level by enabling data source selection in ScTDG. Source selection in ScTDG has many dimensions such as cost, reliability and availability. This thesis formulates this problem as an optimisation problem and presents a multi-objective characterisation of service selection in ScTDG, aiming to reduce the cost of test data generation. A cost-aware pareto optimal test suite minimisation approach addressing testing cost concerns during test execution is also presented. The approach adapts traditional multi-objective minimisation approaches to ScST domain by formulating ScST concerns, such as invocation cost and test case reliability. In experimental analysis, the approach achieved reductions between 69% and 98.6% in monetary cost of service invocations during testin

Teams in agile software development: Design principles and examination of human factors

In response to new customer requirements, market dynamics, mergers, and technological innovation, modern software development organizations are adopting agile software development (ASD). Yet, the simple adoption of agile methods such as Scrum or eXtreme programming does not automatically result in a very agile team. While we understand the introduction and adoption of ASD from a methodical perspective, we have yet to explore design principles that guide methodical extensions of ASD, and we need to learn more about the human factors that influence software development teams. This thesis presents four studies. Studies 1 and 2 investigate the methodical extension of ASD by identifying design principles from secondary data. Study 1 extends ASD with processes and practices from user-centered design. Study 2 investigates early activities that precede development activities. The thesis also investigates human factors of agile software development in studies 3 and 4. Study 3 compares teams along their extents of agility in order to identify influential factors using a multicase study design. Study 4 tests the effects of emotional contagion in virtual software development teams using a large dataset from an open source software repository. Thus, this thesis makes two primary contributions. First, it develops design principles for methodical extensions of ASD; second, it contributes to the human factors that influence software development teams. Managers also receive guidance on the improvement of ASD in their organization

Web information systems: A study of maintenance, change and flexibility

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.Information Systems (IS’s) have provided organisations with huge efficiency gains and benefits over the years; however an outstanding problem that is yet to be successfully tackled is that of the troublesome maintenance phase. Consuming vast resources and thwarting business progression in a competitive global market place, system maintenance has been recognised as one of the key areas where IS is failing organisations. Organisations are too often faced with the dilemma of either replacement or the continual upkeep of an unwieldy system. The ability for IS’s to be able to adapt to exogenous influences is even more acute today than at any time in the past. This is due to IS’s namely, Web Information Systems (WIS’s) increasingly and continually having to accommodate the needs of organisations to interconnect with a plethora of additional systems as well as supporting evolving business models. The richness of the interconnectivity, functionalities and services WIS’s now offer are shaping social, cultural and economic behaviour on a truly global scale, making the maintenance of such systems and evermore pertinent issue. The growth and proliferation of WIS’s shows no sign of abating which leads to the conclusion that what some have termed as the ‘maintenance iceberg’ should not be ignored. The quandary that commercial organisations face is typically driven by two key aspects; firstly, systems are built on the cultural premise of using fixed requirements, with not enough thought or attention being paid to systems abilities to deviate from these requirements. Secondly, systems do not generally cope well with adapting to unpredictable change arising from outside of the organisations environment. Over the recent past, different paradigms, approaches and methods have attempted to make software development more predictable, controllable and adaptable, however, the benefits of such measures in relation to the maintenance dilemma have been limited. The concept of flexible systems that are able to cope with such change in an efficient manner is currently an objective that few can claim to have realised successfully. The primary focus of the thesis was to examine WIS post-development change in order to empirically substantiate and understand the nature of the maintenance phase. This was done with the intention to determine exactly ‘where’ and ‘how’ flexibility could be targeted to address these changes. This study uses an emergent analytical approach to identify and catalogue the nature of change occurring within WIS maintenance. However, the research framework design underwent a significant revision as the initial results indicated that a greater emphasis and refocus was required to achieve the research objective. To study WIS’s in an appropriate and detailed context, a single case study was conducted in a web development software house. In total the case study approach was used to collect empirical evidence from four projects that investigated post-development change requests in order to identify areas of the system susceptible to change. The maintenance phases of three WIS projects were considered in-depth, resulting in the collection of over four hundred change requests. The fourth project served as a validation case. The results are presented and the findings are used to identify key trends and characteristics that depict WIS maintenance change. The analytical information derived from the change requests is consolidated and shown diagrammatically for the key areas of change using profile models developed in this thesis. Based on the results, the thesis concludes and contributes to the ongoing debate that there is a discernable difference when considering WIS maintenance change compared to that of traditional IS maintenance. The detailed characteristics displayed in the profile models are then used to map specific flexibility criteria that ultimately are required to facilitate change. This is achieved using the Flexibility Matrix of Change (FMoC) tool which was developed within the remit of this research. This tool is a qualitative measurement scheme that aligns WIS maintenance changes to a reciprocal flexibility attribute. Thus, the wider aim of this thesis is to also expand the awareness of flexibility and its importance as a key component of the WIS lifecycle