How reliable are systematic reviews in empirical software engineering?

BACKGROUND – the systematic review is becoming a more commonly employed research instrument in empirical software engineering. Before undue reliance is placed on the outcomes of such reviews it would seem useful to consider the robustness of the approach in this particular research context. OBJECTIVE – the aim of this study is to assess the reliability of systematic reviews as a research instrument. In particular we wish to investigate the consistency of process and the stability of outcomes. METHOD – we compare the results of two independent reviews under taken with a common research question. RESULTS – the two reviews find similar answers to the research question, although the means of arriving at those answers vary. CONCLUSIONS – in addressing a well-bounded research question, groups of researchers with similar domain experience can arrive at the same review outcomes, even though they may do so in different ways. This provides evidence that, in this context at least, the systematic review is a robust research method

How repeatable are systematic reviews?

As a discipline relatively new to the conduct of systematic reviews it could be said that we are still "finding our way" in undertaking them. Still far from the norm in empirical software engineering, we are having to borrow models and guidelines from other fields and adapt them as we proceed. This is both appropriate and effective in terms of learning how to conduct a systematic review. Before we come to rely on such reviews too heavily, however, it is important to assess their reliability. We all appreciate the need to consider model credibility in our empirical work – similarly we should be considering review credibility. This talk will report on a comparison of two independent reviews intentionally undertaken with the same research issue in mind

Software source code sizing using fuzzy logic modeling

Knowing the likely size of a software product before it has been constructed is potentially beneficial in project management: for instance, size can be an important factor in determining an appropriate development/integration schedule, and it can be a significant input in terms of the allocation of personnel and other resources. In this study we consider the applicability of fuzzy logic modeling methods to the task of software source code sizing, using a previously published data set. Our results suggest that, particularly with refinement using data and knowledge, fuzzy predictive models can outperform their traditional regression-based counterparts

Understanding Class-level Testability Through Dynamic Analysis

It is generally acknowledged that software testing is both challenging and time-consuming. Understanding the factors that may positively or negatively affect testing effort will point to possibilities for reducing this effort. Consequently there is a significant body of research that has investigated relationships between static code properties and testability. The work reported in this paper complements this body of research by providing an empirical evaluation of the degree of association between runtime properties and class-level testability in object-oriented (OO) systems. The motivation for the use of dynamic code properties comes from the success of such metrics in providing a more complete insight into the multiple dimensions of software quality. In particular, we investigate the potential relationships between the runtime characteristics of production code, represented by Dynamic Coupling and Key Classes, and internal class-level testability. Testability of a class is consider ed here at the level of unit tests and two different measures are used to characterise those unit tests. The selected measures relate to test scope and structure: one is intended to measure the unit test size, represented by test lines of code, and the other is designed to reflect the intended design, represented by the number of test cases. In this research we found that Dynamic Coupling and Key Classes have significant correlations with class-level testability measures. We therefore suggest that these properties could be used as indicators of class-level testability. These results enhance our current knowledge and should help researchers in the area to build on previous results regarding factors believed to be related to testability and testing. Our results should also benefit practitioners in future class testability planning and maintenance activities

Packaged Software Implementation Requirements Engineering by Small Software Enterprises

Small to medium sized business enterprises (SMEs) generally thrive because they have successfully done something unique within a niche market. For this reason, SMEs may seek to protect their competitive advantage by avoiding any standardization encouraged by the use of packaged software (PS). Packaged software implementation at SMEs therefore presents challenges relating to how best to respond to mismatches between the functionality offered by the packaged software and each SME's business needs. An important question relates to which processes small software enterprises - or Small to Medium-Sized Software Development Companies (SMSSDCs) - apply in order to identify and then deal with these mismatches. To explore the processes of packaged software (PS) implementation, an ethnographic study was conducted to gain in-depth insights into the roles played by analysts in two SMSSDCs. The purpose of the study was to understand PS implementation in terms of requirements engineering (or 'PSIRE'). Data collected during the ethnographic study were analyzed using an inductive approach. Based on our analysis of the cases we constructed a theoretical model explaining the requirements engineering process for PS implementation, and named it the PSIRE Parallel Star Model. The Parallel Star Model shows that during PSIRE, more than one RE process can be carried out at the same time. The Parallel Star Model has few constraints, because not only can processes be carried out in parallel, but they do not always have to be followed in a particular order. This paper therefore offers a novel investigation and explanation of RE practices for packaged software implementation, approaching the phenomenon from the viewpoint of the analysts, and offers the first extensive study of packaged software implementation RE (PSIRE) in SMSSDCs

Onshore to near-shore outsourcing transitions: unpacking tensions

This study is directed towards highlighting tensions of incoming and outgoing vendors during outsourcing in a near-shore context. Incoming-and-outgoing of vendors generate a complex form of relationship in which the participating organizations cooperate and compete simultaneously. It is of great importance to develop knowledge about this kind of relationship typically in the current GSE-related multi-sourcing environment. We carried out a longitudinal case study and utilized data from the 'Novo pay' project, which is available in the public domain. This project involved an outgoing New Zealand based vendor and incoming Australian based vendor. The results show that the demand for the same human resources, dependency upon cooperation and collaboration between vendors, reliance on each other system's configurations and utilizing similar strategies by the client, which worked for the previous vendor, generated a set of tensions which needed to be continuously managed throughout the project