How reliable are systematic reviews in empirical software engineering?

BACKGROUND – the systematic review is becoming a more commonly employed research instrument in empirical software engineering. Before undue reliance is placed on the outcomes of such reviews it would seem useful to consider the robustness of the approach in this particular research context. OBJECTIVE – the aim of this study is to assess the reliability of systematic reviews as a research instrument. In particular we wish to investigate the consistency of process and the stability of outcomes. METHOD – we compare the results of two independent reviews under taken with a common research question. RESULTS – the two reviews find similar answers to the research question, although the means of arriving at those answers vary. CONCLUSIONS – in addressing a well-bounded research question, groups of researchers with similar domain experience can arrive at the same review outcomes, even though they may do so in different ways. This provides evidence that, in this context at least, the systematic review is a robust research method

Predicting with sparse data

It is well known that effective prediction of project cost related factors is an important aspect of software engineering. Unfortunately, despite extensive research over more than 30 years, this remains a significant problem for many practitioners. A major obstacle is the absence of reliable and systematic historic data, yet this is a sine qua non for almost all proposed methods: statistical, machine learning or calibration of existing models. In this paper we describe our sparse data method (SDM) based upon a pairwise comparison technique and Saaty's Analytic Hierarchy Process (AHP). Our minimum data requirement is a single known point. The technique is supported by a software tool known as DataSalvage. We show, for data from two companies, how our approach — based upon expert judgement — adds value to expert judgement by producing significantly more accurate and less biased results. A sensitivity analysis shows that our approach is robust to pairwise comparison errors. We then describe the results of a small usability trial with a practising project manager. From this empirical work we conclude that the technique is promising and may help overcome some of the present barriers to effective project prediction

E/Valuating new media in language development

This paper addresses the need for a new approach to the educational evaluation of software that falls under the rubric "new media" or "multimedia" as distinct from previous generations of Computer-Assisted Language Learning (CALL) software. The authors argue that present approaches to CALL software evaluation are not appropriate for a new genre of CALL software distinguished by its shared assumptions about language learning and teaching as well as by its technical design. The paper sketches a research-based program called "E/Valuation" that aims to assist language educators to answer questions about the educational effectiveness of recent multimedia language learning software. The authors suggest that such program needs to take into account not only the nature of the new media and its potential to promote language learning in novel ways, but also current professional knowledge about language learning and teaching

Selection of third party software in Off-The-Shelf-based software development: an interview study with industrial practitioners

The success of software development using third party components highly depends on the ability to select a suitable component for the intended application. The evidence shows that there is limited knowledge about current industrial OTS selection practices. As a result, there is often a gap between theory and practice, and the proposed methods for supporting selection are rarely adopted in the industrial practice. This paper's goal is to investigate the actual industrial practice of component selection in order to provide an initial empirical basis that allows the reconciliation of research and industrial endeavors. The study consisted of semi-structured interviews with 23 employees from 20 different software-intensive companies that mostly develop web information system applications. It provides qualitative information that help to further understand these practices, and emphasize some aspects that have been overlooked by researchers. For instance, although the literature claims that component repositories are important for locating reusable components; these are hardly used in industrial practice. Instead, other resources that have not received considerable attention are used with this aim. Practices and potential market niches for software-intensive companies have been also identified. The results are valuable from both the research and the industrial perspectives as they provide a basis for formulating well-substantiated hypotheses and more effective improvement strategies.Peer ReviewedPostprint (author's final draft

Towards guidelines for building a business case and gathering evidence of software reference architectures in industry

Background: Software reference architectures are becoming widely adopted by organizations that need to support the design and maintenance of software applications of a shared domain. For organizations that plan to adopt this architecture-centric approach, it becomes fundamental to know the return on investment and to understand how software reference architectures are designed, maintained, and used. Unfortunately, there is little evidence-based support to help organizations with these challenges. Methods: We have conducted action research in an industry-academia collaboration between the GESSI research group and everis, a multinational IT consulting firm based in Spain. Results: The results from such collaboration are being packaged in order to create guidelines that could be used in similar contexts as the one of everis. The main result of this paper is the construction of empirically-grounded guidelines that support organizations to decide on the adoption of software reference architectures and to gather evidence to improve RA-related practices. Conclusions: The created guidelines could be used by other organizations outside of our industry-academia collaboration. With this goal in mind, we describe the guidelines in detail for their use.Peer ReviewedPostprint (published version

Identifying Success Factors in Construction Projects: A Case Study

© 2015 by the Project Management Institute. Published online in Wiley Online Library. Defining "project success" has been of interest for many years, and recent developments combine multiple measurable and psychosocial factors that add to this definition. There has also been research into success factors, but little research into the causal chains through which success emerges. Following the multi-dimensionality of "success," this article shows how success factors combine in complex interactions; it describes factors contributing to project performance by a company working on two major construction programs and shows how to map and analyze paths from root causes to success criteria. The study also identifies some specific factors - some generic, some context-dependent - none of these is uncommon but here they come together synergistically

Aligning a Service Provisioning Model of a Service-Oriented System with the ITIL v.3 Life Cycle

Bringing together the ICT and the business layer of a service-oriented system (SoS) remains a great challenge. Few papers tackle the management of SoS from the business and organizational point of view. One solution is to use the well-known ITIL v.3 framework. The latter enables to transform the organization into a service-oriented organizational which focuses on the value provided to the service customers. In this paper, we align the steps of the service provisioning model with the ITIL v.3 processes. The alignment proposed should help organizations and IT teams to integrate their ICT layer, represented by the SoS, and their business layer, represented by ITIL v.3. One main advantage of this combined use of ITIL and a SoS is the full service orientation of the company.Comment: This document is the technical work of a conference paper submitted to the International Conference on Exploring Service Science 1.5 (IESS 2015