Selection of third party software in Off-The-Shelf-based software development: an interview study with industrial practitioners

The success of software development using third party components highly depends on the ability to select a suitable component for the intended application. The evidence shows that there is limited knowledge about current industrial OTS selection practices. As a result, there is often a gap between theory and practice, and the proposed methods for supporting selection are rarely adopted in the industrial practice. This paper's goal is to investigate the actual industrial practice of component selection in order to provide an initial empirical basis that allows the reconciliation of research and industrial endeavors. The study consisted of semi-structured interviews with 23 employees from 20 different software-intensive companies that mostly develop web information system applications. It provides qualitative information that help to further understand these practices, and emphasize some aspects that have been overlooked by researchers. For instance, although the literature claims that component repositories are important for locating reusable components; these are hardly used in industrial practice. Instead, other resources that have not received considerable attention are used with this aim. Practices and potential market niches for software-intensive companies have been also identified. The results are valuable from both the research and the industrial perspectives as they provide a basis for formulating well-substantiated hypotheses and more effective improvement strategies.Peer ReviewedPostprint (author's final draft

A framework for cots software evaluation and selection for COTS mismatches handling and non-functional requirements

The decision to purchase Commercial Off-The-Shelf (COTS) software needs systematic guidelines so that the appropriate COTS software can be selected in order to provide a viable and effective solution to the organizations. However, the existing COTS software evaluation and selection frameworks focus more on functional aspects and do not give adequate attention to accommodate the mismatch between user requirements and COTS software specification, and also integration with non functional requirements of COTS software. Studies have identified that these two criteria are important in COTS software evaluation and selection. Therefore, this study aims to develop a new framework of COTS software evaluation and selection that focuses on handling COTS software mismatches and integrating the nonfunctional requirements. The study is conducted using mixed-mode methodology which involves survey and interview. The study is conducted in four main phases: a survey and interview of 63 organizations to identify COTS software evaluation criteria, development of COTS software evaluation and selection framework using Evaluation Theory, development of a new decision making technique by integrating Analytical Hierarchy Process and Gap Analysis to handle COTS software mismatches, and validation of the practicality and reliability of the proposed COTS software Evaluation and Selection Framework (COTS-ESF) using experts’ review, case studies and yardstick validation. This study has developed the COTS-ESF which consists of five categories of evaluation criteria: Quality, Domain, Architecture, Operational Environment and Vendor Reputation. It also provides a decision making technique and a complete process for performing the evaluation and selection of COTS software. The result of this study shows that the evaluated aspects of the framework are feasible and demonstrate their potential and practicality to be applied in the real environment. The contribution of this study straddles both the research and practical perspectives of software evaluation by improving decision making and providing a systematic guidelines for handling issue in purchasing viable COTS software

System requirements-OSS components: matching and mismatch resolution practices – an empirical study

Developing systems by integrating Open Source Software (OSS) is increasingly gaining importance in the software industry. Although the literature claims that this approach highly impacts Requirements Engineering (RE) practices, there is a lack of empirical evidence to demonstrate this statement. To explore and understand problems and challenges of current system requirement–OSS component matching and mismatches resolution practices in software development projects that integrate one or more OSS components into their software products. Semi-structured in-depth interviews with 25 respondents that have performed RE activities in software development projects that integrate OSS components in 25 different software development companies in Spain, Norway, Sweden, and Denmark. The study uncovers 15 observations regarding system requirements-OSS components matching and mismatch resolution practices used in industrial projects that integrate OSS components. The assessed projects focused mainly on pre-release stages of software applications that integrate OSS components in an opportunistic way. The results also provide details of a set of previously unexplored scenarios when solving system requirement–OSS component mismatches; and clarify some challenges and related problems. For instance, although licensing issues and the potential changes in OSS components by their corresponding communities and/or changes in system requirements have been greatly discussed in the RE literature as problems for OSS component integration, they did not appear to be relevant in our assessed projects. Instead, practitioners highlighted the problem of getting suitable OSS component documentation/information.Peer ReviewedPostprint (author's final draft

The characteristics of successful military IT projects: a cross-country empirical study

In the armed forces, successful digitalization is crucial to ensure effective operations. Much of the existing literature on project factors during the planning and execution phases of public IT projects do not focus specifically on military sector projects. Therefore, the paper aims to provide empirical insights into the characteristics of successful military IT projects. Data from such projects in NATO countries and agencies were collected through interviews and project documents. The findings relating to the main variable of interest, “delivery of client benefit,” supported previous findings on IT project performance. Medium-sized projects performed better than small and large projects, and the agile development method delivered more client benefit than traditional methods. Client involvement apparently had a positive effect on project success. Clearly specified objectives had a statistically significant effect on project success in terms of clients’ benefits. The paper contributes to the gap in research on military IT projects and broadens the project management literature’s focus on time and cost to include delivery of client benefit as a success variable. The use of cross-country data provided unique insights for academics and practitioners regarding which project characteristics affect the successful development and adoption of new software by the armed forces

A Systematic Mapping Study on Off-The-Shelf-based Software Acquisition

Acquiring software from external suppliers and developing less software in-house can help software-developing organizations improve operational efficiency by reducing costs, time and reusing current technologies. Software projects increasingly use Off-The-Shelf (OTS) products. From the acquirer perspective, there is a need to understand in more detail OTS-based software acquisition processes, because they are different to and less well-understood than those for the acquisition of custom software. In this paper we have undertaken a systematic mapping study on OTS-based software acquisition. The study compares and contrasts OTS-based software acquisition and non-OTS-based software acquisition, and identifies factors influencing decision making in OTS-based software acquisition. We find that the main difference is that there is a relationship between determining the software requirements and OTS selection in OTS-based software acquisition. For commercial OTS software, the major factors are functionality and quality of the software, but for open-source OTS software, cost was the most important factor

The Characteristics of Successful Military IT Projects: A Cross-country Empirical Study

In the armed forces, successful digitalization is crucial to ensure effective operations. Much of the existing literature on project factors during the planning and execution phases of public IT projects do not focus specifically on military sector projects. Therefore, the paper aims to provide empirical insights into the characteristics of successful military IT projects. Data from such projects in NATO countries and agencies were collected through interviews and project documents. The findings relating to the main variable of interest, “delivery of client benefit,” supported previous findings on IT project performance. Medium-sized projects performed better than small and large projects, and the agile development method delivered more client benefit than traditional methods. Client involvement apparently had a positive effect on project success. Clearly specified objectives had a statistically significant effect on project success in terms of clients’ benefits. The paper contributes to the gap in research on military IT projects and broadens the project management literature’s focus on time and cost to include delivery of client benefit as a success variable. The use of cross-country data provided unique insights for academics and practitioners regarding which project characteristics affect the successful development and adoption of new software by the armed forces

Data sets and data quality in software engineering

OBJECTIVE - to assess the extent and types of techniques used to manage quality within software engineering data sets. We consider this a particularly interesting question in the context of initiatives to promote sharing and secondary analysis of data sets. METHOD - we perform a systematic review of available empirical software engineering studies. RESULTS - only 23 out of the many hundreds of studies assessed, explicitly considered data quality. CONCLUSIONS - first, the community needs to consider the quality and appropriateness of the data set being utilised; not all data sets are equal. Second, we need more research into means of identifying, and ideally repairing, noisy cases. Third, it should become routine to use sensitivity analysis to assess conclusion stability with respect to the assumptions that must be made concerning noise levels

PABRE: Pattern-Based Requirements Elicitation

This paper presents our PABRE method for facilitating Requirements Elicitation on the basis of Requirement Patterns with the goal of saving time and reducing errors during this activity. The process presented applies for elicitation in Off-The-Shelf selection projects driven by call for tenders processes and uses a Requirement Patterns Catalogue. The process selects patterns from the catalogue that apply to the particular selection project, and convert them into the real requirements that finally configure the project Requirements Book. We show some benefits of the pattern approach for requirements engineers and IT consultants, as well as for customers. Finally we discuss the strengths and weaknesses of the proposal and identify some future work.Peer ReviewedPostprint (published version

A pattern-based method for building requirements documents in call-for-tender processes

This paper presents our PABRE method for facilitating Requirements Elicitation on the basis of Requirement Patterns with the goal of saving time and reducing errors during this activity. The process presented applies for elicitation in Off-The-Shelf selection projects driven by call-for-tender processes and uses a Requirement Patterns Catalogue. The process selects patterns from the catalogue that apply to the particular selection project, and convert them into the real requirements that finally configure the project Requirements Document. We show some benefits of the pattern approach for requirements engineers and IT consultants, as well as for customers. Finally we discuss the strengths and weaknesses of the proposal and identify some future work.Postprint (published version