Towards guidelines for building a business case and gathering evidence of software reference architectures in industry

Background: Software reference architectures are becoming widely adopted by organizations that need to support the design and maintenance of software applications of a shared domain. For organizations that plan to adopt this architecture-centric approach, it becomes fundamental to know the return on investment and to understand how software reference architectures are designed, maintained, and used. Unfortunately, there is little evidence-based support to help organizations with these challenges. Methods: We have conducted action research in an industry-academia collaboration between the GESSI research group and everis, a multinational IT consulting firm based in Spain. Results: The results from such collaboration are being packaged in order to create guidelines that could be used in similar contexts as the one of everis. The main result of this paper is the construction of empirically-grounded guidelines that support organizations to decide on the adoption of software reference architectures and to gather evidence to improve RA-related practices. Conclusions: The created guidelines could be used by other organizations outside of our industry-academia collaboration. With this goal in mind, we describe the guidelines in detail for their use.Peer ReviewedPostprint (published version

Supporting Defect Causal Analysis in Practice with Cross-Company Data on Causes of Requirements Engineering Problems

[Context] Defect Causal Analysis (DCA) represents an efficient practice to improve software processes. While knowledge on cause-effect relations is helpful to support DCA, collecting cause-effect data may require significant effort and time. [Goal] We propose and evaluate a new DCA approach that uses cross-company data to support the practical application of DCA. [Method] We collected cross-company data on causes of requirements engineering problems from 74 Brazilian organizations and built a Bayesian network. Our DCA approach uses the diagnostic inference of the Bayesian network to support DCA sessions. We evaluated our approach by applying a model for technology transfer to industry and conducted three consecutive evaluations: (i) in academia, (ii) with industry representatives of the Fraunhofer Project Center at UFBA, and (iii) in an industrial case study at the Brazilian National Development Bank (BNDES). [Results] We received positive feedback in all three evaluations and the cross-company data was considered helpful for determining main causes. [Conclusions] Our results strengthen our confidence in that supporting DCA with cross-company data is promising and should be further investigated.Comment: 10 pages, 8 figures, accepted for the 39th International Conference on Software Engineering (ICSE'17

A systematic review on the relationship between user involvement and system success

© 2014 Elsevier B.V. All rights reserved. Context: For more than four decades it has been intuitively accepted that user involvement (UI) during system development lifecycle leads to system success. However when the researchers have evaluated the user involvement and system success (UI-SS) relationship empirically, the results were not always positive. Objective: Our objective was to explore the UI-SS relationship by synthesizing the results of all the studies that have empirically investigated this complex phenomenon. Method: We performed a Systematic Literature Review (SLR) following the steps provided in the guidelines of Evidence Based Software Engineering. From the resulting studies we extracted data to answer our 9 research questions related to the UI-SS relationship, identification of users, perspectives of UI, benefits, problems and challenges of UI, degree and level of UI, relevance of stages of software development lifecycle (SDLC) and the research method employed on the UI-SS relationship. Results: Our systematic review resulted in selecting 87 empirical studies published during the period 1980-2012. Among 87 studies reviewed, 52 reported that UI positively contributes to system success, 12 suggested a negative contribution and 23 were uncertain. The UI-SS relationship is neither direct nor binary, and there are various confounding factors that play their role. The identification of users, their degree/level of involvement, stage of SDLC for UI, and choice of research method have been claimed to have impact on the UI-SS relationship. However, there is not sufficient empirical evidence available to support these claims. Conclusion: Our results have revealed that UI does contribute positively to system success. But it is a double edged sword and if not managed carefully it may cause more problems than benefits. Based on the analysis of 87 studies, we were able to identify factors for effective management of UI alluding to the causes for inconsistency in the results of published literature

Maturity Models Architecture: A large systematic mapping

Maturity models are widespread in research and in particular, IT practitioner communities. However, theoretically sound, methodologically rigorous and empirically validated maturity models are quite rare.  This systematic mapping paper focuses on the challenges faced during the development of maturity models. More specifically, it explores the literature on maturity models and standard guidelines to develop maturity models, the challenges identified and solutions proposed.  Our systematic mapping  revealed over six hundred articles on maturity models. Extant literature reveals that researchers have primarily focused on developing new maturity models pertaining to domain-specific problems and/or new enterprise technologies. We find rampant re-use of the design structure of widely adopted models such as Nolan’s Stage of Growth Model, Crosby’s Grid, and Capability Maturity Model (CMM). We also identify three dominant views of maturity models and provide guidelines for various approaches to constructing maturity models with a standard vocabulary. We finally propose using process theories and configurational approaches to address the main theoretical criticisms with regard to maturity models and conclude with some recommendations for maturity model developers

A Method for Evaluating End-User Development Technologies

End-user development (EUD) is a strategy that can reduce a considerable amount of business demand on IT departments. Empowering the end-user in the context of software development is only possible through technologies that allow them to manipulate data and information without the need for deep programming knowledge. The successful selection of appropriate tools and technologies is highly dependent on the context in which the end-user is embedded. End-users should be a central piece in any software package evaluation, being key in the evaluation process in the end-user development context. However, little research has empirically examined software package evaluation criteria and techniques in general, and in the end-user development context in particular. This paper aims to provide a method for technology evaluation in the context of end-user development and to present the evaluation of two platforms. We conclude our study proposing a set of suggestions for future research

Reporting experiments to satisfy professionals information needs

Although the aim of empirical software engineering is to provide evidence for selecting the appropriate technology, it appears that there is a lack of recognition of this work in industry. Results from empirical research only rarely seem to find their way to company decision makers. If information relevant for software managers is provided in reports on experiments, such reports can be considered as a source of information for them when they are faced with making decisions about the selection of software engineering technologies. To bridge this communication gap between researchers and professionals, we propose characterizing the information needs of software managers in order to show empirical software engineering researchers which information is relevant for decision-making and thus enable them to make this information available. We empirically investigated decision makers? information needs to identify which information they need to judge the appropriateness and impact of a software technology. We empirically developed a model that characterizes these needs. To ensure that researchers provide relevant information when reporting results from experiments, we extended existing reporting guidelines accordingly.We performed an experiment to evaluate our model with regard to its effectiveness. Software managers who read an experiment report according to the proposed model judged the technology?s appropriateness significantly better than those reading a report about the same experiment that did not explicitly address their information needs. Our research shows that information regarding a technology, the context in which it is supposed to work, and most importantly, the impact of this technology on development costs and schedule as well as on product quality is crucial for decision makers

Empirical Standards for Software Engineering Research

Empirical Standards are natural-language models of a scientific community's expectations for a specific kind of study (e.g. a questionnaire survey). The ACM SIGSOFT Paper and Peer Review Quality Initiative generated empirical standards for research methods commonly used in software engineering. These living documents, which should be continuously revised to reflect evolving consensus around research best practices, will improve research quality and make peer review more effective, reliable, transparent and fair.Comment: For the complete standards, supplements and other resources, see https://github.com/acmsigsoft/EmpiricalStandard

A novel tool for organisational learning and its impact on safety culture in a hospital dispensary

Incident reporting as a key mechanism for organisational learning and the establishment of a stronger safety culture are pillars of the current patient safety movement. Studies have suggested that incident reporting in healthcare does not achieve its full potential due to serious barriers to reporting and that sometimes staff may feel alienated by the process. The aim of the work reported in this paper was to prototype a novel approach to organisational learning that allows an organisation to assess and to monitor the status of processes that often give rise to latent failure conditions in the work environment, and to assess whether and through which mechanisms participation in this approach affects local safety culture. The approach was prototyped in a hospital dispensary using Plan-Do-Study-Act (PDSA) cycles, and the effect on safety culture was described qualitatively through semi-structured interviews. The results suggest that the approach has had a positive effect on the safety culture within the dispensary, and that staff perceive the approach to be useful and usable

Agile Requirements Engineering: A systematic literature review

Nowadays, Agile Software Development (ASD) is used to cope with increasing complexity in system development. Hybrid development models, with the integration of User-Centered Design (UCD), are applied with the aim to deliver competitive products with a suitable User Experience (UX). Therefore, stakeholder and user involvement during Requirements Engineering (RE) are essential in order to establish a collaborative environment with constant feedback loops. The aim of this study is to capture the current state of the art of the literature related to Agile RE with focus on stakeholder and user involvement. In particular, we investigate what approaches exist to involve stakeholder in the process, which methodologies are commonly used to present the user perspective and how requirements management is been carried out. We conduct a Systematic Literature Review (SLR) with an extensive quality assessment of the included studies. We identified 27 relevant papers. After analyzing them in detail, we derive deep insights to the following aspects of Agile RE: stakeholder and user involvement, data gathering, user perspective, integrated methodologies, shared understanding, artifacts, documentation and Non-Functional Requirements (NFR). Agile RE is a complex research field with cross-functional influences. This study will contribute to the software development body of knowledge by assessing the involvement of stakeholder and user in Agile RE, providing methodologies that make ASD more human-centric and giving an overview of requirements management in ASD.Ministerio de Economía y Competitividad TIN2013-46928-C3-3-RMinisterio de Economía y Competitividad TIN2015-71938-RED