Towards guidelines for building a business case and gathering evidence of software reference architectures in industry

Background: Software reference architectures are becoming widely adopted by organizations that need to support the design and maintenance of software applications of a shared domain. For organizations that plan to adopt this architecture-centric approach, it becomes fundamental to know the return on investment and to understand how software reference architectures are designed, maintained, and used. Unfortunately, there is little evidence-based support to help organizations with these challenges. Methods: We have conducted action research in an industry-academia collaboration between the GESSI research group and everis, a multinational IT consulting firm based in Spain. Results: The results from such collaboration are being packaged in order to create guidelines that could be used in similar contexts as the one of everis. The main result of this paper is the construction of empirically-grounded guidelines that support organizations to decide on the adoption of software reference architectures and to gather evidence to improve RA-related practices. Conclusions: The created guidelines could be used by other organizations outside of our industry-academia collaboration. With this goal in mind, we describe the guidelines in detail for their use.Peer ReviewedPostprint (published version

Estimating Impact and Frequency of Risks to Safety and Mission Critical Systems Using CVSS

Many safety and mission critical systems depend on the correct and secure operation of both supportive and core software systems. E.g., both the safety of personnel and the effective execution of core missions on an oil platform depend on the correct recording storing, transfer and interpretation of data, such as that for the Logging While Drilling (LWD) and Measurement While Drilling (MWD) subsystems. Here, data is recorded on site, packaged and then transferred to an on-shore operational centre. Today, the data is transferred on dedicated communication channels to ensure a secure and safe transfer, free from deliberately and accidental faults. However, as the cost control is ever more important some of the transfer will be over remotely accessible infrastructure in the future. Thus, communication will be prone to known security vulnerabilities exploitable by outsiders. This paper presents a model that estimates risk level of known vulnerabilities as a combination of frequency and impact estimates derived from the Common Vulnerability Scoring System (CVSS). The model is implemented as a Bayesian Belief Network (BBN)

Estimating software project effort using analogies

Accurate project effort prediction is an important goal for the software engineering community. To date most work has focused upon building algorithmic models of effort, for example COCOMO. These can be calibrated to local environments. We describe an alternative approach to estimation based upon the use of analogies. The underlying principle is to characterise projects in terms of features (for example, the number of interfaces, the development method or the size of the functional requirements document). Completed projects are stored and then the problem becomes one of finding the most similar projects to the one for which a prediction is required. Similarity is defined as Euclidean distance in n-dimensional space where n is the number of project features. Each dimension is standardised so all dimensions have equal weight. The known effort values of the nearest neighbours to the new project are then used as the basis for the prediction. The process is automated using a PC based tool known as ANGEL. The method is validated on nine different industrial datasets (a total of 275 projects) and in all cases analogy outperforms algorithmic models based upon stepwise regression. From this work we argue that estimation by analogy is a viable technique that, at the very least, can be used by project managers to complement current estimation techniques

Can k-NN imputation improve the performance of C4.5 with small software project data sets? A comparative evaluation

Missing data is a widespread problem that can affect the ability to use data to construct effective prediction systems. We investigate a common machine learning technique that can tolerate missing values, namely C4.5, to predict cost using six real world software project databases. We analyze the predictive performance after using the k-NN missing data imputation technique to see if it is better to tolerate missing data or to try to impute missing values and then apply the C4.5 algorithm. For the investigation, we simulated three missingness mechanisms, three missing data patterns, and five missing data percentages. We found that the k-NN imputation can improve the prediction accuracy of C4.5. At the same time, both C4.5 and k-NN are little affected by the missingness mechanism, but that the missing data pattern and the missing data percentage have a strong negative impact upon prediction (or imputation) accuracy particularly if the missing data percentage exceeds 40%

An Empirical Study of a Repeatable Method for Reengineering Procedural Software Systems to Object- Oriented Systems

This paper describes a repeatable method for reengineering a procedural system to an object-oriented system. The method uses coupling metrics to assist a domain expert in identifying candidate objects. An application of the method to a simple program is given, and the effectiveness of the various coupling metrics are discussed. We perform a detailed comparison of our repeatable method with an ad hoc, manual reengineering effort based on the same procedural program. The repeatable method was found to be effective for identifying objects. It produced code that was much smaller, more efficient, and passed more regression tests than the ad hoc method. Analysis of object-oriented metrics indicated both simpler code and less variability among classes for the repeatable method