Evaluating Open Source Software Quality Models against ISO 25010

Quite a number of open source software quality models exist today. These models emerged as a result of the need to measure quality in open source software, which is quite unlike closed source, or proprietary software. ISO 9126 standard forms the basis from which most of these models derive. However, ISO 9126 standard has been replaced by ISO 25010. Therefore, as research endeavors progress towards evolving the “silver bullet” open source software quality model, it is the aim of this paper to evaluate existing open source software quality models against the ISO 25010 standard. The findings from this study reveal a candidate model (from among the existing models) that can be leveraged in deriving a generic open source software quality model

A systematic literature review of open source software quality assessment models

Background: Many open source software (OSS) quality assessment models are proposed and available in the litera- ture. However, there is little or no adoption of these models in practice. In order to guide the formulation of newer models so they can be acceptable by practitioners, there is need for clear discrimination of the existing models based on their speci c properties. Based on this, the aim of this study is to perform a systematic literature review to inves- tigate the properties of the existing OSS quality assessment models by classifying them with respect to their quality characteristics, the methodology they use for assessment, and their domain of application so as to guide the formula- tion and development of newer models. Searches in IEEE Xplore, ACM, Science Direct, Springer and Google Search is performed so as to retrieve all relevant primary studies in this regard. Journal and conference papers between the year 2003 and 2015 were considered since the rst known OSS quality model emerged in 2003. Results: A total of 19 OSS quality assessment model papers were selected. To select these models we have devel- oped assessment criteria to evaluate the quality of the existing studies. Quality assessment models are classi ed into ve categories based on the quality characteristics they possess namely: single-attribute, rounded category, community-only attribute, non-community attribute as well as the non-quality in use models. Our study re ects that software selection based on hierarchical structures is found to be the most popular selection method in the existing OSS quality assessment models. Furthermore, we found that majority (47%) of the existing models do not specify any domain of application. Conclusions: In conclusion, our study will be a valuable contribution to the community and helps the quality assess- ment model developers in formulating newer models and also to the practitioners (software evaluators) in selecting suitable OSS in the midst of alternatives

Applying Software Quality Criteria to Blockchain Applications: A Criteria Catalog

The selection of the suitable blockchain software ecosystem has become very complex, given the growing market. More and more products with different functionality (mainly consensus algorithms and smart contracts) are available on the market. To identify the correct blockchain system for the respective application, a catalog of criteria with a focus on software quality is developed in this work. This catalog supports the selection of the right application and can be individually weighted

Framework for engineering design systems architectures evaluation and selection: case study

Engineering companies face the challenge of developing complex Engineering Design Systems. These systems involve huge financial, people, and time investments within an environment that is characterised by continuously changing technologies and processes. Systems architecture provides the strategies and modelling approaches to ensure that adequate resources are spent in developing the possible To Be states for a target system. Architecture selection and evaluation involves evaluating different architectural alternatives with respect to multiple criteria, hence an Architecture Evaluation Framework which evaluates and down selects the appropriate architectures solutions is crucial to assess how these systems will deliver value over their lifetime, and where to channel the financial and human investments to maximize benefit delivered to the business’ bottom line. In this paper, an evaluation and selection architecture framework is proposed, which targets to maximise the alignment of Engineering Design Systems with business goals based on a quality centric architecture evaluation approach. The framework utilised software Quality Attributes as well as SWOT (Strength, Weakness, Opportunity, Threat) and PEST (Political, Economic, Social, Technological) analyses to capture different viewpoints related to technical, political and business context. The framework proposed employing AHP (Analytical Hierarchy Process) to quantitatively elicit relationships between Quality Attributes trade-offs and architectural characteristics. The framework was applied to a real case study considering five Engineering Design Systems alternative architectures, where workshops with subject matter experts and stakeholders were held to reach an informative decision, that maximise architectural quality, whilst maintaining business alignment

BIM for landscape design improving climate adaptation planning: the evaluation of software tools based on the ISO 25010 standard

This paper investigates the capabilities and limitations of different software tools simulating landscape design adaptability. The evaluation of tools is based on the ISO 25010 framework, which investigates software functionality, reliability, performance efficiency, usability, compatibility, and information quality. These quality characteristics of software are analysed during objective experiments where five software tools are used for a case study project at the conceptual design phase. These experiments reveal that the existing software tools for climate adaptation planning are focused on different aspects of climate adaptability, generating different types of information. Moreover, all tools deal with some limitations in terms of compatibility, performance efficiency, and functional operations. The ISO 25010 quality model provides a comprehensive framework to compare the capabilities of different software tools for climate adaptation planning. This paper is part of a wider study including an analysis of the needs of project stakeholders regarding climate adaptation software tools. However, this article focuses on technical capabilities of current climate adaptation software tools

Penetration Testing Frameworks and methodologies: A comparison and evaluation

Cyber security is fast becoming a strategic priority across both governments and private organisations. With technology abundantly available, and the unbridled growth in the size and complexity of information systems, cyber criminals have a multitude of targets. Therefore, cyber security assessments are becoming common practice as concerns about information security grow. Penetration testing is one strategy used to mitigate the risk of cyber-attack. Penetration testers attempt to compromise systems using the same tools and techniques as malicious attackers thus, aim to identify vulnerabilities before an attack occurs. Penetration testing can be complex depending on the scope and domain area under investigation, for this reason it is often managed similarly to that of a project necessitating the implementation of some framework or methodology. Fortunately, there are an array of penetration testing methodologies and frameworks available to facilitate such projects, however, determining what is a framework and what is methodology within this context can lend itself to uncertainty. Furthermore, little exists in relation to mature frameworks whereby quality can be measured. This research defines the concept of “methodology” and “framework” within a penetration testing context. In addition, the research presents a gap analysis of the theoretical vs. the practical classification of nine penetration testing frameworks and/or methodologies and subsequently selects two frameworks to undergo quality evaluation using a realworld case study. Quality characteristics were derived from a review of four quality models, thus building the foundation for a proposed penetration testing quality model. The penetration testing quality model is a modified version of an ISO quality model whereby the two chosen frameworks underwent quality evaluation. Defining methodologies and frameworks for the purposes of penetration testing was achieved. A suitable definition was formed by way of analysing properties of each category respectively, thus a Framework vs. Methodology Characteristics matrix is presented. Extending upon the nomenclature resolution, a gap analysis was performed to determine if a framework is actually a framework, i.e., it has a sound underlying ontology. In contrast, many “frameworks” appear to be simply collections of tools or techniques. In addition, two frameworks OWASP’s Testing Guide and Information System Security Assessment Framework (ISSAF), were employed to perform penetration tests based on a real-world case study to facilitate quality evaluation based on a proposed quality model. The research suggests there are various ways in which quality for penetration testing frameworks can be measured; therefore concluded that quality evaluation is possible

A Review on Software Quality Forensics: Techniques, Challenges, and Limitations

Software quality forensics plays a vibrant role related to software quality, security, and integrity. The paper aims to derive a software quality forensics model through existing software quality models and their factors. The papers explore quality models, factors, approaches, tools, techniques, and standards regarding software quality investigation and confine the research area for software quality integrity breach forensics. The explore the deviations of quality attributes, standards, factors, and artifacts, it leads to further investigation of root-cause followed by digital evidence procedure for alleged software quality issues. Therefore, there is a need for a software quality forensics model and dedicated standards to fulfill the digital evidence procedure validation, satisfiable, and prosecution in the court of law in the context of alleged or illegal activity investigation quality of software. The paper has  derived the techniques, challenges, and limitations of software quality forensics based on the review of research questions

Selection of penetration testing methodologies: A comparison and evaluation

Cyber security is fast becoming a strategic priority across both governments and private organisations. With technology abundantly available, and the unbridled growth in the size and complexity of information systems, cyber criminals have a multitude of targets. Therefore, cyber security assessments are becoming common practice as concerns about information security grow. Penetration testing is one strategy used to mitigate the risk of cyber-attack. Penetration testers attempt to compromise systems using the same tools and techniques as malicious attackers thus attempting to identify vulnerabilities before an attack occurs. This research details a gap analysis of the theoretical vs. the practical classification of six penetration testing frameworks and/or methodologies. Additionally, an analysis of two of the frameworks was undertaken to evaluate each against six quality characteristics. The characteristics were derived from a modified version of an ISO quality model