Analysis of limitations and metrology weaknesses of enterprise architecture (EA) measurement solutions & proposal of a COSMIC-based approach to EA measurement

The literature on enterprise architecture (EA) posits that EA is of considerable value for organizations. However, while the EA literature documents a number of proposals for EA measurement solutions, there is little evidence-based research supporting their achievements and limitations. This thesis aims at helping the EA community to understand the existing trends in EA measurement research and to recognize the existing gaps, limitations, and weaknesses in EA measurement solutions. Furthermore, this thesis aims to assist the EA community to design EA measurement solutions based on measurement and metrology best practices. The research goal of this thesis is to contribute to the EA body of knowledge by shaping new perspectives for future research avenues in EA measurement research. To achieve the research goal, the following research objectives are defined: 1. To classify the EA measurement solutions into specific categories in order to identify research themes and explain the structure of the research area. 2. To evaluate the EA measurement solutions from a measurement and metrology perspective. 3. To identify the measurement and metrology issues in EA measurement solutions. 4. To propose a novel EA measurement approach based on measurement and metrology guidelines and best practices. To achieve the first objective, this thesis conducts a systematic mapping study (SMS to help understand the state-of-the-art of EA measurement research and classify the research area in order to acquire a general understanding about the existing research trends. To achieve the second and third objectives, this thesis conducts a systematic literature review (SLR) to evaluate the EA measurement solutions from a measurement and metrology perspective, and hence, to reveal the weaknesses of EA measurement solutions and propose relevant solutions to these weaknesses. To perform this evaluation, we develop an evaluation process based on combining both the components of the evolution theory and the concepts of measurement and metrology best practices, such as ISO 15939. To achieve the fourth objective, we propose a mapping between two international standards: • COSMIC - ISO/IEC 19761: a method for measuring the functional size of software. • ArchiMate: a modelling language for EA. This mapping results in proposing a novel EA measurement approach that overcomes the weaknesses and limitations found in the existing EA measurement solutions. The research results demonstrate that: 1. The current publications on EA measurement are trending toward an increased focus on the “enterprise IT architecting” school of thought, lacks the rigorous terminology found in science and engineering and shows limited adoption of knowledge from other disciplines in the proposals of EA measurement solutions. 2. There is a lack of attention to attaining appropriate metrology properties in EA measurement proposals: all EA measurement proposals are characterized with insufficient metrology coverage scoring, theoretical and empirical definitions. 3. The proposed novel EA measurement approach demonstrates that it is handy for EA practitioners, and easy to adopt by organizations

SPQmm : a software product quality maturity model using ISO/IEEE standards, metrology, and sigma concepts

In the software engineering literature, there are numerous maturity models for assessing and evaluating a set of software processes. By contrast, there is no corresponding maturity model for assessing the quality of a software product. The design of such a model to assess the quality of a software product therefore represents a new research challenge in software engineering. Our main goal is to make available to industry (and consumers) a maturity model for assessing and improving the quality of the software product. This Software Product Quality Maturity Model (SPQMM) consists of three quality maturity submodels (viewpoints) that can be used not only once the software product has been delivered, but also throughout the life-cycle: • Software Product Internal Quality Maturity Model- SPIQMM, • Software Product External Quality Maturity Model - SPEQMM, and • Software Product Quality-in-Use Maturity Model- SPQiUMM In this thesis, we introduce the Software Product Quality Maturity Model (SPQMM), which could be used from three different viewpoints: the software product internal quality, the software product external quality, and the software product quality in-use. This quality maturity model is a quantitative model, and it based on the ISO 9126(software product quality measures), ISO 15026 (software integrity levels), IEEE Std. 1012 (software verification and validation) and on six-sigma concepts. To build such a quality maturity model, we have combined the set of quality measures into a single number for each quality characteristic by assuming that all the measures for a single quality characteristic have an equal weight in the computation of a single value for that quality characteristic (they all make an equal contribution), yielding a quality level for that quality characteristic. The resulting quality level is then transformed based on the software integrity level into a sigma value positioned within a quality maturity level

Towards a homogeneous characterization of the model-driven web development methodologies

In recent years a large number of Model-Driven Web development approaches have been designed and are being applied with success in real environments. However, as new ones are frequently emerging in this changing time, authors have to change and update them constantly and, consequently; development teams do not know which is the most suitable for them because in many cases it depends on their project scope. Furthermore, approaches are usually appearing with different concepts and terminologies in many cases, although all lack the use of standards and practical experience. Thus, the need of managing quality in this type of approach arises every day. This paper suggests a characterization of these methodologies in order to use this information for the quality management of Model-Driven Web development methodologies for authors and development teams alike. In addition, an experimental study in order to analyse and evaluate a Model-Driven Web development methodology (the NDT methodology) has been carried out within a specific work context.Junta de Andalucía TIC-5789Ministerio de Educación y Ciencia TIN2010-20057-C03-0

Processus de vérification des mesures de logiciels selon la perspective de métrologie

En génie logiciel, la plupart des mesures ont été proposées en se basant uniquement sur les concepts de la «théorie de mesure». Cependant, dans les autres disciplines matures du génie, c'est le domaine de connaissance se référant à la «métrologie» qui représente la base pour le développement et l'utilisation des instruments de mesure et des processus de mesure. L'objectif principal de cette thèse consiste à développer un cadre intégré pour le processus de vérification des méthodes de mesure de logiciels et qui intègre la perspective de métrologie au domaine de mesure de logiciels. Pour atteindre cet objectif, le document ISO de la métrologie VIM a été modélisé et un cadre de référence de vérification de mesure a été développé. Ce cadre a été utilisé pour analyser les normes ISO et le domaine de connaissance de la mesure en génie logiciel. Le résultat de cette analyse a révélé l'importance de la métrologie non seulement pour améliorer le design des mesures proposées à l'industrie mais aussi pour analyser la qualité des instruments de mesure qui lui sont associés. Le cadre intégré pourra être utilisé comme un modèle d'une méta-norme pour concevoir de nouvelles mesures. Enfin, l'ensemble des modèles développés n'est pas restreint au génie logiciel et il pourrait être utile également à l'ensemble des domaines de la mesure

Quality measurement in agile and rapid software development: A systematic mapping

Context: In despite of agile and rapid software development (ARSD) being researched and applied extensively, managing quality requirements (QRs) are still challenging. As ARSD processes produce a large amount of data, measurement has become a strategy to facilitate QR management. Objective: This study aims to survey the literature related to QR management through metrics in ARSD, focusing on: bibliometrics, QR metrics, and quality-related indicators used in quality management. Method: The study design includes the definition of research questions, selection criteria, and snowballing as search strategy. Results: We selected 61 primary studies (2001-2019). Despite a large body of knowledge and standards, there is no consensus regarding QR measurement. Terminology is varying as are the measuring models. However, seemingly different measurement models do contain similarities. Conclusion: The industrial relevance of the primary studies shows that practitioners have a need to improve quality measurement. Our collection of measures and data sources can serve as a starting point for practitioners to include quality measurement into their decision-making processes. Researchers could benefit from the identified similarities to start building a common framework for quality measurement. In addition, this could help researchers identify what quality aspects need more focus, e.g., security and usability with few metrics reported.This work has been funded by the European Union’s Horizon 2020 research and innovation program through the Q-Rapids project (grant no. 732253). This research was also partially supported by the Spanish Ministerio de Economía, Industria y Competitividad through the DOGO4ML project (grant PID2020-117191RB-I00). Silverio Martínez-Fernández worked in Fraunhofer IESE before January 2020.Peer ReviewedPostprint (published version

A performance measurement model for cloud computing applications

Cloud Computing is a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources. One of the main objectives of this technology is processing and storing very large amounts of data by means of Cloud Computing Applications. Sometimes, anomalies and defects found in the Cloud platforms affect the performance of these applications resulting in degradation of the Cloud performance. These anomalies can be identified by performance concepts of Cloud Computing based on software engineering quality models. One of the challenges in Cloud Computing is how to analyze the performance of Cloud Computing Applications in order to determine the main factors which affect the quality of the Cloud. Performance measurement results are very important because they help to detect the source of the degradation of the Cloud and, as a consequence, improve its performance. Furthermore, such results can be used in future resource planning stages or for the design of Service Level Agreements. This thesis presents Cloud Computing Application concepts that are directly related to the measurement of performance from a quantitative viewpoint. One of the challenges in defining such concepts is how to determine what type of relationships exist between the various performance base measures that define the performance concepts in a Cloud environment. For example, what is the extent of the relationship between CPU processing time and performance concepts such as time behavior? In addition, this thesis proposes a performance measurement model for Cloud Computing Applications, which integrates software quality concepts from ISO 25010 and makes use of the Taguchi´s method for the design of experiments in order to present an example of how to apply the model to a practical case

Auditing for ISO 9001 requirements in the context of agile software processes

ISO 9001 demands of (software) organizations that a rigorous demonstration of their software processes be implemented and a set of guidelines followed at various levels of abstraction. What these organizations need to show, in other words, is that their software processes have been designed and implemented in a way that allows for a level of configuration and operation that complies with ISO 9001 requirements. For software organizations needing ISO 9001 certification, it is important that they establish a software process life cycle that can manage the requirements imposed by this certification standard. However, software organizations that develop their software products using the agile software processes, such as Extreme Programming (agile-XP), face a number of challenges in their effort to demonstrate that their process activities conform to ISO 9001 requirements, major ones being: product construction, traceability, and measurement. Agile software organizations must provide evidence of ISO 9001 conformity, and they need to develop their own procedures, tools, and methodologies to do so. As yet, there is no consensus on how to audit the agile software organization to ensure that their software processes have been designed and implemented in conformity with ISO 9001 requirements. Moreover, it is challenging to ensure that such lightweight documentation methodologies meet these requirements for certification purposes. The motivation of this research is to help software organizations that use agile software processes in their effort to meet the ISO 9001 certification requirements. This research project is also aimed at helping IS auditors extract auditing evidence that demonstrates conformity to the ISO 9001 requirements that must be met by agile software organizations. Extreme programming (agile-XP) has been selected for improvement as a candidate agile process. This selection was based on the literature indicating a higher adoption of agile-XP over other agile software processes. The goal of this research project is to improve the ability of the agile-XP process to meet the auditing requirements of ISO 9001. The goal of the research also focuses on helping agile software organizations in their effort to become ISO 9001 certified. The main objective of this research project is to design an auditing model that covers the measurement and traceability requirements of ISO 9001. The auditing model should provide IS auditors with auditing evidence that the software projects developed with the agile-XP process have fulfilled the requirements of ISO 9001. The objective also proposes several sub processes to enhance the early planning activities of agile-XP according to ISO 9001 requirements. To achieve these objectives, the main phases of the research methodology are: Investigation of the capability of agile-XP to achieve the requirements of ISO 9001 software process certification; modification of the early phases of agile-XP (i.e. release planning phase) using CMMI-DEV; and design of an auditing model for ISO 9001 traceability and measurement requirements. The main outcome of this research study, which is an auditing model that is aligned with the principles of agile-XP and focuses on ISO 9001 traceability and measurement requirements to provide the IS auditors with a methodological approach for the auditing process. The auditing model has been assessed based on case studies selected from the literature

The SMPI model : a stepwise process model to facilitate software measurement process improvement along the measurement paradigms

Software engineering, software measurement, software process engineering, capability, maturityMagdeburg, Univ., Fak. für Informatik, Diss., 2007René Braungarte

Development of a prototype for multidimensional performance management in software engineering

Managing performance is an important, and difficult, topic, and tools are needed to help organizations manage their performance. Understanding, and improving performance is an important problem. Performance management has become more and more important for organizations, and managers are always on the lookout for better solutions to manage performance within their organizations. One of the most important consequences of not having a Performance Management Framework (PMF) in place is the difficulty of differentiating organizational success from failure over time. Performance Management Frameworks have become important to organizations that need to plan, monitor, control, and improve their decisions. Use of a PMF can show an organization how it is performing and indicate whether or not an organization is going in the right direction to achieve its objectives. Over the years, several frameworks have been developed to address the management of organizational assets, both tangible and intangible. Performance measurement has always mostly been focused on the economic viewpoint. The framework developed by Kaplan and Norton adds three other viewpoints to this, and this addition represents a significant improvement to PMFs. The PMFs currently proposed do not meet the analytical requirements of software engineering management when various viewpoints must be taken into account concurrently. This difficulty is compounded by the fact that the underlying quantitative data are multidimensional, and so the usual two- and three-dimensional approaches to visualization are generally not sufficient to represent such models. Organizations vary considerably in the wide variety of viewpoints that influence their performance, and every organization has their own viewpoints that they want to manage, and which must be represented in a consolidated manner. The purpose of this thesis is to develop a prototype for managing multidimensional performance in software engineering. The thesis begins by defining the important terms or key concepts used in the research: software, performance, management, model, multidimensional, development, engineering, and prototype, and the various associations of these terms. This is followed by a review of the multidimensional PMFs that are specific to software engineering and the generic multidimensional performance models that are available to management. A framework for managing performance in software engineering in four phases: design, implementation, use of the framework, and performance improvement is then presented. Based on this framework, a prototype tool is developed. The prototype notably includes visual analytical tools to manage, interpret, and understand the results in a consolidated manner, while at the same time keeping track of the values of the individual dimensions of performance. The repository of software project data made available by the International Software Benchmarking Standard Group (ISBSG) is integrated into and used by the prototype as well