Software process quality models: a comparative evaluation

Numerous software processes are implemented by software organisations in the production and maintenance of software products. Varying levels of success are observed in their execution, as processes vary in content and quality. A number of quality models for software processes have been published, each of which is intended to encompass the totality of quality factors and issues relevant to a specific notion of process quality. These quality models may be used to develop a new process, measure the quality of existing processes, or guide improvement of existing processes. It is therefore desirable that mechanisms exist to select the model of highest intrinsic quality and greatest relevance. In this thesis, mechanisms are proposed for the comparative evaluation of software process quality models. Case studies are performed in which existing software process quality models are applied to existing software processes. Case study results are used in empirical evaluation of models to augment theoretical evaluation results. Specific recommendations are made for selection of models against typical selection criteria. Assessment is performed of the assessment procedures against defined success criteria. Theoretical evaluation procedures are developed to measure process quality models against defined quality criteria. Measurements are performed of conformance of models to the requirements set for an ideal process quality model, and the relevance of model content to defined stakeholders in software processes. Comparison is also made of the scope and size of models. Empirical evaluation procedures are developed to assess model performance in the context of application to real software processes. These procedures assess the extent to which the results of process measurement using process quality models are observed to differ, and hence the importance of selecting one model in preference to others. Measurement is also performed of the extent of difference in the software processes evaluated in the case studies

MS

thesisSeveral methods exist for monitoring software development. Few formal evaluation methods have been applied to measure and improve clinical software application problems once the software has been implemented in the clinical setting. A standardized software problem classification system was developed and implemented at the University of Utah Health Sciences Center. External validity was measured by a survey of 14 University Healthcare Consortium (UHC) hospitals. Internal validation was accomplished by: an indepth analysis of problems details; revision in the problem ticket format; verification from staff within the information systems department; and mapping of old problems to the new classification system. Cohen's Kappa statistics of agreement, used for reliability testing of the new classification systems, revealed good agreement (Kappa = .6162) among HELP Desk agents in consistency of classifying problems calls. A monthly quality improvement report template with the following categories was developed from the new classification system: top 25 problems; unplanned server downtimes; problem summaries; customer satisfaction survey results; top problems details; case analyses; and follow-up of case analysis. Continuous Quality Improvement (CQ) methodology was applied to problem reporting within the Office of Information Resources (OIR) and a web-based ticket entry system was implemented. The new system has resulted in the following benefits: reduction in problem resolution times by one third; improved problem ticket information; shift of 2 FTEs from call center to dispatch due to the increased efficiency of the HELP DESK; and a trend in improvement of customer satisfaction as measured by an online survey. The study provided an internal quality model for the OIR department and the UUHSC. The QM report template provided a method for tracking and trending software problems to use in conducting evaluation and quality improvement studies. The template also provided data for analysis and improvement studies. The template also provided data for analysis and improvement of customer satisfaction. The study has further potential as a model for information system departments at other health care institutions for implementing quality improvement methods. There is potential for improvement in the information technology, social, organizational, and cultural aspects as key issues emerge over time. There can be many consequences to the data collected and many consequences of change can be studied

Developing Socially-Constructed Quality Metrics in Agile: A Multi-Faceted Perspective

This research proposes development of socially-constructed metrics for quality assessment and improvement in Agile Software Development (ASD) projects. The first phase of our research includes an extensive literature review, which indicates that traditional (outcome-focused) metrics that evaluate quality are not directly transferable to adaptive, ASD projects. We then conduct semi-structured interviews confirming the necessity of considering people and process aspects for quality considerations in agile. We propose three dimensions for composite metrics in ASD, namely, (1) evidence (2) expectation and (3) critical evaluation. This combines quantitative and qualitative information drawn from people, process, and outcome-related factors. The proposed model allows ASD teams to concurrently conduct quality assessment and improvement during their projects, producing innovative metrics, adhering to the core principles of the agile manifesto. In our next research stage, this reference model will be tested and validated in practice

Characterizing and evaluating the quality of software process modeling language: Comparison of ten representative model-based languages

Software organizations are very conscious that deployments of well-defined software processes improve software product development and its quality. Over last decade, many Software Process Modeling Languages (SPMLs) have been proposed to describe and manage software processes. However, each one presents advantages and disadvantages. The main challenge for an organization is to choose the best and most suitable SPML to meet its requirements. This paper proposes a Quality Model (QM) which has been defined conforms to QuEF (Quality Evaluation Framework). This QM allows to compare model-based SPMLs and it could be used by organizations to choose the most useful model-based SPML for their particular requirements. This paper also instances our QM to evaluate and compare 10 representative SPMLs of the various alternative approaches (metamodel-level approaches; SPML based on UML and approaches based on standards). Finally, this paper concludes there are many model-based proposals for SPM, but it is very difficult to establish with could be the commitment to follow. Some non-considered aspects until now have been identified (e.g., validation within enterprise environments, friendly support tools, mechanisms to carry out continuous improvement, mechanisms to establish business rules and elements for software process orchestrating).Ministerio de Economía y Competitividad TIN2016-76956-C3-2-R (POLOLAS

Improving requirements engineering by artefact orientation

The importance of continuously improving requirements engineering (RE) has been recognised for many years. Similar to available software process improvement approaches, most RE improvement approaches focus on a normative and solution-driven assessment of companies rather than on a problem-driven RE improvement. The approaches dictate the implementation of a one-size-fits-all reference model without doing a proper problem investigation first, whereas the notion of quality factually depends on whether RE achieves company-specific goals. The approaches furthermore propagate process areas and methods, without proper awareness of the quality in the created artefacts on which the quality of many development phases rely. Little knowledge exists about how to conduct a problem-driven RE improvement that gives attention to the improvement of the artefacts. A promising solution is to start an improvement with an empirical investigation of the RE stakeholders, goals, and artefacts in the company to identify problems while abstracting from inherently complex processes. The RE improvement is then defined and implemented in joint action research workshops with the stakeholders to validate potential solutions while again concentrating on the artefacts. In this paper, we contribute an artefact-based, problem-driven RE improvement approach that emerged from a series of completed RE improvements. We discuss lessons learnt and present first result from an ongoing empirical evaluation at a German company. Our results suggest that our approach supports process engineers in a problem-driven RE improvement, but we need deeper examination of the resulting RE company standard, which is in scope of the final evaluation

A Quality Improvement Initiative to Provide Comprehensive Diabetic Management in a Rural Midwestern Clinic

An estimated 25.8 million people in the United States (US) have diabetes (Center for Disease Control and Prevention [CDC], 2014). The 2010 Patient Protection and Affordable Care Act led to an increased focus on Quality Improvement (QI) programs to reduce healthcare associated expenditures (Patient Protection and Affordable Care Act, 2010). The purpose of this Doctor of Nursing Practice (DNP) project was to address the following clinical question: Can the comprehensive diabetic bundle of quality metrics for type 2 diabetes mellitus (T2DM) patients in a rural Midwestern clinic be improved through (1) the creation of a QI protocol, (2) increased provider and staff education regarding quality metrics, and (3) the optimal use of QI dashboard software? The DNP project utilized the Donabedian model as the conceptual model to explore the phenomenon of interest, improving the quality of diabetes management. The Promoting Action on Research Implementation in Health Services (PARiHS) framework served as the implementation model to guide the design and development of the implementation strategies to support the evidence-based interventions. The interventions designed to answer this clinical question were: (1) the utilization of the Symphony Performance Health (SPH) QI dashboard software to obtain a T2DM QI metrics baseline, (2) development of a diabetic QI protocol regarding electronic health record (EHR) documentation on diabetes management and optimal SPH QI dashboard software utilization, (3) implementation and evaluation of an educational session intervention on T2DM QI metrics, and (4) the utilization of SPH QI dashboard software to obtain the overall changes in T2DM QI metrics. As a result of these interventions, 5 of 12 (41.6%) diabetic quality metrics were improved with significance (p= 0.05). After the implementation of this DNP project, 4 out of 12 diabetic quality metrics have met the National Committee for Quality Assurance (NCQA) QI metric benchmark percentiles. Prior to this DNP project, only one of the T2DM QI metrics met the NCQA diabetic QI metric benchmark percentiles. This improvement in quality indicator metrics demonstrated the effectiveness of a continuous multicomponent QI initiative to impact care in a rural primary care practice

Defining and validating a multimodel approach for product architecture derivation and improvement

The final publication is available at Springer via http://dx.doi.org/10.1007/978-3-642-41533-3_24Software architectures are the key to achieving the non-functional requirements (NFRs) in any software project. In software product line (SPL) development, it is crucial to identify whether the NFRs for a specific product can be attained with the built-in architectural variation mechanisms of the product line architecture, or whether additional architectural transformations are required. This paper presents a multimodel approach for quality-driven product architecture derivation and improvement (QuaDAI). A controlled experiment is also presented with the objective of comparing the effectiveness, efficiency, perceived ease of use, intention to use and perceived usefulness with regard to participants using QuaDAI as opposed to the Architecture Tradeoff Analysis Method (ATAM). The results show that QuaDAI is more efficient and perceived as easier to use than ATAM, from the perspective of novice software architecture evaluators. However, the other variables were not found to be statistically significant. Further replications are needed to obtain more conclusive results.This research is supported by the MULTIPLE project (MICINN TIN2009-13838) and the Vali+D fellowship program (ACIF/2011/235).González Huerta, J.; Insfrán Pelozo, CE.; Abrahao Gonzales, SM. (2013). Defining and validating a multimodel approach for product architecture derivation and improvement. En Model-Driven Engineering Languages and Systems. Springer. 388-404. https://doi.org/10.1007/978-3-642-41533-3_24S388404Ali-Babar, M., Lago, P., Van Deursen, A.: Empirical research in software architecture: opportunities, challenges, and approaches. Empirical Software Engineering 16(5), 539–543 (2011)Ali-Babar, M., Zhu, L., Jeffery, R.: A Framework for Classifying and Comparing Software Architecture Evaluation Methods. In: 15th Australian Software Engineering Conference, Melbourne, Australia, pp. 309–318 (2004)Basili, V.R., Rombach, H.D.: The TAME project: towards improvement-oriented software environments. IEEE Transactions on Software Engineering 14(6), 758–773 (1988)Barkmeyer, E.J., Feeney, A.B., Denno, P., Flater, D.W., Libes, D.E., Steves, M.P., Wallace, E.K.: Concepts for Automating Systems Integration NISTIR 6928. National Institute of Standards and Technology, U.S. Dept. of Commerce (2003)Bosch, J.: Design and Use of Software Architectures. Adopting and Evolving Product-Line Approach. Addison-Wesley, Harlow (2000)Botterweck, G., O’Brien, L., Thiel, S.: Model-driven derivation of product architectures. In: 22th Int. Conf. on Automated Software Engineering, New York, USA, pp. 469–472 (2007)Buschmann, F., Meunier, R., Rohnert, H., Sommerlad, P., Stal, M.: Pattern-Oriented software architecture, vol. 1: A System of Patterns. Wiley (1996)Cabello, M.E., Ramos, I., Gómez, A., Limón, R.: Baseline-Oriented Modeling: An MDA Approach Based on Software Product Lines for the Expert Systems Development. In: 1st Asia Conference on Intelligent Information and Database Systems, Vietnam (2009)Carifio, J., Perla, R.J.: Ten Common Misunderstandings, Misconceptions, Persistent Myths and Urban Legends about Likert Scales and Likert Response Formats and their Antidotes. Journal of Social Sciences 3(3), 106–116 (2007)Clements, P., Northrop, L.: Software Product Lines: Practices and Patterns. Addison-Wesley, Boston (2007)Czarnecki, K., Kim, C.H.: Cardinality-based feature modeling and constraints: A progress report. In: Int. Workshop on Software Factories, San Diego-CA (2005)Datorro, J.: Convex Optimization & Euclidean Distance Geometry. Meboo Publishing (2005)Davis, F.D.: Perceived usefulness, perceived ease of use and user acceptance of information technology. MIS Quarterly 13(3), 319–340 (1989)Douglass, B.P.: Real-Time Design Patterns: Robust Scalable Architecture for Real-Time Systems. Addison-Wesley, Boston (2002)Feiler, P.H., Gluch, D.P., Hudak, J.: The Architecture Analysis & Design Language (AADL): An Introduction. Tech. Report CMU/SEI-2006-TN-011. SEI, Carnegie Mellon University (2006)Gómez, A., Ramos, I.: Cardinality-based feature modeling and model-driven engineering: Fitting them together. In: 4th Int. Workshop on Variability Modeling of Software Intensive Systems, Linz, Austria (2010)Gonzalez-Huerta, J., Insfran, E., Abrahao, S.: A Multimodel for Integrating Quality Assessment in Model-Driven Engineering. In: 8th International Conference on the Quality of Information and Communications Technology (QUATIC 2012), Lisbon, Portugal, September 3-6 (2012)Gonzalez-Huerta, J., Insfran, E., Abrahao, S., McGregor, J.D.: Non-functional Requirements in Model-Driven Software Product Line Engineering. In: 4th Int. Workshop on Non-functional System Properties in Domain Specific Modeling Languages, Insbruck, Austria (2012)Guana, V., Correal, V.: Variability quality evaluation on component-based software product lines. In: 15th Int. Software Product Line Conference, Munich, Germany, vol. 2, pp. 19.1–19.8 (2011)Insfrán, E., Abrahão, S., González-Huerta, J., McGregor, J.D., Ramos, I.: A Multimodeling Approach for Quality-Driven Architecture Derivation. In: 21st Int. Conf. on Information Systems Development (ISD 2012), Prato, Italy (2012)ISO/IEC 25000:2005, Software Engineering. Software product Quality Requirements and Evaluation SQuaRE (2005)Kazman, R., Klein, M., Clements, P.: ATAM: Method for Architecture Evaluation (CMU/SEI-2000-TR-004, ADA382629). Software Engineering Institute, Carnegie Mellon University, Pittsburgh (2000), http://www.sei.cmu.edu/publications/documents/00.reports/00tr004.htmlKim, T., Ko, I., Kang, S., Lee, D.: Extending ATAM to assess product line architecture. In: 8th IEEE Int. Conference on Computer and Information Technology, Sydney, Australia, pp. 790–797 (2008)Kitchenham, B.A., Pfleeger, S.L., Hoaglin, D.C., Rosenber, J.: Preliminary Guidelines for Empirical Research in Software Engineering. IEEE Transactions on Software Engineering 28(8) (2002)Kruchten, P.B.: The Rational Unified Process: An Introduction. Addison-Wesley (1999)Martensson, F.: Software Architecture Quality Evaluation. Approaches in an Industrial Context. Ph. D. thesis, Blekinge Institute of Technology, Karlskrona, Sweden (2006)Maxwell, K.: Applied Statistics for Software Managers. Software Quality Institute Series. Prentice-Hall (2002)Olumofin, F.G., Mišic, V.B.: A holistic architecture assessment method for software product lines. Information and Software Technology 49, 309–323 (2007)Perovich, D., Rossel, P.O., Bastarrica, M.C.: Feature model to product architectures: Applying MDE to Software Product Lines. In: IEEE/IFIP & European Conference on Software Architecture, Helsinki, Findland, pp. 201–210 (2009)Robertson, S., Robertson, J.: Mastering the requirements process. ACM Press, New York (1999)Roos-Frantz, F., Benavides, D., Ruiz-Cortés, A., Heuer, A., Lauenroth, K.: Quality-aware analysis in product line engineering with the orthogonal variability model. Software Quality Journal (2011), doi:10.1007/s11219-011-9156-5Saaty, T.L.: The Analytical Hierarchical Process. McGraw- Hill, New York (1990)Taher, L., Khatib, H.E., Basha, R.: A framework and QoS matchmaking algorithm for dynamic web services selection. In: 2nd Int. Conference on Innovations in Information Technology, Dubai, UAE (2005)Wohlin, C., Runeson, P., Host, M., Ohlsson, M.C., Regnell, B., Weslen, A.: Experimentation in Software Engineering - An Introduction. Kluwer (2000

Quality-oriented Move Method Refactoring

International audienceRestructuring is an important activity to improve software internal structure. Even though there are many restructuring approaches, very few consider the refactoring impact on the software quality. In this paper, we propose an semi-automatic software restructuring approach based on quality attributes. We rely on the measurements of the Quality Model for Object Oriented Design (QMOOD) to recommend Move Method refactorings that improve software quality. In a nutshell , given a software system S, our approach recommends a sequence of refactorings R1, R2,. .. , Rn that result in system versions S1, S2,. .. , Sn, where quality(Si+1) > quality(Si). We empirically calibrated our approach to find the best criteria to measure the improvement of quality. In our preliminary evaluation on three open-source systems, our approach achieved an average recall of 57%

Usability Inspection in Model-Driven Web Development: Empirical Validation in WebML

There is a lack of empirically validated usability evaluation methods that can be applied to models in model-driven Web development. Evaluation of these models allows an early detection of usability problems perceived by the end-user. This motivated us to propose WUEP, a usability inspection method which can be integrated into different model-driven Web development processes. We previously demonstrated how WUEP can effectively be used when following the Object-Oriented Hypermedia method. In order to provide evidences about WUEP’s generalizability, this paper presents the operationalization and empirical validation of WUEP into another well-known method: WebML. The effectiveness, efficiency, perceived ease of use, and satisfaction of WUEP were evaluated in comparison to Heuristic Evaluation (HE) from the viewpoint of novice inspectors. The results show that WUEP was more effective and efficient than HE when detecting usability problems on models. Also, inspectors were satisfied when applying WUEP, and found it easier to use than HE.Fernández Martínez, A.; Abrahao Gonzales, SM.; Insfrán Pelozo, CE.; Matera, M. (2013). Usability Inspection in Model-Driven Web Development: Empirical Validation in WebML. Lecture Notes in Computer Science. 8107:740-756. doi:10.1007/978-3-642-41533-3_457407568107Abrahão, S., Iborra, E., Vanderdonckt, J.: Usability Evaluation of User Interfaces Generated with a Model-Driven Architecture Tool. In: Maturing Usability: Quality in Software, Interaction and Value, pp. 3–32. Springer (2007)Atterer, R., Schmidt, A.: Adding Usability to Web Engineering Models and Tools. In: Lowe, D.G., Gaedke, M. (eds.) ICWE 2005. LNCS, vol. 3579, pp. 36–41. Springer, Heidelberg (2005)Basili, V., Rombach, H.: The TAME Project: Towards Improvement-Oriented Software Environments. IEEE Transactions on Software Engineering 14(6), 758–773 (1988)Briand, L., Labiche, Y., Di Penta, M., Yan-Bondoc, H.: An experimental investigation of formality in UML-based development. IEEE TSE 31(10), 833–849 (2005)Carifio, J., Perla, R.: Ten Common Misunderstandings, Misconceptions, Persistent Myths and Urban Legends about Likert Scales and Likert Response Formats and their Antidotes. Journal of Social Sciences 3(3), 106–116 (2007)Ceri, S., Fraternali, P., Bongio, A.: Web modeling language (WebML): a modeling language for designing Web sites. In: 9th International World Wide Web Conference, pp. 137–157 (2000)Ceri, S., Fraternali, P., Acerbis, R., Bongio, A., Butti, S., Ciapessoni, F., Conserva, C., Elli, R., Greppi, C., Tagliasacchi, M., Toffetti, G.: Architectural issues and solutions in the development of data-intensive Web applications. In: Proceedings of the 1st Biennial Conference on Innovative Data Systems Research, Asilomar, CA (2003)Conte, T., Massollar, J., Mendes, E., Travassos, G.H.: Usability Evaluation Based on Web Design Perspectives. In: Proceedings of the International Symposium on Empirical Software Engineering and Measurement (ESEM 2007), pp. 146–155 (2007)Fernandez, A., Insfran, E., Abrahão, S.: Usability evaluation methods for the Web: a systematic mapping study. Information and Software Technology 53, 789–817 (2011)Fernandez, A., Abrahão, S., Insfran, E.: A Web usability evaluation process for model-driven Web development. In: Mouratidis, H., Rolland, C. (eds.) CAiSE 2011. LNCS, vol. 6741, pp. 108–122. Springer, Heidelberg (2011)Fernandez, A., Abrahão, S., Insfran, E., Matera, M.: Further Analysis on the Validation of a Usability Inspection Method for Model-Driven Web Development. In: 6th International Symposium on Empirical Software Engineering and Measurement (ESEM 2012), pp. 153–156 (2012)Fernandez, A., Abrahão, S., Insfran, E.: Empirical Validation of a Usability Inspection Method for Model-Driven Web Development. Journal of Systems and Software 86, 161–186 (2013)Fraternali, P., Matera, M., Maurino, A.: WQA: an XSL Framework for Analyzing the Quality of Web Applications. In: Proceedings of IWWOST 2002 - ECOOP 2002 Workshop, Malaga, Spain (2002)Hornbæk, K.: Dogmas in the assessment of usability evaluation methods. Behaviour & Information Technology 29(1), 97–111 (2010)Hwang, W., Salvendy, G.: Number of people required for usability evaluation: the 10±2 rule. Communications of the ACM 53(5), 130–113 (2010)International Organization for Standardization: ISO/IEC 25000, Software Engineering – Software Product Quality Requirements and Evaluation (SQuaRE) – Guide to SQuaRE (2005)Juristo, N., Moreno, A.M.: Basics of Software Engineering Experimentation. Kluwer Academic Publishers (2001)Juristo, N., Moreno, A., Sanchez-Segura, M.I.: Guidelines for eliciting usability functionalities. IEEE Transactions on Software Engineering 33(11), 744–758 (2007)Matera, M., Costabile, M.F., Garzotto, F., Paolini, P.: SUE inspection: an effective method for systematic usability evaluation of hypermedia. IEEE Transactions on Systems, Man, and Cybernetics, Part A 32(1), 93–103 (2002)Matera, M., Rizzo, F., Carughi, G.: Web Usability: Principles and Evaluation Methods. In: Web Engineering, pp. 143–180. Springer (2006)Maxwell, K.: Applied Statistics for Software Managers. Software Quality Institute Series. Prentice Hall (2002)Molina, F., Toval, A.: Integrating usability requirements that can be evaluated in design time into Model Driven Engineering of Web Information Systems. Advances in Engineering Software 40(12), 1306–1317 (2009)Moreno, N., Vallecillo, A.: Towards interoperable Web engineering methods. Journal of the American Society for Information Science and Technolog 59(7), 1073–1092 (2008)Neuwirth, C.M., Regli, S.H.: IEEE Internet Computing Special Issue on Usability and the Web 6(2) (2002)Nielsen, J.: Heuristic evaluation. In: Usability Inspection Methods. John Wiley & Sons, NY (1994)Offutt, J.: Quality attributes of Web software applications. IEEE Software: Special Issue on Software Engineering of Internet Software, 25–32 (2002)Panach, I., Condori, N., Valverde, F., Aquino, N., Pastor, O.: Understandability measurement in an early usability evaluation for MDD. In: International Symposium on Empirical Software Engineering (ESEM 2008), pp. 354–356 (2008)Webratio. Success stories, Online article, http://www.webratio.com/portal/content/en/success-storiesWohlin, C., Runeson, P., Host, M., Ohlsson, M.C., Regnell, B., Weslen, A.: Experimentation in Software Engineering - An Introduction. Kluwer (2000

Evaluating Quality in Agile Developments. A first validation experience with NEA Software SMEs

To obtain products of high quality software it is necessary to carry out good processes management in which measurement is a key factor. Therefore, companies should focus on continuous improvement cycles that integrate both the development process and the product obtained, to increase quality in both aspects. This cycle of improvement involves the adoption of a quality model appropriate to the characteristics of the company and a methodology that guides the software development cycle. In this sense, the agile philosophy proves to be the most suitable approach for the current development environments, and they are positioned as an alternative to the development processes with high cost in documentation and excessively long processes. This paper presents the results obtained by automation of QuAM Model for the quality evaluation of agile projects on actual production environments and the subsequent analysis based on these projects.XIV Workshop de Ingeniería de Software (WIS).Red de Universidades con Carreras en Informática (RedUNCI