Research on a Defect-driven Quantitative Management Method of Software Process

随着软件应用范围的不断扩大和复杂程度的不断提高，软件开发过程越来越难以控制，软件质量也越来越难以保障。质量管理的思想和理念，已经从单纯的以面向软件产品的检验为主要手段的质量控制，发展到更加成熟、更加主动地对软件产品生产过程进行管理的质量保障。 作为高成熟度软件过程的特征，量化过程管理逐渐被软件组织接受并实施。通过实施量化管理，能够刻画项目或过程目标的满足程度，找到造成过程或产品重大偏差的根本原因。然而，在量化过程管理实施期间，软件组织面对不同的软件开发过程、众多的过程性能度量指标、复杂的统计分析方法，既要考虑量化管理方法的合理性和复杂程度，又要权衡量化管理的实施成本，这使得实施有效的量化过程管理充满挑战。本文以缺陷数据为中心，提出了一种缺陷驱动的量化过程管理框架，以及基于该框架的两个量化管理方法，支持软件组织收集量化过程管理所需数据，建立过程性能基线和过程性能模型，量化管理软件项目。该框架适合迭代、瀑布等不同的开发方法，支持项目全生命周期的量化管理。 本文主要贡献包括： 提出了一种缺陷驱动的量化过程管理框架(Defect-driven Quantitative Process Management framework, DefQPM)。量化管理中，保障软件质量是核心。质量和缺陷密切相关，软件开发过程中各类工程活动（如：需求、设计、编码、测试等）都伴随着缺陷的注入、排除和遗留。DefQPM框架以缺陷数据作为量化管理的出发点，自底向上的通过数据层、模型层、使用层来指导软件组织分析过程性能，识别度量指标间的相关性，建立符合自身情况的过程性能基线和过程性能模型，有效的实施量化过程管理。DefQPM框架给出了实施量化管理的过程和机制。基于DefQPM框架，可以建立针对特定应用场景的量化管理方法，以及针对特定软件组织的量化管理解决方案。 提出了一种基于DefQPM的迭代项目量化管理方法(process performance Baseline based Defect-driven iteration management, BiDefect)。迭代开发方法由于其灵活性和管理需求变更的能力，得到了广泛应用。然而，如何对迭代项目实施量化管理依然充满挑战。迭代项目中，各种活动多次并行执行，难以找到合适的控制点，也缺乏针对迭代项目的度量指标及分析方法。基于DefQPM框架，本文研究了迭代开发项目典型的量化管理需要（例如：通过控制每次迭代工作产品的质量来保障最终交付软件产品的质量），提出了一种针对迭代项目的量化管理方法，解决了量化管理迭代项目的几个主要挑战。该方法关注缺陷的注入、排除、遗留情况，指导项目策划期间建立整体估算和度量，在项目执行期间评价软件过程执行情况及软件产品的质量，及时识别异常并采取纠正措施，进而为项目后续工作中成本、进度、质量等方面提供估算、控制方面的指导。 提出了一种基于DefQPM的测试过程量化管理方法(Quantitatively Managing Testing process, TestQM)。测试是重要的质量控制活动，对于高成熟度软件组织来说也是需要进行量化管理的活动。缺陷检测和缺陷修复是测试过程的两类主要活动，需要不同技能的人员执行。目前流行的软件估算方法多是将缺陷检测和缺陷修复的工作量和进度统一纳入测试活动中进行估算和管理，不够准确。基于DefQPM框架，本文提出了一种专门针对测试过程的量化管理方法。该方法关注缺陷按注入阶段分布情况，缺陷与修复工作量的相关性，以及缺陷与修复进度的相关性，指导在早期项目建立测试过程的估算，在测试过程中根据缺陷按注入阶段分布情况调整缺陷修复工作量和进度，使得测试过程受控。同时，介绍了TestQM针对Web应用开发项目的经验模型。 最后，介绍了上述量化管理方法在国内软件组织中的应用，包括BiDefect方法在迭代开发项目中的应用，以及TestQM方法在Web应用开发项目中的应用。软件组织实施量化过程管理前后的过程性能变化表明，应用本文方法能够对项目进行有效的估算、度量、重新估算和控制，进而提高产品质量，改善客户满意度。Software is being heavily used in various areas and getting more complicated. Therefore, managing the development of software is much harder than before, so is ensuring the software quality. The concept and understanding of quality management has been changed from the quality control through verification of software product to the quality assurance of the entire development process. Quantitative management is among the advanced features of high maturity process, and has been adopted and deployed widely by software organizations. It provides insights on the degree of goal fulfillment and root causes of significant process/product deviation. However, how to select processes under quantitative management, how to identify process performance measures and how to maximize return of investment through the implementation of quantitative management remain challenging issues. In this thesis, based on analysis of defect related data, we propose a defect-driven quantitative process management framework and two quantitative management methods based on the framework, to support constructing data samples, establishing process performance baselines and models with appropriate statistical techniques. The framework can be deployed in different development processes (e.g., waterfall-based and iterative development) through the whole lifecycle. The main contributions of the thesis are summarized below. A Defect-driven Quantitative Process Management framework (DefQPM) is propoesed. Quality control is the key to quantitative management. Defect is an important indicator for software quality. Defects are introduced in software development activities (e.g., requirements, design, coding and testing), and detected and removed by verification and validation activities. The structure of the DefQPM framework is bottom-up including data layer, model layer and application layer. Based on the structure, the DefQPM framework provides supports on analyzing process performance, identifying correlation between measures, establishing process performance baselines and models, and managing software projects quantitatively. The framework provides the process and mechanism for implementing quantitative process management. Based on the framework, quantitative management method for specific application and quantitative management solution for specific software organization can be established. A DefQPM-based process performance Baseline based Defect-driven iteration management (BiDefect) method for iterative development is presented. Iterative development methodology has been widely adopted in recent years since it is flexible and capable of dealing with requirement volatility. However, how to quantitatively manage iterative projects remains a challenging issue. There is lack of quantitative management support for iterative projects due to the difficulty in selecting appropriate control points and measures to collect and analyze data. Based on the DefQPM framework, we study the general needs of quantitative management in iterative development (e.g., controlling the quality of iterative development projects through controlling the quality of each iterations release and the final product), and present a quantitative management method for iterative development, named BiDefect. The method aims to solve the main challenges of quantitatively managing iterative development. The BiDefect method focuses on defect injection and removal, and it is helpful in establishing initial estimation at the early stage of project, as well as performing quantitative control and re-estimation according to actual performance data of the project. A DefQPM-based method on Quantitatively Managing Testing process (TestQM) is proposed. Testing is an important method of quality control. It is also an important process that needs to be managed quantitatively for high maturity organizations. Detecting and fixing defects are key activities in a testing process, which require two kinds of skill sets. Unfortunately, many current leading software estimation methods mainly allocate testing effort proportionally among various activities. Both efforts on detecting and fixing defects are simply counted into software testing process/phase and cannot be estimated and managed satisfactorily. Based on the DefQPM framework, we present a quantitative management method for testing process, named TestQM. The method focuses on defect injection distribution, as well as correlation between defects and fixing effort/schedule. The method is helpful in estimating and controlling defects, effort and schedule of the testing process. Furthermore, we provide some empirical results on establishing a quantitative management model for testing process in Web-based system development projects by using the TestQM method. The above research results have been successfully applied in Chinese software organizations, including applying the BiDefect method in iterative development and deploying the TestQM method in Web-based system development projects. The evaluation of implementing quantitative management shows that, the BiDefect and TestQM methods are helpful in initial estimating, analyzing, re-estimating, and controlling projects. Through the application of the methods, customer satisfaction of the organization has been promoted together with the improvement of product quality

applying statistical process control to monitor software testing process

统计过程控制(SPC)是通过使用控制图来制定过程决策和预测过程行为的一种质量控制方法.SPC的方法用于软件过程,可以通过描述过程行为来监控过程的稳定性.讨论了将SPC应用于软件测试过程,针对测试过程中所度量的不同分布形式的数据而采用不同计算方式应用SPC的控制图,然后根据控制图判断测试过程是否稳定,并分析可能存在的可归属原因

software reliability growth model considering defect correlation

非齐次泊松过程类软件可靠性增长模型(NHPP-SRGMs)是评价软件产品可靠性指标的有效工具,但大多数该类模型都未考虑软件缺陷关联这一测试过程中普遍存在的现象.该文在考虑软件缺陷关联关系的基础上对缺陷进行分类,提出一个改进的NHPP类软件可靠性增长模型.在一组失效数据上的实验分析表明,改进的模型具有较好的拟合效果和预测能力