Applying Code Coverage Approach To An Infinite Failure Software Reliability Model

An approach to software reliability modeling based on code coverage is used to derive the Infinite Failure software reliability Model Based on Code Coverage - IFMBC. Our aim was to verify the soundness of the approach under different assumptions. The IFMBC was assessed with test data from a real application, making use of the following structural testing criteria: all-nodes, all-edges, and potential-uses - a data-flow based family of criteria. The IFMBC was shown to be as good as the Geometric Model - GEO, found to be the best traditional time-based model that fits the data. Results from the analysis also show that the IFMBC is as good as the BMBC - Binomial software reliability Model Based on Coverage - a model previously derived using the code coverage approach, indicating it to be effective under different modeling assumptions. © 2009 IEEE.216226Musa, J.D., Ianino, A., Okumoto, K., (1987) Software Reliability - Measurement, Prediction, Application, , McGraw-Hill, New YorkHudson, A., Program Errors as a Birth and Death Process (1967) Technical Report SP - 3011, , Santa Monica, Cal, Systems Development CorporationJelinski, Z., Moranda, P.B., Software Reliability Research (1972) Proceedings of the Statistical Methods for the Evaluation of Computer System Performance, pp. 465-484. , Academic PressShooman, M.L., Probabilistic Models for Software Reliability Prediction (1972) Statistical Computer Performance Evaluation, pp. 485-502. , W. Freiberg, Ed. New York: Academic PressLyu, M.R., Handbook of Software Reliability Engineering (1996) IEEE Computer Society Press, , Los Alamitos, CAPrediction and Measurement of Software Reliability (1990) Dependability of Critical Computer Systems 3, , Bishop, P, ed, Elsevier Science Publisher, LondonButler, R.W., Finelli, G.B., The Unfeasibility of Experimental Quantification of Life-critical Software Reliability (1991) Proceedings of the ACM SIGSOFT '91 Conference on Software for Critical Systems, pp. 66-76. , ACM Press, DecemberHamlet, D., Are We testing for True Reliability (1992) IEEE Software, , IEEE Computer Society, JulyLittlewood, B., String, L., Validation of Ultrahigh Dependability for Software-based Systems (1993) Communications of the ACM, , ACM, JanVaradan, G.S., Trends in Reliability and Test Strategies (1995) IEEE Software, p. 10. , IEEE Computer Society, MayChen, M., Lyu, M.R., Wong, W.E., An Empirical Study of the Correlation between Code Coverage and Reliability Estimation (1996) Proceedings of the Third International Software Metrics Symposium, , Berlin, Germany, March 25-26Chen, M., Tools and Techniques for Testing Based Software Reliability Estimation, (1994), Ph. D. Dissertation, Purdue University, West Lafayette, Indiana, AugustChen, M., Mathur, A.P., Rego, V.J., Effect of Testing Technique on Software Reliability Estimates Obtained Using A Time-Domain Model (1995) IEEE Transactions on Reliability, pp. 97-103. , IEEE, MarchRamsey, J., Basili, V.R., Analyzing the Test Process Using Structural Coverage (1985) Proceedings ICSE, pp. 306-312. , 85, ppGarg, P., Investigating Coverage - Reliability Relationship and Sensitivity of Reliability to Errors in Operational Profile (1994), Technical Report, Department of Computer Sciences, Purdue University, West Lafayette, MayVeevers, A., Marshall, A., A Relationship Between Software Coverage Metrics and Reliability (1994) Software Testing, Verification and Reliability, pp. 3-8Malaiya, Y.K., Li, N., Bieman, J., Karcich, R., Skibe, B., The Relationship Between Test Coverage and Reliability (1994) Proceedings of the Fifth International Symposium on Software Reliability Engineering, pp. 186-195. , Monterey, CA, November 6-9Jalote, P., Muralidhara, Y.R., A Coverage Based Model for Software Reliability Estimation (1994) Proceedings of the First International Conference on Software Testing, Reliability and Quality Assurance, pp. 6-10. , New Delhi, ÍndiaHorgan, J.R., Mathur, A.P., Pasquini, A., Rego, V.J., Perils of Software Reliability Modeling (1995), Technical Report SERC-TR-160-P, Purdue UniversityMalaiya, Y.K., Li, N., Bieman, J., Karcich, R., Skibbe, B., Software Test Coverage and Reliability (1996), Technical Report CS-96-128, Colorado State UniversityGokhale, S.S., Philip, T., Marinos, P.N., Trivedi, K.S., Unification of Finite Failure Non-Homogeneous Poisson Process Models through Test Coverage (1996) Proceedings of The Seventh International Symposium on Software Reliability Engineering, pp. 289-299. , White Plains, NY, USAKrishnamurthy, S., Mathur, A.P., On Predicting Reliability of Modules Using Code Coverage (1996) Proceedings of the Conference of the Centre for Advanced Studies on Collaborative Research, pp. 1-12. , Toronto, CanadaChen, M., Lyu, M.R., Wong, W.E., Incorporating Code Coverage in the Reliability Estimation for Fault-Tolerant Software (1997) Proceedings of the Symposium on Reliable Distributed Systems, , Durham, NCChen, M., Lyu, M.R., Wong, W.E., Effect of Code Coverage on Software Reliability Measurement (2001) IEEE Transactions on Reliability, pp. 165-170Ye, R., Malaiya, Y.K., Relationship Between Test Effectiveness and Coverage (2002) Proceedings of the 13th International Symposium on Software Reliability Engineering - ISSREMalaiya, Y.K., Li, N., Bieman, J., Karcich, R., Software Reliability Growth with Test Coverage (2002) IEEE Transactions on Reliability, pp. 420-426Grottke, M., A Vector Markov Model for Structural Coverage Growth and the Number of Failure Occurrences (2002) Proceedings of the 13th International Symposium on Software Reliability Engineering - ISSREPham, H., Zhang, X., NHPP Software Reliability and Cost Models with Testing Coverage (2003) European Journal of Operational Research, 145, pp. 443-454Li, M., Wei, Y., Desovski, D., Nejad, H., Ghose, S., Cukic, B., Smidts, C., Validation of a Methodology for Assessing Software Reliability (2004) Proceedings of the 15th International Symposium on Software Reliability Engineering, pp. 66-76. , Saint-Malo, FranceFrate, F.D., Garg, P., Mathur, A.P., Pasquini, A., Experiments to Investigate the Correlation Between Code Coverage and Software Reliability (1995) SERC-TR-162-P, Software Engineering Research Center, , Purdue University, West Lafayette, Indiana 47907, AprilCrespo, A.N., Pasquini, A., Jino, M., Madonado, J.C., Cobertura dos Critérios Potenciais-usos e Confiabilidade do Software (1997) Anais do XI Simpósio Brasileiro de Engenharia de Software, , Fortaleza, in PortugueseCrespo, A.N., Pasquini, A., Jino, M., Madonado, J.C., A Binomial Software Reliability Model Based on Coverage of Structural Testing Criteria (2000) Anais do XI Simpósio Brasileiro de Engenharia de Software, , João Pessoa, PBCrespo, A.N., Pasquini, A., Jino, M., Madonado, J.C., A Binomial Software Reliability Model Based on Coverage of Structural Testing Criteria (2008) Empirical Software Engineering, Springer Netherlands, 13 (2), pp. 185-209. , AprilCai, K.Y., Towards a conceptual framework of software run reliability modeling (2000) Information Sciences, 126, pp. 137-163Rapps, S., Weyuker, E.J., Selecting Software Test Data Using Data Flow Information (1985) IEEE Transactions on Software Engineering, pp. 367-375Maldonado, J.C., Chaim, M.L., Jino, M., Bridging the Gap in the Presence of Infeasible Paths: Potential Uses Testing Criteria (1992) XII International Conference of the SCCC, pp. 323-340. , Sociedad Chilena de Ciencia de la Computacion, Santiago, ChileFrankl, P.G., Weyuker, E.J., An Applicable Family of Data Flow Testing Criteria (1988) IEEE Transactions on Software Engineering, pp. 1483-1498Crespo, A.N., Modelos de Confiabilidade de Software Baseados em Cobertura de Critérios Estruturais de Teste (1997) Tese de Doutorado, , DCA/FEEC/UNICAMP, Campinas, SP, in PortuguesePasquini, A., Crespo, A.N., Matrella, P., Sensitivity of Reliability Growth Models to Operational Profile Errors vs Testing Accuracy (1996) IEEE Transactions on Reliability, pp. 531-540Chaim, M.L., (1991) Uma Ferramenta para Suporte ao Teste Estrutural de Programas Baseado em Análise de Fluxo de Dados, , Dissertação de Mestrado, DCA/FEE/UNICAMP, Campinas, SP, in PortugueseNelson, F., Estimating Software Reliability From Test Data (1978) Microelectronics and Reliability, pp. 67-73Farr, W.H., Smith, O.D., Statistical Modeling and Estimation of Reliability Functions for Software (SMERFS) User's Guide (1993) Naval Surface Warfare Center, , VirginiaSiegel, S., (1956) Nonparametric Statistics for the Behavioral Sciences, , McGraw Hill Kogakush