Uso de riscos na validação de sistemas baseados em componentes

Orientadores: Eliane Martins, Henrique Santos do Carmo MadeiraTese (doutorado) - Universidade Estadual de Campinas, Instituto de ComputaçãoResumo: A sociedade moderna está cada vez mais dependente dos serviços prestados pelos computadores e, conseqüentemente, dependente do software que está sendo executado para prover estes serviços. Considerando a tendência crescente do desenvolvimento de produtos de software utilizando componentes reutilizáveis, a dependabilidade do software, ou seja, a segurança de que o software irá funcionar adequadamente, recai na dependabilidade dos componentes que são integrados. Os componentes são normalmente adquiridos de terceiros ou produzidos por outras equipes de desenvolvimento. Dessa forma, os critérios utilizados na fase de testes dos componentes dificilmente estão disponíveis. A falta desta informação aliada ao fato de se estar utilizando um componente que não foi produzido para o sistema e o ambiente computacional específico faz com que a reutilização de componentes apresente um risco para o sistema que os integra. Estudos tradicionais do risco de um componente de software definem dois fatores que caracteriza o risco, a probabilidade de existir uma falha no componente e o impacto que isso causa no sistema computacional. Este trabalho propõe o uso da análise do risco para selecionar pontos de injeção e monitoração para campanhas de injeção de falhas. Também propõe uma abordagem experimental para a avaliação do risco de um componente para um sistema. Para se estimar a probabilidade de existir uma falha no componente, métricas de software foram combinadas num modelo estatístico. O impacto da manifestação de uma falha no sistema foi estimado experimentalmente utilizando a injeção de falhas. Considerando esta abordagem, a avaliação do risco se torna genérica e repetível embasando-se em medidas bem definidas. Dessa forma, a metodologia pode ser utilizada como um benchmark de componentes quanto ao risco e pode ser utilizada quando é preciso escolher o melhor componente para um sistema computacional, entre os vários componentes que provêem a mesma funcionalidade. Os resultados obtidos na aplicação desta abordagem em estudos de casos nos permitiram escolher o melhor componente, considerando diversos objetivos e necessidades dos usuáriosAbstract: Today's societies have become increasingly dependent on information services. A corollary is that we have also become increasingly dependent on computer software products that provide such services. The increasing tendency of software development to employ reusable components means that software dependability has become even more reliant on the dependability of integrated components. Components are usually acquired from third parties or developed by unknown development teams. In this way, the criteria employed in the testing phase of components-based systems are hardly ever been available. This lack of information, coupled with the use of components that are not specifically developed for a particular system and computational environment, makes components reutilization risky for the integrating system. Traditional studies on the risk of software components suggest that two aspects must be considered when risk assessment tests are performed, namely the probability of residual fault in software component, and the probability of such fault activation and impact on the computational system. The present work proposes the use of risk analysis to select the injection and monitoring points for fault injection campaigns. It also proposes an experimental approach to evaluate the risk a particular component may represent to a system. In order to determine the probability of a residual fault in the component, software metrics are combined in a statistical mode!. The impact of fault activation is estimated using fault injection. Through this experimental approach, risk evaluation becomes replicable and buttressed on well-defined measurements. In this way, the methodology can be used as a components' risk benchmark, and can be employed when it is necessary to choose the most suitable among several functionally-similar components for a particular computational system. The results obtained in the application of this approach to specific case studies allowed us to choose the best component in each case, without jeopardizing the diverse objectives and needs of their usersDoutoradoDoutor em Ciência da Computaçã

Using Stratified Sampling For Fault Injection

In a previous work we validated an ODBMS component injecting errors in the application's interface. The aim was to observe the robustness of the component when the application that interacted with it failed. In this work we tackle the injection of errors directly into the interfaces among the target component's classes. As the component under test has several classes, we use stratified sampling to reduce the amount of injections without losing the ability to detect faults. Strata are defined based on a complexity metric, Weighted Methods in a Class - WMC. Experiments show that this metric alone is not sufficient to select strata for testing purposes. © Springer-Verlag Berlin Heidelberg 2005.3747 LNCS919Bach, J., Heuristic risk-based testing (1999) Software Testing and Quality Engineering MagazineBeydeda, S., Volker, G., State of the art in testing components (2003) Proc. of the International Conference on Quality SoftwareCarey, M.J., DeWitt, D.J., Naughton, J.F., (1994) The OO7 Benchmark, , http://www.columbia.edu/, recovered February (2005)Chidamber, K., (1994) Principal Components of Orthogonal Object-oriented Metrics, , http://satc.gsfc.nasa.gov, recovered November (2004)De Millo, R.A., Li, T., Mathur, A.P., (1994) Architecture of TAMER: A Tool for Dependability Analysis of Distributed Fault-tolerant Systems, , Purdue UniversityFabre, J.-C., Rodriguez, M., Arlat, J., Sizum, J.-M., Building dependable COTS microkernel-based systems using MAFALDA (2000) Proc. of 2000 Pacific Rim International Symposium on Dependable Computing - PRDC'00, , Los Angeles, USAHsueh, M.C., Tsai, T., Iyer, R., Fault injection techniques and tools (1997) IEEE Computer, pp. 75-82Koopman, P., Siewiorek, D., DeVale, K., DeVale, J., Fernsler, K., Guttendorf, D., Kropp, N., Shi, Y., (2003) Ballista Project : COTS Software Robustness Testing, , http://www.ece.cmu.edu/~koopman/ballista/, Carnegie Mellon UniversityMartins, E., Rubira, C.M.F., Leme, N.G.M., Jaca: A reflective fault injection tool based on patterns (2002) Proc. of the 2002 Intern Conference on Dependable Systems & Networks, 23 (267), pp. 483-487. , Washington D.C. USAMoraes, R., Martins, E., A strategy for validating an ODBMS component using a high-level software fault injection tool (2003) Proc. of the First Latin-American Symposium, LADC 2003, pp. 56-68. , São Paulo, BrazilMoraes, R., Martins, E., Mendes, N., Fault injection approach based on dependence analysis (2005) Proc. of the First International Workshop on Testing and Quality Assurance for Component-based Systems - TQACBS(2004) Object Oriented Database Management System, , www.ozone-db.org/Podgurski, A., Yang, C., Partition testing, stratified sampling and cluster analysis (1993) Proc.of the 1st ACM SIGSOFT Symposium on Foundations of Software Engineering, pp. 169-181. , Los Angeles, USAPressman, R.S., (1997) Software Engineering A Practitioner Approach, 4 th Edition, , Mc Graw HillRosenberg, L., Stapko, R., Gallo, A., Risk-based object oriented testing (2000) Proc. 13 th International Software/Internet Quality Week (QW2000), , San Francisco, California USA(2005) Transaction Processing Performance Council "TPC-C - Benchmarks", , http://www.tpc.org/tpcc/default.aspTriola, M.F., (1999) Introcução A Estatística, 7th Edition, , LTC Editor, Rio de Janeiro, (in Portuguese)Voas, J., McGraw, G., (1998) Software Fault Injection: Inoculating Programs Against Errors, , John Wiley & Sons, New York, EUAVoas, J.M., Charron, F., McGraw, G., Miller, K., Friedman, M., Predicting how badly good software can behave (1997) IEEE Software, pp. 73-83Voas, J., Marrying software fault injection technology results with software reliability growth models (2003) Fast Abstract ISSRE 2003, , Chillarege PressVoas, J., An approach to certifying off-the-shelf software components (1998) IEEE Computer, 31 (6), pp. 53-5