Uso de riscos na validação de sistemas baseados em componentes

Orientadores: Eliane Martins, Henrique Santos do Carmo MadeiraTese (doutorado) - Universidade Estadual de Campinas, Instituto de ComputaçãoResumo: A sociedade moderna está cada vez mais dependente dos serviços prestados pelos computadores e, conseqüentemente, dependente do software que está sendo executado para prover estes serviços. Considerando a tendência crescente do desenvolvimento de produtos de software utilizando componentes reutilizáveis, a dependabilidade do software, ou seja, a segurança de que o software irá funcionar adequadamente, recai na dependabilidade dos componentes que são integrados. Os componentes são normalmente adquiridos de terceiros ou produzidos por outras equipes de desenvolvimento. Dessa forma, os critérios utilizados na fase de testes dos componentes dificilmente estão disponíveis. A falta desta informação aliada ao fato de se estar utilizando um componente que não foi produzido para o sistema e o ambiente computacional específico faz com que a reutilização de componentes apresente um risco para o sistema que os integra. Estudos tradicionais do risco de um componente de software definem dois fatores que caracteriza o risco, a probabilidade de existir uma falha no componente e o impacto que isso causa no sistema computacional. Este trabalho propõe o uso da análise do risco para selecionar pontos de injeção e monitoração para campanhas de injeção de falhas. Também propõe uma abordagem experimental para a avaliação do risco de um componente para um sistema. Para se estimar a probabilidade de existir uma falha no componente, métricas de software foram combinadas num modelo estatístico. O impacto da manifestação de uma falha no sistema foi estimado experimentalmente utilizando a injeção de falhas. Considerando esta abordagem, a avaliação do risco se torna genérica e repetível embasando-se em medidas bem definidas. Dessa forma, a metodologia pode ser utilizada como um benchmark de componentes quanto ao risco e pode ser utilizada quando é preciso escolher o melhor componente para um sistema computacional, entre os vários componentes que provêem a mesma funcionalidade. Os resultados obtidos na aplicação desta abordagem em estudos de casos nos permitiram escolher o melhor componente, considerando diversos objetivos e necessidades dos usuáriosAbstract: Today's societies have become increasingly dependent on information services. A corollary is that we have also become increasingly dependent on computer software products that provide such services. The increasing tendency of software development to employ reusable components means that software dependability has become even more reliant on the dependability of integrated components. Components are usually acquired from third parties or developed by unknown development teams. In this way, the criteria employed in the testing phase of components-based systems are hardly ever been available. This lack of information, coupled with the use of components that are not specifically developed for a particular system and computational environment, makes components reutilization risky for the integrating system. Traditional studies on the risk of software components suggest that two aspects must be considered when risk assessment tests are performed, namely the probability of residual fault in software component, and the probability of such fault activation and impact on the computational system. The present work proposes the use of risk analysis to select the injection and monitoring points for fault injection campaigns. It also proposes an experimental approach to evaluate the risk a particular component may represent to a system. In order to determine the probability of a residual fault in the component, software metrics are combined in a statistical mode!. The impact of fault activation is estimated using fault injection. Through this experimental approach, risk evaluation becomes replicable and buttressed on well-defined measurements. In this way, the methodology can be used as a components' risk benchmark, and can be employed when it is necessary to choose the most suitable among several functionally-similar components for a particular computational system. The results obtained in the application of this approach to specific case studies allowed us to choose the best component in each case, without jeopardizing the diverse objectives and needs of their usersDoutoradoDoutor em Ciência da Computaçã

Reachability analysis of fault-tolerant protocols

Due to the increasing requirements imposed on fault-tolerant protocols, their complexity is steadily growing. Thus verification of the functionality of the fault-tolerance mechanisms is also more difficult to accomplish. In this thesis a model-based approach towards efficiently finding ``loopholes'' in the fault-tolerance properties of large protocols is provided. The contributions comprise thinning out the state space without missing behavior with respect to the validation goal through a partial ordering strategy based on single fault regions. Two algorithms for (partial) analysis are designed, implemented and evaluated: the H-RAFT algorithm is based on SDL elements constituting each transition and requires no user-knowledge. The Close-to-Failure algorithm on the other hand is purely based on user-provided information. Combination of the two algorithms is also investigated. All contributions exploit the fault-tolerant nature of the protocols. In order to compare the performances of the novel techniques to well-known algorithms, a tool has been developed to allow for easy integration of different algorithms. All contributions are thoroughly investigated through experiments summing up to several CPU-month. The results show unambiguously the advantages of the developed methods and algorithms.Durch die zunehmenden Anforderungen an fehlertolerante Protokolle steigt auch deren Komplexität zusehends. Dadurch ist es deutlich schwieriger die Funktionalität der Fehlertoleranzmechanismen zu überprüfen. In dieser Arbeit wird ein modellbasierter Ansatz vorgestellt, dessen Ziel es ist ``Lücken'' in den Fehlertoleranzeigenschaften effizient zu finden. Dazu wird ein Algorithmus entwickelt, der eine partiellen Ordnung erzeugt und es somit erlaubt den Zustandsraum zu verkleinern ohne Verhalten bezüglich der zu prüfenden Eigenschaften zu verlieren. Weiterhin werden zwei Algorithmen zur (partiellen) Analyse entworfen, implementiert und bewertet: Der H-RAFT Algorithmus basiert auf den SDL-Elementen der jeweiligen Transitionen und erfordert keinerlei weiteres Domänen-Wissen des Benutzers. Der Close-to-Failure Algorithmus hingegen ist nur von Benutzerinformationen abhängig. Kombinationen der beiden Ansätze werden ebenfalls untersucht. Für alle vorgestellten Methoden und Algorithmen wird ausgenutzt, dass es sich um fehlertolerante Protokolle handelt. Um die neuen Ansätze mit weitverbreiteten Algorithmen vergleichen zu können wird ein Werkzeug entwickelt, welches eine einfache Integration von Algorithmen ermöglicht. Die vorgestellten Techniken werden ausführlich in Experimenten mit einem Gesamtaufwand von etlichen CPU-Monaten untersucht. Die Ergebnisse dieser Experimentreihen zeigen eindeutig die Vorteile der entwickelten Algorithmen und Methoden

Injection Of Faults At Component Interfaces And Inside The Component Code: Are They Equivalent?

The injection of interface faults through API parameter corruption is a technique commonly used in experimental dependability evaluation. Although the interface faults injected by this approach can be considered as a possible consequence of actual software faults in real applications, the question of whether the typical exceptional inputs and invalid parameters used in these techniques do represent the consequences of software bugs is largely an open issue. This question may not be an issue in the context of robustness testing aimed at the identification of weaknesses in software components. However, the use of interface faults by API parameter corruption as a general approach for dependability evaluation in component-based systems requires an in depth study of interface faults and a close observation of the way internal component faults propagate to the component interfaces. In this work we present the results of experimental evaluation of realistic component-based applications developed in Java and C using the injection of interface faults by API parameter corruption and the injection of software faults inside the components by modification of the target code. The faults injected inside software components emulate typical programming errors and are based on an extensive field data study previously published. The results show the consequences of internal component faults in several operational scenarios and provide empirical evidences that interface faults and software component faults cause different impact in the system. © 2006 IEEE.5362Arlat, J., Crouzet, Y., Faultload Representativeness for Dependability Benchmarking (2002) Workshop on Dependability Benchmarking, DSN02Bieman, J., Dreilinger, D., Lin, L., Using Fault Injection to Increase Test Coverage (1996) Proc of The 7th IEEE International Symposyum on Software Reliability Engineering, ISSRE, , 96, New York, NY, USABlough, D., Torii, T., Fault-Injection-Based Testing of Fault-Tolerant Algorithms in Message Passing Parallel Computers (1997) Proc. of The 27th IEEE Int.Fault Tolerant Computer Symposium, FCTS-27, pp. 258-267. , Seattle, USACarey, M.J., DeWitt, D.J., Naughton, J.F., (1994) The OO7 Benchmark, , http://www.columbia.edu, accessed FebCarreira, J., Madeira, H., Silva, J., Xception: Software Fault Injection and Monitorintg in Processor Functional Units (1998) IEEE Trans. on Software Engineering, 24Chiba, S., Javassist - A Reflection-based ProgrammingWizard for Java (1998) proceedings of the ACM OOPSLA'98 Workshop onReflective Programming in C++ and Java, , OctChillarege, R., Orthogonal Defect Classification (1995) Handbook of Software Reliability, , Engineering, M. Lyu, Ed, IEEE Computer Society Press, McGraw-Hill, Ch. 9Christmansson, J., Chillarege, R., Generation of an Error Set that Emulates Software Faults (1996) Proc. of The 26th IEEE Fault Tolerant Computing Symp. - FCTS-26, , Sendai, JapanChristmansson, J., Killer, M., Rimén, M., An Experimental Comparison of Fault and Error Injection (1998) Proc. of The 9th Int. Symposium on Software Reliability Engineering - ISSRE, 98, pp. 369-378Dingman, C., Marshall, J., Siewiorek, D., Measuring Robustness of a Fault Tolerant Aerospace System (1995) Proc. of The 25th IEEE International Symp. on Fault Tolerant Computing - FTCS, pp. 522-527. , 95, Passadena, pp, CA, USADurães, J., Madeira, H., Emulation of Software Faults by Educated Mutations at Machine-Code Level (2002) Proc. of The Thirteenth International Symposium on Software Reliability Engineering - ISSRE'02, , Annapolis, USADurães, J., Madeira, H., Definition of Software Fault Emulation Operators: A Field Data Study (2003) Proc. of The International Conference on Dependable Systems and Networks, DSN2003, pp. 105-114. , San Francisco, USAFabre, J.-C., Salles, F., Moreno, M., Arial, J., Assessment of COTS Microkernels by Fault Injection (1999) Proc.of The 7th IFIP Working Conference on Dependable Computing for Critical Applications- DCCA, pp. 25-44. , 99, pp, San Jose, CA, USAFabre, J.C., Rodríguez, M., Arial, J., Salles, F., Sizun, J., Bulding Dependable COTS Microkernel-based Systems using MAFALDA (2000) Proc. of the 2000 Pacific Rim International Symposium on Dependable Computing PRDC'00, pp. 85-92Ghosh, A., Schmid, M., Shah, V., Testing the Robustness of Windows NT Software (1998) Proc. of the 9th IEEE International Symposium on Software Reliability Engineering - ISSRE, pp. 231-236. , 98, ppGhosh, A., Shah, V., Schmid, M., An Approach for Analyzing the Robustness of Windows NT Software (1999) Proc.of The 10th IEEE International Symposium on Software Reliability Engineering - ISSRE, , 99Hiller, M., Jhumka, A., Suri, N., An Approach for Analysing the Propagation of Data Errors in Software (2001) Int. Conf. on Dependable Systems and Networks, DSN, , Gothenburg, SwedenKoopman, P., Sung, J., Dingman, C., Siewiorek, D., Marz, T., Comparing Operating Systems Using Robustness Benchmarks (1997) Proc.of The 16th International Symposium on Reliable Distributed Systems - SRDS, pp. 72-79. , 97, Durham, NC, USA, ppKoopman, P., DeVale, J., The Exception Handling Effectiveness of POSK Operating Systems (2000) IEEE Transactions on Software Engineering, 26, pp. 837-848Koopman, P. What's Wrong With Fault Injection As A Benchmarking Tool?, in Proc. of The Internat. Conf. on Dependable Systems and Networks - DSN2002, Washington D.C, USA, 2002Kropp, N., Koopman, P., Siewiorek, D., Automated Robustness Testing of Off-the-Shelf Software Components (1998) 28th Fault Tolerant Computing Symposium, pp. 230-239Madeira, H., Vieira, M., Costa, D., On the Emulation of Software Faults by Software Fault Injection (2000) Proc. of The Int. Conf. on Dependable System and Networks, pp. DSN00. , NY, USAMartins, E.Rubira, C. M. F.Lerne N.G.M. Jaca: A reflective fault injection tool based on patterns Proc of the 2002 Intern Conference on Dependable Systems & Networks, pp. 483-487, Washington D.C. USA, 23-267, 2002Moraes, R., Martins, E., A Strategy for Validating an ODBMSComponent Using a High-Level Software Fault Injection Tool (2003) proc.of the First Latin-American Symp, pp. 56-68. , SP, BrazilMoraes, R., Martins, E., An Architecture-based Strategy for Interface Fault Injection (2004) Workshop on Architecting Dependable Systems, IEEE/IFIP International Conf. on Dependable Systems and Networks, , Florence, Italy, June 28-July 1Mukherjee, A., Siewiorek, D., Measuring Software Dependability by Robustness Benchmarking (1997) IEEE Transactions on Software Engineering, 23, pp. 366-378Ng, W., Aycock, C., Chen, P., Comparing Disk and Memory's Resistance to Operating System Crashes (1996) Proc. of The 7th IEEE International Symposium on Software Reliability Engineering, ISSRE, , 96, New York, NY, USANg, W., Chen, P., Systematic improvement of fault tolerance in the RIO file cache (1999) Proc. of The 30th IEEE Fault Tolerant Computing Symp., FTCS-29, , Madison, WI, USATsai, T., Singh, N., Reliability Testing of Applications on Windows NT (2000) Proceedings of the IEEE International Symposium on Dependable Systems and Networks - DSN00, pp. 427-436. , New York, NY, USA, ppVoas, J., Charron, F., McGraw, G., Miller, K., Friedman, M., Predicting How Badly 'Good' Software can Behave (1997) IEEE SoftwareVoas, J., (1997) A Defensive Approach to Certifying COTS Software, , Reliable Software TechnologiesWeyuker, E.J., Testing Component-Based Software: A Cautionary Tale (1998) IEEE Software, pp. 54-5