Hadooptest : um controlador de testes para sistemas baseados em mapreduce

Orientador : Prof. Dr. Eduardo Cunha de AlmeidaDissertação (mestrado) - Universidade Federal do Paraná, Setor de Ciências Exatas, Programa de Pós-Graduação em Informática. Defesa: Curitiba, 08/12/2011Bibliografia: fls. 51-45Resumo: O MapReduce (MR) e uma das soluções mais populares na área de processamento de dados em grande escala. Os sistemas baseados em MapReduce frequentemente são implantados sobre agrupamentos de computadores, onde falhas acontecem constantemente, devido a defeitos de software, problemas de hardware e interrupções. Testar sistemas baseados em MapReduce é difícil, uma vez que é necessário um grande esforço do controlador de testes para executar casos de teste distribuídos em ambientes com a presença de falhas. Neste trabalho, apresentamos uma nova solução de testes para resolver isso, que foi chamada de HadoopTest. Esta solução baseia-se em uma abordagem de controle escalável, onde um coordenador gerência diversos testadores distribuídos, que controlam os componentes do MR. Os testadores podem simular falhas sobre os componentes do MR e monitorar suas execuções. O HadoopTest foi utilizado para testar duas aplicações distribuídas juntamente com o Hadoop (i.e., a implementação MapReduce de código aberto mantida pela fundação Apache). Nossos experimentos apresentaram resultados promissores, sendo que o HadoopTest conseguiu coordenar casos de teste distribuídos, injetar falhas nos componentes do MR e encontrar alguns defeitos de software que foram propositalmente inseridos.Abstract: MapReduce (MR) is one of the most popular solution on large-scale data processing area. The MR-based systems are often deployed over clusters of computers, where failures happen constantly due to bugs, hardware problems, and outages. Testing MR-based systems is hard, since it is needed a great eort of test controller to execute distributed test cases upon failures. In this work, we present a novel testing solution to tackle this issue called HadoopTest. This solution is based on a scalable control approach, where a coordinator manages many distributed testers which control the MR components. Testers are allowed to simulate failures on MR components and monitor their behavior. HadoopTest was used to test two applications bundled into Hadoop (i.e., a open source MapReduce implementation mantained by Apache Foundation). On our experiments HadoopTest was able to coordinate distributed test cases, inject faults on MR components and nd some bugs which were purposely inserted

Testing Grid Application Workflows Using TTCN-3

The collective and coordinated usage of distributed resources for problem solution within dynamic virtual organizations can be realized with the Grid computing technology. For distributing and solving a task, a Grid application involves a complex workflow of dividing a task into smaller sub-tasks, scheduling and submitting jobs for solving those sub-tasks, and eventually collecting and combining the results of the sub-tasks into a final result. The quality assurance of Grid applications is a challenge due to the highly distributed nature of the Grid environment in which the Grid application is deployed. This paper investigates the applicability of the Testing and Test Control Notation (TTCN-3) for testing the workflows of distributed Grid applications. To this aim, a case study has been created that consists of a distributed Grid application which includes a typical Grid application workflow; as the main contribution, this case study contains a corresponding distributed TTCN-3 test suite that tests the correct execution of the Grid application TTCN-3 test suite to a specific Grid environment, corresponding reusable test adapters have been implemented for the Grid middleware Globus Toolkit 4 (GT4). The realized test system demonstrates that TTCN-3 is applicable for testing the workflow of distributed Grid applications. 1