A comparison of the reliability growth of open source and in-House software

As commercial developers have established processes to assure software quality, open source software depends largely on community usage and defect reporting to achieve some level of quality. Thus, quality of open source software may vary. We examined defects reported in two active and popular open source software projects and an in-house project. The results of this analysis indicate that the reliability growth of each is quite distinct and that the defect profile of open source software appears to be a consequence of the open source software development method itself. © 2008 IEEE

Reproducibility of environment-dependent software failures: An experience report

Abstract-We investigate the dependence of software failure reproducibility on the environment in which the software is executed. The existence of such dependence is ascertained in literature, but so far it is not fully characterized. In this paper we pinpoint some of the environmental components that can affect the reproducibility of a failure and show this influence through an experimental campaign conducted on the MySQL Server software system. The set of failures of interest is drawn from MySQL's failure reports database and an experiment is designed for each of these failures. The experiments expose the influence of disk usage and level of concurrency on MySQL failure reproducibility. Furthermore, the results show that high levels of usage of these factors increase the probabilities of failure reproducibility

MiSFIT: Mining Software Fault Information and Types

As software becomes more important to society, the number, age, and complexity of systems grow. Software organizations require continuous process improvement to maintain the reliability, security, and quality of these software systems. Software organizations can utilize data from manual fault classification to meet their process improvement needs, but organizations lack the expertise or resources to implement them correctly. This dissertation addresses the need for the automation of software fault classification. Validation results show that automated fault classification, as implemented in the MiSFIT tool, can group faults of similar nature. The resulting classifications result in good agreement for common software faults with no manual effort. To evaluate the method and tool, I develop and apply an extended change taxonomy to classify the source code changes that repaired software faults from an open source project. MiSFIT clusters the faults based on the changes. I manually inspect a random sample of faults from each cluster to validate the results. The automatically classified faults are used to analyze the evolution of a software application over seven major releases. The contributions of this dissertation are an extended change taxonomy for software fault analysis, a method to cluster faults by the syntax of the repair, empirical evidence that fault distribution varies according to the purpose of the module, and the identification of project-specific trends from the analysis of the changes

Classificação e resolução de defeitos em manutenção de software utilizando ODC e histórico de soluções

Nowadays with the increasing of demand in support services such as high availability and performance faced by customers and low cost operations to software maintenance and hosting enterprises, the resolution of software incidents in less time and defect prevention turned a key point. Moreover, the software maintenance is one most time consuming and effort demanding and by consequence cost in software development life cycle. Balancing effectiveness and costs turn a challenge to any enterprise that manages support in software maintenance. The approach used on this research defines a process to classify defects and boundary a set of best solutions associated to these classes from defects history and customer knowledge base. Also this classification separates different problem complexities that can be handled to different support teams. The base method used is the ODC (Orthogonal Defect Classification). With this research is possible to verify that the classification and solutions associations with problem class can undertake a time reduction in incidents resolution in software maintenance. It was verified using four service provider samples in two different customers (X and Y) that classification of defects, correct support team redirection and best solution grouping helps on reduction of incidents resolution time between 70% of incidents to customer Y and 92,5 % of incidents to customer X. The reduction was reached with reduction in number of transfers between supporting teams and incidents number reduction. The process is incremental because it is unfolded from history information and from effectiveness into solutions purposed. This process can leverage human resources nedded to support computational systems in service providers.Nos dias atuais, com o aumento da demanda de serviços de suporte, como, por exemplo, no campo da alta disponibilidade e do desempenho para o cliente e da demanda de custos mais baixos para as empresas de manutenção e hospedagem de software, a resolução de incidentes de software em um tempo menor junto ao cliente e a prevenção de defeitos tornaram-se um tópicos fundamentais. Além disso, a atividade de manutenção de software é uma das fases que consome mais tempo, esforço e consequentemente custo no ciclo de desenvolvimento de software. Balancear eficiência e custo torna-se um desafio para qualquer empresa de suporte em manutenção de software. A abordagem utilizada nesta pesquisa estabelece um processo para classificar defeitos e delinear um conjunto de melhores soluções para os defeitos classificados a partir do histórico de defeitos e a base de conhecimento do cliente. Esta classificação também separa complexidade de problemas para serem gerenciados pelo time de suporte mais adequado. O método base utilizado é o ODC (Classificação Ortogonal de Defeitos) e extensões voltadas ao suporte de software são propostas e utilizadas. Por meio desta pesquisa, é possível verificar se a classificação e associação de soluções podem acarretar em uma redução no tempo de atendimento dos incidentes de suporte. Foi observado em quatro amostras de dois clientes diferentes (X e Y) que utilizando a classificação dos defeitos, direcionamento correto aos times de suporte e agrupamento de soluções, promoveu uma redução no tempo de atendimento em 70% dos incidentes de suporte no cliente Y e 92,5% dos incidentes de suporte para o cliente X. A redução de tempo foi obtida pela redução no número de transferências entre os times de suporte e a redução de incidentes. O processo apresentado é incremental, pois é baseado no aumento das informações históricas e na eficácia das soluções propostas. Este método de soluções pode favorecer a redução dos recursos necessários para suportar sistemas computacionais em provedores de serviço

Automatic generation of test scenarios from the models of systems specification

Orientador: Eliane MartinsDissertação (mestrado) - Universidade Estadual de Campinas, Instituto de ComputaçãoResumo: As crescentes exigências em relação à melhoria de qualidade e a redução de custos e prazos têm tornado comum à busca por soluções mais eficientes para desenvolvimento e testes de sistemas. Com relação aos testes, uma recomendação é a de começá-los mais cedo, e, de preferência, automatizar o que for possível para evitar enganos cometidos pelos testadores. Assim, uma área de pesquisa que está em evidência atualmente é a de testes baseados em modelos (MBT), onde muitos estudos têm sido realizados visando o desenvolvimento de soluções para automação de testes a partir de modelos criados durante o ciclo de desenvolvimento. Nesta dissertação é proposto um método para geração automática de cenários de teste a partir de modelos da especificação de sistemas, baseando-se em um trabalho prévio para geração de testes de componentes de software. Os modelos utilizados para geração dos testes são os Diagramas de Atividades UML, criados a partir da descrição dos casos de uso, para Testes de Sistemas. A partir deles são gerados cenários de teste que descrevem as interações do sistema, tais como, as ações dos atores e as situações esperadas, incluindo também os cenários de exceção. A aplicação deste método na prática foi feita com êxito por uma equipe de testadores em sistemas reais. De forma geral, os modelos especificados para derivação dos testes têm facilitado o entendimento do sistema pelos testadores envolvidos e as informações presentes nos cenários de teste têm apoiado a realização dos testes.Abstract: The increasing requirements for quality improvement, reduction of costs and deadlines have promoted the search for more efficient solutions for systems development and testing. In the case of the tests, the recommendation is to start them earlier and, preferably, automatize what is possible to prevent the testers mistakes. Thus, a research area that is in evidence currently is the Model-Based Testing (MBT), in which many studies have been carried out with the aim of development solutions for test automation from the models created during the software development cycle. In this dissertation is proposed a method for automatic generation of test scenarios from the models of systems specification, it is based on a previous work for tests generation of software components. The models used for tests generation are the UML Activities Diagrams, generated from the description of the use cases for System Testing. From them, test scenarios are generated that describe the interactions of the system, such as, the actors actions and the expected situations, including also the exception scenarios. The application of this method in practice was successfully made by a team of testers in real systems. In general, the models specified for tests derivation have facilitated the agreement of the system by the involved testers and, the generated test scenarios contain information that have supported the test execution.MestradoMestre em Ciência da Computaçã

Aus Fehlern in der Softwareentwicklung lernen. Wie durch Fehleranalysen die Prozesse der Anforderungsanalyse und der Qualitätssicherung verbessert werden können

Softwarefehler existieren, seit Menschen Software entwickeln. Fehler können mitunter zu erheblichen wirtschaftlichen Verlusten und im schlimmsten Fall zum Verlust von Leben führen. Viele Fehler können auf Mängel im Prozess der Anforderungsanalyse zurückgeführt werden. Je später ein Anforderungsfehler entdeckt und behoben wird, desto aufwändiger wird die Korrektur. Die vorliegende Arbeit beschreibt, wie aus Fehlern in der Softwareentwicklung gelernt werden kann. Sie beschreibt ein Verfahren zu Fehleranalyse, auf dessen Basis insbesondere Prozesse der Anforderungsanalyse und der Qualitätssicherung verbessert werden können. Ziel der Verbesserungen ist es, Anforderungsfehler und mögliche Folgefehler im Entwurf und der Implementierung zu vermeiden oder zumindest früher zu finden. In dieser Arbeit wird zunächst ein Modell hergeleitet, das erklärt, warum Anforderungsfehler entstehen. Für bestimmte Typen von Anforderungsfehlern werden auf der Grundlage empirische Befunde konkrete Ursachen im Prozess der Anforderungsanalyse aufgezeigt. Dieses Erklärungsmodell ist Bestandteil eines Verfahrens zur Fehleranalyse, das den Anspruch erhebt, über die Auswertung von Fehlern Rückschlüsse über mögliche Ursachen im Prozess zu ziehen. Das Verfahren ist eine Weiterentwicklung der Orthogonal Defect Classification, kurz ODC. ODC wird in der Arbeit ausführlich dargestellt und auf der Grundlage empirischer Befunde kritisch gewürdigt. Das weiterentwickelte Verfahren zur Fehleranalyse wurde im Rahmen einer einjährigen Fallstudie bei dem IT-Dienstleister einer großen deutschen Versicherung erfolgreich angewandt. Hierbei wurden nachträglich reale Fehler von zwei Softwareentwicklungsprojekten einer geschäftskritischen Anwendungssoftware klassifiziert und analysiert, um Verbesserungspotenziale zu identifizieren. Das in der Arbeit entwickelte Verfahren zur Fehleranalyse leistet einen unmittelbaren Beitrag zur Lösung des aufgezeigten Praxisproblems: sie ist ein Instrument, um Prozessmängel der Anforderungsanalyse zu identifizieren, die systematisch Anforderungsfehler und Folgefehler verursachen