Integration of an Automatic Fault Localization Tool in an IDE and its Evaluation

Debugging is one of the most demanding and error-prone tasks in software development. Trying to address bugs has become overall more expensive as the software complexity and size have increased. As a result, several researchers attempted to improve the developers’ debugging experience and efficiency by automating as much of the process as possible. Existing auto-finding tools will assist developers in automatically detecting bugs, however, they are not yet widely available to software engineers. Making such tools available to developers can save debugging time and increase the productivity. Subsequently, the main goal of this dissertation is to incorporate an automatic fault localization tool into an Integrated Development Environment (IDE). The selected IDE was Visual Studio Code, a source-code editor developed by Microsoft for Windows, Linux, and macOS. Visual Studio Code is one of the most used IDEs and is known for its flexible API, which allows nearly every aspect of it to be customized. Furthermore, the chosen automatic fault localization tool was FLACOCO, a recent fault localization tool for Java that supports up to the most recent versions. Nonetheless, this document contains a full overview of several fault localization methodologies and tools, as well as an explanation of the complete planning and development process of the produced Visual Studio Code extension. After the development and deployment were completed, an evaluation was carried out. The extension was evaluated through a user study in which thirty Java professionals took part. The test had two parts: the first involved users using the extension to complete two debugging tasks in previously unknown projects, and the second had them filling out a satisfaction questionnaire for further analysis. Finally, the results show that the extension was a success, with the system being rated positively in all areas. However, it may be revised in light of the questionnaire responses, with the suggestions received being considered for future work.A depuração é uma das tarefas mais exigentes e propensas a erros no desenvolvimento de software. Tentar resolver esses erros tornou-se mais dispendioso com os incrementos de complexidade e tamanho do software. Deste modo, ao longo dos últimos anos, vários investigadores tentaram melhorar a experiência de depuração e a eficiência dos desenvolvedores automatizando o máximo possível do processo. Existem ferramentas de localização de defeitos que assistem os desenvolvedores na detecção automática de bugs, no entanto estas ainda não se encontram amplamente disponíveis para os programadores. Tornar essas ferramentas disponíveis para todos certamente iria resultar na redução do tempo de depuração e no aumento da produtividade. Assim sendo, o principal objetivo desta dissertação é incorporar uma ferramenta de localização automática de defeitos num IDE. Em termos de IDE, o Visual Studio Code, um editor de código-fonte desenvolvido pela Microsoft para Windows, Linux e macOS, foi selecionado. Este IDE tem ganho bastante popularidade, sendo um dos IDEs mais utilizados mundialmente. Além disso, o Visual Studio Code é reconhecido pela sua API flexível, que permite que quase todos os seus aspectos sejam personalizados. Adicionalmente, o FLACOCO, uma ferramenta de localização de defeitos baseada em SFL que suporta até as versões mais recentes do Java, foi escolhida como ferramenta de localização automática de defeitos. Além do mais, esta dissertação contém um estudo sobre as técnicas de localização automática de defeitos e as suas ferramentas, bem como uma explicação do planeamento e implementação da extensão criada para o Visual Studio Code. Após o término da implementação e a posterior implantação, foi efetuada a sua avaliação. Procedeu-se a um teste de utilização com a participação de treze utilizadores proficientes na linguagem Java. O teste foi composto por duas componentes: na primeira os utilizadores utilizaram a extensão para completar duas tarefas de depuração em projetos por eles desconhecidos e na segunda foi-lhes fornecido um questionário de satisfação para posterior análise. Os resultados obtidos sugerem que a extensão foi um sucesso, sendo que o sistema foi positivamente avaliado em todos os aspetos. No entanto a mesma poderá ser aprimorada tendo em consideração o feedback obtido na secção de resposta livre do questionário, sendo que o mesmo foi bastante valioso e as sugestões apuradas vieram a ser consideradas para trabalho futuro

Fault Diagnosis in Enterprise Software Systems Using Discrete Monitoring Data

Success for many businesses depends on their information software systems. Keeping these systems operational is critical, as failure in these systems is costly. Such systems are in many cases sophisticated, distributed and dynamically composed. To ensure high availability and correct operation, it is essential that failures be detected promptly, their causes diagnosed and remedial actions taken. Although automated recovery approaches exists for specific problem domains, the problem-resolution process is in many cases manual and painstaking. Computer support personnel put a great deal of effort into resolving the reported failures. The growing size and complexity of these systems creates the need to automate this process. The primary focus of our research is on automated fault diagnosis and recovery using discrete monitoring data such as log files and notifications. Our goal is to quickly pinpoint the root-cause of a failure. Our contributions are: Modelling discrete monitoring data for automated analysis, automatically leveraging common symptoms of failures from historic monitoring data using such models to pinpoint faults, and providing a model for decision-making under uncertainty such that appropriate recovery actions are chosen. Failures in such systems are caused by software defects, human error, hardware failures, environmental conditions and malicious behaviour. Our primary focus in this thesis is on software defects and misconfiguration

Model-based fault localization in large-scale computing systems

We describe a new fault localization technique for software bugs in large-scale computing systems. Our technique always collects per-process function call traces of a target system, and derives a concise execu-tion model that reflects its normal function calling be-haviors using the traces. To find the cause of a failure, we compare the derived model with the traces collected when the system failed, and compute a suspect score that quantifies how likely a particular part of call traces ex-plains the failure. The execution model consists of a call probability of each function in the system that we esti-mate using the normal traces. Functions with low prob-abilities in the model give high anomaly scores when called upon a failure. Frequently-called functions in the model also give high scores when not called. Finally, we report the function call sequences ranked with the suspect scores to the human analyst, narrowing further manual localization down to a small part of the overall system. We have applied our proposed method to fault localization of a known non-deterministic bug in a dis-tributed parallel job manager. Experimental results on a three-site, 78-node distributed environment demonstrate that our method quickly locates an anomalous event that is highly correlated with the bug, indicating the effec-tiveness of our approach.