How Software Developers Mitigate their Errors when Developing Code

Code remains largely hand-made by humans and, as such, writing code is prone to error. Many previous studies have focused on the technical reasons for these errors and provided developers with increasingly sophisticated tools. Few studies have looked in detail at why code errors have been made from a human perspective. We use Human Error Theory to frame our exploratory study and use semi-structured interviews to uncover a preliminary understanding of the errors developers make while coding. We look particularly at the skill-based errors reported by 27 professional software developers. We found that the complexity of the development environment is one of the most frequently reported reasons for errors. Maintaining concentration and focus on a particular task also underpins many developer errors. We found that developers struggle with effective mitigation strategies for their errors, reporting strategies largely based on improving their own willpower to concentrate better on coding tasks. We discuss how using Reason's Swiss Cheese model may help reduce errors during software development. This model ensures that layers of tool, process and management mitigation are in place to prevent developer errors from causing system failures

Understanding the Issues, Their Causes and Solutions in Microservices Systems: An Empirical Study

Many small to large organizations have adopted the Microservices Architecture (MSA) style to develop and deliver their core businesses. Despite the popularity of MSA in the software industry, there is a limited evidence-based and thorough understanding of the types of issues (e.g., errors, faults, failures, and bugs) that microservices system developers experience, the causes of the issues, and the solutions as potential fixing strategies to address the issues. To ameliorate this gap, we conducted a mixed-methods empirical study that collected data from 2,641 issues from the issue tracking systems of 15 open-source microservices systems on GitHub, 15 interviews, and an online survey completed by 150 practitioners from 42 countries across 6 continents. Our analysis led to comprehensive taxonomies for the issues, causes, and solutions. The findings of this study inform that Technical Debt, Continuous Integration and Delivery, Exception Handling, Service Execution and Communication, and Security are the most dominant issues in microservices systems. Furthermore, General Programming Errors, Missing Features and Artifacts, and Invalid Configuration and Communication are the main causes behind the issues. Finally, we found 177 types of solutions that can be applied to fix the identified issues. Based on our study results, we formulated future research directions that could help researchers and practitioners to engineer emergent and next-generation microservices systems.Comment: 35 pages, 5 images, 7 tables, Manuscript submitted to a Journal (2023

Evaluating and Improving Risk Analysis Methods for Critical Systems

At the same time as our dependence on IT systems increases, the number of reports of problems caused by failures of critical IT systems has also increased. Today, almost every societal system or service, e.g., water supply, power supply, transportation, depends on IT systems, and failures of these systems have serious and negative effects on society. In general, public organizations are responsible for delivering these services to society. Risk analysis is an important activity for the development and operation of critical IT systems, but the increased complexity and size of critical systems put additional requirements on the effectiveness of risk analysis methods. Even if a number of methods for risk analysis of technical systems exist, the failure behavior of information systems is typically very different from mechanical systems. Therefore, risk analysis of IT systems requires different risk analysis techniques, or at least adaptations of traditional approaches. The research objective of this thesis is to improve the analysis process of risks pertaining to critical IT systems, which is addressed in the following three ways. First, by understanding current literature and practices related to risk analysis of IT systems, then by evaluating and comparing existing risk analysis methods, and by suggesting improvements in the risk analysis process and by developing new effective and efficient risk analysis methods to analyze IT systems. To understand current risk analysis methods and practices we carried out a systematic mapping study. The study found only few empirical research papers on the evaluation of existing risk analysis methods. The results of the study suggest to empirically investigate risk analysis methods for analyzing IT systems to conclude which methods are more effective than others. Then, we carried out a semi-structured interview study to investigate several factors regarding current practices and existing challenges of risk analysis and management, e.g., its importance, identification of critical resources, involvement of different stakeholders, used methods, and follow-up analysis. To evaluate and compare the effectiveness of risk analysis methods we carried out a controlled experiment. In that study, we evaluated the effectiveness of risk analysis methods by counting the number of relevant and non-relevant risks identified by the experiment participants. The difficulty level of risk analysis methods and the experiment participants’ confidence about the identified risks were also investigated. Then, we carried out a case study to evaluate the effectiveness and efficiency of existing risk analysis methods, Failure Mode and Effect Analysis (FMEA) and System Theoretic Process Analysis (STPA). The case study investigates the effectiveness of the methods by performing a comparison of how a hazard analysis is conducted for the same system. It also evaluates the analysis process of risk analysis methods by using a set of qualitative criteria, derived from the Technology Acceptance Model (TAM). After this, another case study was carried out to evaluate and assess the resilience of critical IT systems and networks by applying a simulation method. A hybrid modeling approach was used which considers the technical network, represented using graph theory, as well as the repair system, represented by a queuing model. To improve the risk analysis process, this thesis also presents a new risk analysis method, Perspective Based Risk Analysis (PBRA), that uses different perspectives while analyzing IT systems. A perspective is a point of view or a specific role adopted by risk analyst while doing risk analysis, i.e., system engineer, system tester, or system user. Based on the findings, we conclude that the use of different perspectives improves effectiveness of risk analysis process. Then, to improve the risk analysis process we carried out a data mining study to save historical information about IT incidents to be used later for risk analysis. It could be an important aid in the process of building a database of occurred IT incidents that later can be used as an input to improve the risk analysis process. Finally, based on the findings of the studies included in this thesis a list of suggestions is presented to improve the risk analysis process. This list of potential suggestions was evaluated in a focus group meeting. The suggestions are for example, risk analysis awareness and education, defining clear roles and responsibilities, easy-to-use and adapt risk analysis methods, dealing with subjectivity, carry out risk analysis as early as possible and finally using historical risk data to improve the risk analysis process. Based on the findings it can be concluded that these suggestions are important and useful for risk practitioners to improve the risk analysis process.The presented research work in this thesis provides research about methods to improve the risk analysis and management practices. Moreover, the presented work in this thesis is based on solid empirical studies

Data management and Data Pipelines: An empirical investigation in the embedded systems domain

Context: Companies are increasingly collecting data from all possible sources to extract insights that help in data-driven decision-making. Increased data volume, variety, and velocity and the impact of poor quality data on the development of data products are leading companies to look for an improved data management approach that can accelerate the development of high-quality data products. Further, AI is being applied in a growing number of fields, and thus it is evolving as a horizontal technology. Consequently, AI components are increasingly been integrated into embedded systems along with electronics and software. We refer to these systems as AI-enhanced embedded systems. Given the strong dependence of AI on data, this expansion also creates a new space for applying data management techniques. Objective: The overall goal of this thesis is to empirically identify the data management challenges encountered during the development and maintenance of AI-enhanced embedded systems, propose an improved data management approach and empirically validate the proposed approach.Method: To achieve the goal, we conducted this research in close collaboration with Software Center companies using a combination of different empirical research methods: case studies, literature reviews, and action research.Results and conclusions: This research provides five main results. First, it identifies key data management challenges specific to Deep Learning models developed at embedded system companies. Second, it examines the practices such as DataOps and data pipelines that help to address data management challenges. We observed that DataOps is the best data management practice that improves the data quality and reduces the time tdevelop data products. The data pipeline is the critical component of DataOps that manages the data life cycle activities. The study also provides the potential faults at each step of the data pipeline and the corresponding mitigation strategies. Finally, the data pipeline model is realized in a small piece of data pipeline and calculated the percentage of saved data dumps through the implementation.Future work: As future work, we plan to realize the conceptual data pipeline model so that companies can build customized robust data pipelines. We also plan to analyze the impact and value of data pipelines in cross-domain AI systems and data applications. We also plan to develop AI-based fault detection and mitigation system suitable for data pipelines

Prognostics and health management for maintenance practitioners - Review, implementation and tools evaluation.

In literature, prognostics and health management (PHM) systems have been studied by many researchers from many different engineering fields to increase system reliability, availability, safety and to reduce the maintenance cost of engineering assets. Many works conducted in PHM research concentrate on designing robust and accurate models to assess the health state of components for particular applications to support decision making. Models which involve mathematical interpretations, assumptions and approximations make PHM hard to understand and implement in real world applications, especially by maintenance practitioners in industry. Prior knowledge to implement PHM in complex systems is crucial to building highly reliable systems. To fill this gap and motivate industry practitioners, this paper attempts to provide a comprehensive review on PHM domain and discusses important issues on uncertainty quantification, implementation aspects next to prognostics feature and tool evaluation. In this paper, PHM implementation steps consists of; (1) critical component analysis, (2) appropriate sensor selection for condition monitoring (CM), (3) prognostics feature evaluation under data analysis and (4) prognostics methodology and tool evaluation matrices derived from PHM literature. Besides PHM implementation aspects, this paper also reviews previous and on-going research in high-speed train bogies to highlight problems faced in train industry and emphasize the significance of PHM for further investigations

A novel approach for No Fault Found decision making

Within aerospace and defence sectors, organisations are adding value to their core corporate offerings through services. These services tend to emphasise the potential to maintain future revenue streams and improved profitability and hence require the establishment of cost effective strategies that can manage uncertainties within value led services e.g. maintenance activities. In large organisations, decision-making is often supported by information processing and decision aiding systems; it is not always apparent whose decision affects the outcome the most. Often, accountability moves away from the designated organisation personnel in unforeseen ways, and depending on the decisions of individual decision makers, the structure of the organisation, or unregulated operating procedures may change. This can have far more effect on the overall system reliability – leading to inadequate troubleshooting, repeated down-time, reduced availability and increased burden on Through-life Engineering Services. This paper focuses on outlining current industrial attitudes regarding the No Fault Found (NFF) phenomena and identifies the drivers that influence the NFF decision-making process. It articulates the contents of tacit knowledge and addresses a knowledge gap by developing NFF management policies. The paper further classifies the NFF phenomenon into five key processes that must be controlled by using the developed policies. In addition to the theoretical developments, a Petri net model is also outlined and discussed based on the captured information regarding NFF decision-making in organisations. Since NFF decision-making is influenced by several factors, Petri nets are sought as a powerful tool to realise a meta-model capability to understand the complexity of situations. Its potential managerial implications can help describe decision problems under conditions of uncertainty. Finally, the conclusions indicate that engineering processes, which allow decision-making at various maintenance echelons, can often obfuscate problems that then require a systems approach to illustrate the impact of the issue

A study on marine accident causation models employed by marine casualty investigators / by Fatoumatta Cassama

This research highlights relevant issues related to marine casualties and presents an overview on casualty investigation, a review of marine accidents, the regulatory framework on marine casualty investigations, a brief discussion on system’s design complexity and coupling characteristics, accident causation models used in casualty analysis and the marine accident investigation organizations. The principal objective of the study was to identify and evaluate marine casualty investigators’ endeavors of determining causes of a marine accident with the help of accident causation models or investigation procedures involving accident causation models. The study therefore focuses on the marine accident causation models one could utilize for conducting investigation into marine accidents. States establish an accident investigation regime to determine why an accident happened and to learn lessons that prevent similar accidents from happening in the future. The overall approach towards the research methodology was to employ mixed methods to complement the data as well as to obtain increased response from the target group. In pursuance of this goal, a mixed methods approach comprising questionnaires and structured interviews was adopted towards data collection for the study. The models applied by practitioners ranged from none to a plethora of models. The SHEL and Reason’s Swiss cheese model were common to the questionnaire respondents and interview participants while the other models mentioned were the ATSB, IMO-MAIIF, HTO, FRAM, AcciMap, MTO, ISIM and Heinrich’s Domino model. The utilization of event and causal factors diagrams was also mentioned along with path dependency. This highlights the diversity in the available models. The reasons the participants gave for the utilization of models largely depended upon the ability of the model to capture maritime accidents including complex accidents and the level of training required in the application of the model. The ability of the model to address organizational aspects rather than mechanical failures was highlighted. Also highlighted was the juxtaposition of models – that is utilizing a model to identify the technical aspects of the accident and another to explore how it was managed. Another reason highlighted was the requirement by organizations which mandated a particular model to be used. Various reasons have been stipulated by these marine accident investigators for their preferences of using particular models or none at all. The reduction of marine accidents in the maritime industry as a result of the use of models or not, is in conclusive

Evidence-driven testing and debugging of software systems

Program debugging is the process of testing, exposing, reproducing, diagnosing and fixing software bugs. Many techniques have been proposed to aid developers during software testing and debugging. However, researchers have found that developers hardly use or adopt the proposed techniques in software practice. Evidently, this is because there is a gap between proposed methods and the state of software practice. Most methods fail to address the actual needs of software developers. In this dissertation, we pose the following scientific question: How can we bridge the gap between software practice and the state-of-the-art automated testing and debugging techniques? To address this challenge, we put forward the following thesis: Software testing and debugging should be driven by empirical evidence collected from software practice. In particular, we posit that the feedback from software practice should shape and guide (the automation) of testing and debugging activities. In this thesis, we focus on gathering evidence from software practice by conducting several empirical studies on software testing and debugging activities in the real-world. We then build tools and methods that are well-grounded and driven by the empirical evidence obtained from these experiments. Firstly, we conduct an empirical study on the state of debugging in practice using a survey and a human study. In this study, we ask developers about their debugging needs and observe the tools and strategies employed by developers while testing, diagnosing and repairing real bugs. Secondly, we evaluate the effectiveness of the state-of-the-art automated fault localization (AFL) methods on real bugs and programs. Thirdly, we conducted an experiment to evaluate the causes of invalid inputs in software practice. Lastly, we study how to learn input distributions from real-world sample inputs, using probabilistic grammars. To bridge the gap between software practice and the state of the art in software testing and debugging, we proffer the following empirical results and techniques: (1) We collect evidence on the state of practice in program debugging and indeed, we found that there is a chasm between (available) debugging tools and developer needs. We elicit the actual needs and concerns of developers when testing and diagnosing real faults and provide a benchmark (called DBGBench) to aid the automated evaluation of debugging and repair tools. (2) We provide empirical evidence on the effectiveness of several state-of-the-art AFL techniques (such as statistical debugging formulas and dynamic slicing). Building on the obtained empirical evidence, we provide a hybrid approach that outperforms the state-of-the-art AFL techniques. (3) We evaluate the prevalence and causes of invalid inputs in software practice, and we build on the lessons learned from this experiment to build a general-purpose algorithm (called ddmax) that automatically diagnoses and repairs real-world invalid inputs. (4) We provide a method to learn the distribution of input elements in software practice using probabilistic grammars and we further employ the learned distribution to drive the test generation of inputs that are similar (or dissimilar) to sample inputs found in the wild. In summary, we propose an evidence-driven approach to software testing and debugging, which is based on collecting empirical evidence from software practice to guide and direct software testing and debugging. In our evaluation, we found that our approach is effective in improving the effectiveness of several debugging activities in practice. In particular, using our evidence-driven approach, we elicit the actual debugging needs of developers, improve the effectiveness of several automated fault localization techniques, effectively debug and repair invalid inputs, and generate test inputs that are (dis)similar to real-world inputs. Our proposed methods are built on empirical evidence and they improve over the state-of-the-art techniques in testing and debugging.Software-Debugging bezeichnet das Testen, Aufspüren, Reproduzieren, Diagnostizieren und das Beheben von Fehlern in Programmen. Es wurden bereits viele Debugging-Techniken vorgestellt, die Softwareentwicklern beim Testen und Debuggen unterstützen. Dennoch hat sich in der Forschung gezeigt, dass Entwickler diese Techniken in der Praxis kaum anwenden oder adaptieren. Das könnte daran liegen, dass es einen großen Abstand zwischen den vorgestellten und in der Praxis tatsächlich genutzten Techniken gibt. Die meisten Techniken genügen den Anforderungen der Entwickler nicht. In dieser Dissertation stellen wir die folgende wissenschaftliche Frage: Wie können wir die Kluft zwischen Software-Praxis und den aktuellen wissenschaftlichen Techniken für automatisiertes Testen und Debugging schließen? Um diese Herausforderung anzugehen, stellen wir die folgende These auf: Das Testen und Debuggen von Software sollte von empirischen Daten, die in der Software-Praxis gesammelt wurden, vorangetrieben werden. Genauer gesagt postulieren wir, dass das Feedback aus der Software-Praxis die Automation des Testens und Debuggens formen und bestimmen sollte. In dieser Arbeit fokussieren wir uns auf das Sammeln von Daten aus der Software-Praxis, indem wir einige empirische Studien über das Testen und Debuggen von Software in der echten Welt durchführen. Auf Basis der gesammelten Daten entwickeln wir dann Werkzeuge, die sich auf die Daten der durchgeführten Experimente stützen. Als erstes führen wir eine empirische Studie über den Stand des Debuggens in der Praxis durch, wobei wir eine Umfrage und eine Humanstudie nutzen. In dieser Studie befragen wir Entwickler zu ihren Bedürfnissen, die sie beim Debuggen haben und beobachten die Werkzeuge und Strategien, die sie beim Diagnostizieren, Testen und Aufspüren echter Fehler einsetzen. Als nächstes bewerten wir die Effektivität der aktuellen Automated Fault Localization (AFL)- Methoden zum automatischen Aufspüren von echten Fehlern in echten Programmen. Unser dritter Schritt ist ein Experiment, um die Ursachen von defekten Eingaben in der Software-Praxis zu ermitteln. Zuletzt erforschen wir, wie Häufigkeitsverteilungen von Teileingaben mithilfe einer Grammatik von echten Beispiel-Eingaben aus der Praxis gelernt werden können. Um die Lücke zwischen Software-Praxis und der aktuellen Forschung über Testen und Debuggen von Software zu schließen, bieten wir die folgenden empirischen Ergebnisse und Techniken: (1) Wir sammeln aktuelle Forschungsergebnisse zum Stand des Software-Debuggens und finden in der Tat eine Diskrepanz zwischen (vorhandenen) Debugging-Werkzeugen und dem, was der Entwickler tatsächlich benötigt. Wir sammeln die tatsächlichen Bedürfnisse von Entwicklern beim Testen und Debuggen von Fehlern aus der echten Welt und entwickeln einen Benchmark (DbgBench), um das automatische Evaluieren von Debugging-Werkzeugen zu erleichtern. (2) Wir stellen empirische Daten zur Effektivität einiger aktueller AFL-Techniken vor (z.B. Statistical Debugging-Formeln und Dynamic Slicing). Auf diese Daten aufbauend, stellen wir einen hybriden Algorithmus vor, der die Leistung der aktuellen AFL-Techniken übertrifft. (3) Wir evaluieren die Häufigkeit und Ursachen von ungültigen Eingaben in der Softwarepraxis und stellen einen auf diesen Daten aufbauenden universell einsetzbaren Algorithmus (ddmax) vor, der automatisch defekte Eingaben diagnostiziert und behebt. (4) Wir stellen eine Methode vor, die Verteilung von Schnipseln von Eingaben in der Software-Praxis zu lernen, indem wir Grammatiken mit Wahrscheinlichkeiten nutzen. Die gelernten Verteilungen benutzen wir dann, um den Beispiel-Eingaben ähnliche (oder verschiedene) Eingaben zu erzeugen. Zusammenfassend stellen wir einen auf der Praxis beruhenden Ansatz zum Testen und Debuggen von Software vor, welcher auf empirischen Daten aus der Software-Praxis basiert, um das Testen und Debuggen zu unterstützen. In unserer Evaluierung haben wir festgestellt, dass unser Ansatz effektiv viele Debugging-Disziplinen in der Praxis verbessert. Genauer gesagt finden wir mit unserem Ansatz die genauen Bedürfnisse von Entwicklern, verbessern die Effektivität vieler AFL-Techniken, debuggen und beheben effektiv fehlerhafte Eingaben und generieren Test-Eingaben, die (un)ähnlich zu Eingaben aus der echten Welt sind. Unsere vorgestellten Methoden basieren auf empirischen Daten und verbessern die aktuellen Techniken des Testens und Debuggens