7,185 research outputs found
Extending Provenance For Deep Diagnosis Of Distributed Systems
Diagnosing and repairing problems in complex distributed systems has always been challenging. A wide variety of problems can happen in distributed systems: routers can be misconfigured, nodes can be hacked, and the control software can have bugs. This is further complicated by the complexity and scale of today’s distributed systems. Provenance is an attractive way to diagnose faults in distributed systems, because it can track the causality from a symptom to a set of root causes. Prior work on network provenance has successfully applied provenance to distributed systems. However, they cannot explain problems beyond the presence of faulty events and offer limited help with finding repairs.
In this dissertation, we extend provenance to handle diagnostics problems that require deeper investigations. We propose three different extensions: negative provenance explains not just the presence but also the absence of events (such as missing packets); meta provenance can suggest repairs by tracking causality not only for data but also for code (such as bugs in control plane programs); temporal provenance tracks causality at the temporal level and aims at diagnosing timing-related faults (such as slow requests). Compared to classical network provenance, our approach tracks richer causality at runtime and applies more sophisticated reasoning and post-processing. We apply the above techniques to software-defined networking and the border gateway protocol. Evaluations with real world traffic and topology show that our systems can diagnose and repair practical problems, and that the runtime overhead as well as the query turnarounds are reasonable
Recommended from our members
Enhancing Usability and Explainability of Data Systems
The recent growth of data science expanded its reach to an ever-growing user base of nonexperts, increasing the need for usability, understandability, and explainability in these systems. Enhancing usability makes data systems accessible to people with different skills and backgrounds alike, leading to democratization of data systems. Furthermore, proper understanding of data and data-driven systems is necessary for the users to trust the function of the systems that learn from data. Finally, data systems should be transparent: when a data system behaves unexpectedly or malfunctions, the users deserve proper explanation of what caused the observed incident. Unfortunately, most existing data systems offer limited usability and support for explanations: these systems are usable only by experts with sound technical skills, and even expert users are hindered by the lack of transparency into the systems\u27 inner workings and functions. The aim of my thesis is to bridge the usability gap between nonexpert users and complex data systems, aid all sort of users, including the expert ones, in data and system understanding, and provide explanations that help reason about unexpected outcomes involving data systems. Specifically, my thesis has the following three goals: (1) enhancing usability of data systems for nonexperts, (2) enable data understanding that can assist users in a variety of tasks such as achieving trust in data-driven machine learning, gaining data understanding, and data cleaning, and (3) explaining causes of unexpected outcomes involving data and data systems.
For enhancing usability, we focus on example-driven user intent discovery. We develop systems based on example-driven interactions in two different settings: querying relational databases and personalized document summarization. Towards data understanding, we develop a new data-profiling primitive that can characterize tuples for which a machine-learned model is likely to produce untrustworthy predictions. We also develop an explanation framework to explain causes of such untrustworthy predictions. Additionally, this new data-profiling primitive enables interactive data cleaning. Finally, we develop two explanation frameworks, tailored to provide explanations in debugging data system components, including the data itself. The explanation frameworks focus on explaining the root cause of a concurrent application\u27s intermittent failure and exposing issues in the data that cause a data-driven system to malfunction
Human Rights Treaty Commitment and Compliance: A Machine Learning-based Causal Inference Approach
Why do states ratify international human rights treaties? How much do human rights treaties influence state behaviors directly and indirectly? Why are some human rights treaty monitoring procedures more effective than others? What are the most predictively and causally important factors that can reduce and prevent state repression and human rights violations? This dissertation provide answers to these keys causal questions in political science research, using a novel approach that combines machine learning and the structural causal model framework. The four research questions are arranged in a chronological order that refects the causal process relating to international human rights treaties, going from (a) the causal determinants of treaty ratification to (b) the causal mechanisms of human rights treaties to (c) the causal effects of human rights treaty monitoring procedures to (d) other factors that causally influence human rights violations. Chapter 1 identifies the research traditions within which this dissertation is located, offers an overview of the methodological advances that enable this research, specifies the research questions, and previews the findings. Chapters 2, 3, 4, and 5 present in chronological order four empirical studies that answer these four research questions. Finally, Chapter 6 summarizes the substantive findings, suggests some other research questions that could be similarly investigated, and recaps the methodological approach and the contributions of the dissertation
Recommended from our members
Use of External Representations in Reasoning about Causality
This research investigated if diagrams aid in deductive reasoning with formal causal models. Four studies were conducted exploring participants' ability to discover causal paths, identify causes and effects, and create alternative explanations for variable relationships. In Study 1, abstract variables of the causal model were compared to contextually grounded variables and causal models presented as text or diagrams were compared. Participants given abstract diagrams did better in most tasks than participants in the other conditions, who all did similarly. Studies 2 and 3 compared causal models expressed in text to diagrammed causal models, and compared models using arrows to models using words when connecting variables. Participants who had arrowheads replaced with words made more errors than participants in other diagram conditions. Diagrammed causal models led to better performance than did other conditions, and there was no difference between different text models. Studies 4 and 5 tested the hypothesis that predictive reasoning (from cause to effect) is easier than diagnostic reasoning (from effect to cause). The two studies did not find any such effec
Mapping the intuitive investigation: Seeking, evaluating and explaining the evidence
The human mind has developed numerous cognitive tools to allow us to navigate the uncertainty of the world and make sense of situations and events. In this thesis I present a descriptive account of some of these tools by probing people’s ability to: evaluate, seek, and explain evidence and information. This was achieved by appraising people’s behaviour in controlled experiments – predominantly representing legal-investigative scenarios – utilising normative causal models (e.g., causal Bayesian networks), and uncovering the alternative strategies that people employed when reasoning under uncertainty.
In Chapter 4, I investigate people’s ability to engage in a pattern of reasoning termed ‘explaining away’ and propose, and find empirical support towards, intuitive theories that address why the observed inference errors were made. In Chapter 5, I outline how people search for, and evaluate, evidence in a sequential investigative information-seeking paradigm – finding that people do not seek information simply to maximize a given utility function but rather are driven by additional strategies which are sensitive to factors such as demands of the task and a novel form of risk aversion. I extend these findings to forensic professionals, and utilise a naturalistic study employing mobile eye-trackers during a mock crime scene investigation to elucidate the key role that ‘asking the right questions’ plays when engaging in sense-making practices ‘in the wild’.
In Chapter 6, I explore people’s preferences for certain types of information relating to opportunity and motive at various stages of the legal-investigative process. Here, I demonstrate that people prefer ‘motive’ accounts of crimes (analogous to a teleology preference) at different stages of the investigative process. In an additional two studies I demonstrate that these preferences are context-sensitive: namely, that ‘motive’ information tends to be moreincriminating and less exculpatory. In a final set of experiments, outlined in Chapter 7, I investigate how drawing causal models of competing explanations of the evidence affects how these same explanations are evaluated – arguing that graphically representing the evidence bolsters people’s understanding of the probabilistic and logical significance of the causal structures drawn.
In sum, this thesis provides a rich descriptive account of how people engage in various aspects of sense-making and decision-making under uncertainty. The work presented in this thesis ultimately aims to increase the ecological and descriptive validity of normative causal frameworks utilised in the cognitive sciences – whilst informing ways to formalise decision-making practices in real-world specialised domains
Design Science Epistemology. A pragmatist inquiry
This paper contributes to the clarification of a design science epistemology. It presents different epistemic types related to three stages of the design science process: 1) Evaluative and explanatory background knowledge (pre-design knowledge), 2) prospective knowledge with design hypotheses (in-design knowledge) and 3) prescriptive knowledge with design principles (post-design knowledge). The epistemological inquiry adopts a pragmatist approach and is pursued through a review of design science literature and informed by an empirical design case on digital support for social welfare allowances. The clarified design science epistemology shows a diversified epistemological landscape with several epistemic types: evaluative, critical, appreciative, normative, explanatory, prospective, prescriptive, categorial and attributive knowledge. Ways to express these epistemic types have been proposed in principal clauses. Ways of grounding have been clarified for each epistemic type. Proposals are given on how to utilize the design science epistemology in relation to design science process models and publication schemas
- …