A Survey of Algorithmic Debugging

"© ACM, 2017. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in ACM Computing Surveys, {50, 4, 2017} https://dl.acm.org/doi/10.1145/3106740"[EN] Algorithmic debugging is a technique proposed in 1982 by E. Y. Shapiro in the context of logic programming. This survey shows how the initial ideas have been developed to become a widespread debugging schema ftting many diferent programming paradigms and with applications out of the program debugging feld. We describe the general framework and the main issues related to the implementations in diferent programming paradigms and discuss several proposed improvements and optimizations. We also review the main algorithmic debugger tools that have been implemented so far and compare their features. From this comparison, we elaborate a summary of desirable characteristics that should be considered when implementing future algorithmic debuggers.This work has been partially supported by the EU (FEDER) and the Spanish Ministerio de Economia y Competitividad under grant TIN2013-44742-C4-1-R, TIN2016-76843-C4-1-R, StrongSoft (TIN2012-39391-C04-04), and TRACES (TIN2015-67522-C3-3-R) by the Generalitat Valenciana under grant PROMETEO-II/2015/013 (SmartLogic) and by the Comunidad de Madrid project N-Greens Software-CM (S2013/ICE-2731).Caballero, R.; Riesco, A.; Silva, J. (2017). A Survey of Algorithmic Debugging. ACM Computing Surveys. 50(4):1-35. https://doi.org/10.1145/3106740S135504Abramson, D., Foster, I., Michalakes, J., & Sosič, R. (1996). Relative debugging. Communications of the ACM, 39(11), 69-77. doi:10.1145/240455.240475K. R. Apt H. A. Blair and A. Walker. 1988. Towards a theory of declarative knowledge. In Foundations of Deductive Databases and Logic Programming J. Minker (Ed.). Morgan Kaufmann Publishers Inc. San Francisco CA 89--148. 10.1016/B978-0-934613-40-8.50006-3 K. R. Apt H. A. Blair and A. Walker. 1988. Towards a theory of declarative knowledge. In Foundations of Deductive Databases and Logic Programming J. Minker (Ed.). Morgan Kaufmann Publishers Inc. San Francisco CA 89--148. 10.1016/B978-0-934613-40-8.50006-3Arora, T., Ramakrishnan, R., Roth, W. G., Seshadri, P., & Srivastava, D. (1993). Explaining program execution in deductive systems. Lecture Notes in Computer Science, 101-119. doi:10.1007/3-540-57530-8_7E. Av-Ron. 1984. Top-Down Diagnosis of Prolog Programs. Ph.D. Dissertation. Weizmann Institute. E. Av-Ron. 1984. Top-Down Diagnosis of Prolog Programs. Ph.D. Dissertation. Weizmann Institute.A. Beaulieu. 2005. Learning SQL. O’Reilly Farnham UK. A. Beaulieu. 2005. Learning SQL. O’Reilly Farnham UK.D. Binks. 1995. Declarative Debugging in Gödel. Ph.D. Dissertation. University of Bristol. D. Binks. 1995. Declarative Debugging in Gödel. Ph.D. Dissertation. University of Bristol.B. Braßel and H. Siegel. 2008. Debugging Lazy Functional Programs by Asking the Oracle. Springer-Verlag Berlin 183--200. DOI:http://dx.doi.org/10.1007/978-3-540-85373-2_11 10.1007/978-3-540-85373-2_11 B. Braßel and H. Siegel. 2008. Debugging Lazy Functional Programs by Asking the Oracle. Springer-Verlag Berlin 183--200. DOI:http://dx.doi.org/10.1007/978-3-540-85373-2_11 10.1007/978-3-540-85373-2_11Caballero, R. (2005). A declarative debugger of incorrect answers for constraint functional-logic programs. Proceedings of the 2005 ACM SIGPLAN workshop on Curry and functional logic programming - WCFLP ’05. doi:10.1145/1085099.1085102Caballero, R., García-Ruiz, Y., & Sáenz-Pérez, F. (2012). Declarative Debugging of Wrong and Missing Answers for SQL Views. Lecture Notes in Computer Science, 73-87. doi:10.1007/978-3-642-29822-6_9Caballero, R., García-Ruiz, Y., & Sáenz-Pérez, F. (2015). Debugging of wrong and missing answers for datalog programs with constraint handling rules. Proceedings of the 17th International Symposium on Principles and Practice of Declarative Programming - PPDP ’15. doi:10.1145/2790449.2790522Caballero, R., Martin-Martin, E., Riesco, A., & Tamarit, S. (2015). A zoom-declarative debugger for sequential Erlang programs. Science of Computer Programming, 110, 104-118. doi:10.1016/j.scico.2015.06.011Caballero, R., & Rodríguez-Artalejo, M. (2002). A Declarative Debugging System for Lazy Functional Logic Programs. Electronic Notes in Theoretical Computer Science, 64, 113-175. doi:10.1016/s1571-0661(04)80349-9Ceri, S., Gottlob, G., & Tanca, L. (1989). What you always wanted to know about Datalog (and never dared to ask). IEEE Transactions on Knowledge and Data Engineering, 1(1), 146-166. doi:10.1109/69.43410Chen, M., Mao, S., & Liu, Y. (2014). Big Data: A Survey. Mobile Networks and Applications, 19(2), 171-209. doi:10.1007/s11036-013-0489-0Chitil, O., & Davie, T. (2008). Comprehending finite maps for algorithmic debugging of higher-order functional programs. Proceedings of the 10th international ACM SIGPLAN symposium on Principles and practice of declarative programming - PPDP ’08. doi:10.1145/1389449.1389475Chitil, O., Faddegon, M., & Runciman, C. (2016). A Lightweight Hat. Proceedings of the 28th Symposium on the Implementation and Application of Functional Programming Languages - IFL 2016. doi:10.1145/3064899.3064904O. Chitil C. Runciman and M. Wallace. 2001. Freja Hat and Hood—A Comparative Evaluation of Three Systems for Tracing and Debugging Lazy Functional Programs. Springer Berlin 176--193. O. Chitil C. Runciman and M. Wallace. 2001. Freja Hat and Hood—A Comparative Evaluation of Three Systems for Tracing and Debugging Lazy Functional Programs. Springer Berlin 176--193.O. Chitil C. Runciman and Malcolm Wallace. 2003. Transforming Haskell for Tracing. Springer-Verlag Berlin 165--181. DOI:http://dx.doi.org/10.1007/3-540-44854-3_11 10.1007/3-540-44854-3_11 O. Chitil C. Runciman and Malcolm Wallace. 2003. Transforming Haskell for Tracing. Springer-Verlag Berlin 165--181. DOI:http://dx.doi.org/10.1007/3-540-44854-3_11 10.1007/3-540-44854-3_11Minh Ngoc Dinh, Abramson, D., & Chao Jin. (2014). Scalable Relative Debugging. IEEE Transactions on Parallel and Distributed Systems, 25(3), 740-749. doi:10.1109/tpds.2013.86Faddegon, M., & Chitil, O. (2015). Algorithmic debugging of real-world haskell programs: deriving dependencies from the cost centre stack. ACM SIGPLAN Notices, 50(6), 33-42. doi:10.1145/2813885.2737985Faddegon, M., & Chitil, O. (2016). Lightweight computation tree tracing for lazy functional languages. Proceedings of the 37th ACM SIGPLAN Conference on Programming Language Design and Implementation - PLDI 2016. doi:10.1145/2908080.2908104Ferrand, G. (1987). Error diagnosis in logic programming an adaptation of E.Y. Shapiro’s method. The Journal of Logic Programming, 4(3), 177-198. doi:10.1016/0743-1066(87)90001-xFritzson, P., Shahmehri, N., Kamkar, M., & Gyimothy, T. (1992). Generalized algorithmic debugging and testing. ACM Letters on Programming Languages and Systems, 1(4), 303-322. doi:10.1145/161494.161498Fromherz, M. P. J. (s. f.). Towards declarative debugging of concurrent constraint programs. Lecture Notes in Computer Science, 88-100. doi:10.1007/bfb0019403Harman, M., & Hierons, R. (2001). An overview of program slicing. Software Focus, 2(3), 85-92. doi:10.1002/swf.41F. Henderson T. Conway Z. Somogyi D. Jeffery P. Schachte S. Taylor C. Speirs T. Dowd R. Becket M. Brown and P. Wang. 2014. The Mercury Language Reference Manual (Version 14.01.1). The University of Melbourne. F. Henderson T. Conway Z. Somogyi D. Jeffery P. Schachte S. Taylor C. Speirs T. Dowd R. Becket M. Brown and P. Wang. 2014. The Mercury Language Reference Manual (Version 14.01.1). The University of Melbourne.C. Hermanns and H. Kuchen. 2013. Hybrid Debugging of Java Programs. Springer-Verlag Berlin 91--107. DOI:http://dx.doi.org/10.1007/978-3-642-36177-7_6 10.1007/978-3-642-36177-7_6 C. Hermanns and H. Kuchen. 2013. Hybrid Debugging of Java Programs. Springer-Verlag Berlin 91--107. DOI:http://dx.doi.org/10.1007/978-3-642-36177-7_6 10.1007/978-3-642-36177-7_6Hirunkitti, V., & Hogger, C. J. (s. f.). A generalised query minimisation for program debugging. Lecture Notes in Computer Science, 153-170. doi:10.1007/bfb0019407Hughes, J. (2010). Software Testing with QuickCheck. Lecture Notes in Computer Science, 183-223. doi:10.1007/978-3-642-17685-2_6G. Hutton. 2016. Programming in Haskell. Cambridge University Press Cambridge UK. G. Hutton. 2016. Programming in Haskell. Cambridge University Press Cambridge UK.Insa, D., & Silva, J. (2010). An algorithmic debugger for Java. 2010 IEEE International Conference on Software Maintenance. doi:10.1109/icsm.2010.5609661Insa, D., & Silva, J. (2011). Optimal Divide and Query. Lecture Notes in Computer Science, 224-238. doi:10.1007/978-3-642-24769-9_17Insa, D., & Silva, J. (2011). An optimal strategy for algorithmic debugging. 2011 26th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011). doi:10.1109/ase.2011.6100055D. Insa and J. Silva. 2011c. Scaling Up Algorithmic Debugging with Virtual Execution Trees. Springer-Verlag Berlin 149--163. DOI:http://dx.doi.org/10.1007/978-3-642-20551-4_10 10.1007/978-3-642-20551-4_10 D. Insa and J. Silva. 2011c. Scaling Up Algorithmic Debugging with Virtual Execution Trees. Springer-Verlag Berlin 149--163. DOI:http://dx.doi.org/10.1007/978-3-642-20551-4_10 10.1007/978-3-642-20551-4_10D. Insa and J. Silva. 2015a. Automatic transformation of iterative loops into recursive methods. Information 8 Software Technology 58 (2015) 95--109. DOI:http://dx.doi.org/10.1016/j.infsof.2014.10.001 10.1016/j.infsof.2014.10.001 D. Insa and J. Silva. 2015a. Automatic transformation of iterative loops into recursive methods. Information 8 Software Technology 58 (2015) 95--109. DOI:http://dx.doi.org/10.1016/j.infsof.2014.10.001 10.1016/j.infsof.2014.10.001Insa, D., & Silva, J. (2015). A Generalized Model for Algorithmic Debugging. Lecture Notes in Computer Science, 261-276. doi:10.1007/978-3-319-27436-2_16Insa, D., Silva, J., & Riesco, A. (2013). Speeding Up Algorithmic Debugging Using Balanced Execution Trees. Lecture Notes in Computer Science, 133-151. doi:10.1007/978-3-642-38916-0_8Insa, D., Silva, J., & Tomás, C. (2013). Enhancing Declarative Debugging with Loop Expansion and Tree Compression. Lecture Notes in Computer Science, 71-88. doi:10.1007/978-3-642-38197-3_6K. Jensen and N. Wirth. 1974. PASCAL User Manual and Report. Springer-Verlag Berlin. 10.1007/978-3-662-21554-8 K. Jensen and N. Wirth. 1974. PASCAL User Manual and Report. Springer-Verlag Berlin. 10.1007/978-3-662-21554-8Jia, Y., & Harman, M. (2011). An Analysis and Survey of the Development of Mutation Testing. IEEE Transactions on Software Engineering, 37(5), 649-678. doi:10.1109/tse.2010.62Kamkar, M., Shahmehri, N., & Fritzson, P. (s. f.). Bug localization by algorithmic debugging and program slicing. Lecture Notes in Computer Science, 60-74. doi:10.1007/bfb0024176S. Köhler B. Ludäscher and Y. Smaragdakis. 2012. Declarative Datalog Debugging for Mere Mortals. Springer-Verlag Berlin 111--122. S. Köhler B. Ludäscher and Y. Smaragdakis. 2012. Declarative Datalog Debugging for Mere Mortals. Springer-Verlag Berlin 111--122.Kouh, H.-J., & Yoo, W.-H. (2003). The Efficient Debugging System for Locating Logical Errors in Java Programs. Lecture Notes in Computer Science, 684-693. doi:10.1007/3-540-44839-x_72Benzmüller, C., & Miller, D. (2014). Automation of Higher-Order Logic. Handbook of the History of Logic, 215-254. doi:10.1016/b978-0-444-51624-4.50005-8Kowalski, R., & Kuehner, D. (1971). Linear resolution with selection function. Artificial Intelligence, 2(3-4), 227-260. doi:10.1016/0004-3702(71)90012-9K. Kuchcinski W. Drabent and J. Maluszynski. 1993. Automatic Diagnosis of VLSI Digital Circuits Using Algorithmic Debugging. Springer-Verlag Berlin 350--367. DOI:http://dx.doi.org/10.1007/BFb0019419 10.1007/BFb0019419 K. Kuchcinski W. Drabent and J. Maluszynski. 1993. Automatic Diagnosis of VLSI Digital Circuits Using Algorithmic Debugging. Springer-Verlag Berlin 350--367. DOI:http://dx.doi.org/10.1007/BFb0019419 10.1007/BFb0019419S. Liang. 1999. Java Native Interface: Programmer’s Guide and Reference (1st ed.). Addison-Wesley Longman Publishing Co. Inc. Boston MA. S. Liang. 1999. Java Native Interface: Programmer’s Guide and Reference (1st ed.). Addison-Wesley Longman Publishing Co. Inc. Boston MA.Lloyd, J. W. (1987). Declarative error diagnosis. New Generation Computing, 5(2), 133-154. doi:10.1007/bf03037396J. W. Lloyd. 1987b. Foundations of Logic Programming (2nd ed.). Springer-Verlag Berlin. 10.1007/978-3-642-83189-8 J. W. Lloyd. 1987b. Foundations of Logic Programming (2nd ed.). Springer-Verlag Berlin. 10.1007/978-3-642-83189-8W. Lux. 2006. Münster Curry User’s guide (Release 0.9.10 of May 10 2006). Retrieved from http://danae.uni-muenster.de/∼lux/curry/user.pdf. W. Lux. 2006. Münster Curry User’s guide (Release 0.9.10 of May 10 2006). Retrieved from http://danae.uni-muenster.de/∼lux/curry/user.pdf.Lux, W. (2008). Declarative Debugging Meets the World. Electronic Notes in Theoretical Computer Science, 216, 65-77. doi:10.1016/j.entcs.2008.06.034I. MacLarty. 2005. Practical Declarative Debugging of Mercury Programs. Ph.D. Dissertation. Department of Computer Science and Software Engineering The University of Melbourne. I. MacLarty. 2005. Practical Declarative Debugging of Mercury Programs. Ph.D. Dissertation. Department of Computer Science and Software Engineering The University of Melbourne.Naganuma, J., Ogura, T., & Hoshino, T. (s. f.). High-level design validation using algorithmic debugging. Proceedings of European Design and Test Conference EDAC-ETC-EUROASIC. doi:10.1109/edtc.1994.326833Naish, L. (1992). Declarative diagnosis of missing answers. New Generation Computing, 10(3), 255-285. doi:10.1007/bf03037939H. Nilsson. 1998. Declarative Debugging for Lazy Functional Languages. Ph.D. Dissertation. Linköping Sweden. H. Nilsson. 1998. Declarative Debugging for Lazy Functional Languages. Ph.D. Dissertation. Linköping Sweden.NILSSON, H. (2001). How to look busy while being as lazy as ever: the Implementation of a lazy functional debugger. Journal of Functional Programming, 11(6), 629-671. doi:10.1017/s095679680100418xNilsson, H., & Fritzson, P. (s. f.). Algorithmic debugging for lazy functional languages. Lecture Notes in Computer Science, 385-399. doi:10.1007/3-540-55844-6_149Nilsson, H., & Fritzson, P. (1994). Algorithmic debugging for lazy functional languages. Journal of Functional Programming, 4(3), 337-369. doi:10.1017/s095679680000109xNilsson, H., & Sparud, J. (1997). Automated Software Engineering, 4(2), 121-150. doi:10.1023/a:1008681016679Ostrand, T. J., & Balcer, M. J. (1988). The category-partition method for specifying and generating fuctional tests. Communications of the ACM, 31(6), 676-686. doi:10.1145/62959.62964Pereira, L. M. (1986). Rational debugging in logic programming. Third International Conference on Logic Programming, 203-210. doi:10.1007/3-540-16492-8_76B. Pope. 2006. A Declarative Debugger for Haskell. Ph.D. Dissertation. The University of Melbourne Australia. B. Pope. 2006. A Declarative Debugger for Haskell. Ph.D. Dissertation. The University of Melbourne Australia.Ramakrishnan, R., & Ullman, J. D. (1995). A survey of deductive database systems. The Journal of Logic Programming, 23(2), 125-149. doi:10.1016/0743-1066(94)00039-9Riesco, A., Verdejo, A., Martí-Oliet, N., & Caballero, R. (2012). Declarative debugging of rewriting logic specifications. The Journal of Logic and Algebraic Programming, 81(7-8), 851-897. doi:10.1016/j.jlap.2011.06.004DeRose, L., Gontarek, A., Vose, A., Moench, R., Abramson, D., Dinh, M. N., & Jin, C. (2015). Relative debugging for a highly parallel hybrid computer system. Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis on - SC ’15. doi:10.1145/2807591.2807605Runeson, P. (2006). A survey of unit testing practices. IEEE Software, 23(4), 22-29. doi:10.1109/ms.2006.91Russo, F., & Sancassani, M. (1992). A declarative debugging environment for DATALOG. Lecture Notes in Computer Science, 433-441. doi:10.1007/3-540-55460-2_32E. Y. Shapiro. 1982a. Algorithmic Program Debugging. MIT Press Cambridge MA. E. Y. Shapiro. 1982a. Algorithmic Program Debugging. MIT Press Cambridge MA.Shapiro, E. Y. (1982). Algorithmic program diagnosis. Proceedings of the 9th ACM SIGPLAN-SIGACT symposium on Principles of programming languages - POPL ’82. doi:10.1145/582153.582185Shmueli, O., & Tsur, S. (1991). Logical diagnosis ofLDL programs. New Generation Computing, 9(3-4), 277-303. doi:10.1007/bf03037166Silva, J. (s. f.). A Comparative Study of Algorithmic Debugging Strategies. Lecture Notes in Computer Science, 143-159. doi:10.1007/978-3-540-71410-1_11Silva, J. (2011). A survey on algorithmic debugging strategies. Advances in Engineering Software, 42(11), 976-991. doi:10.1016/j.advengsoft.2011.05.024Silva, J., & Chitil, O. (2006). Combining algorithmic debugging and program slicing. Proceedings of the 8th ACM SIGPLAN symposium on Principles and practice of declarative programming - PPDP ’06. doi:10.1145/1140335.1140355J. A. Silva E. R. Faria R. C. Barros E. R. Hruschka A. C. P. L. F. de Carvalho and J. Gama. 2013. Data stream clustering: A survey. Comput. Surv. 46 1 Article 13 (July 2013) 31 pages.DOI:http://dx.doi.org/10.1145/2522968.2522981 10.1145/2522968.2522981 J. A. Silva E. R. Faria R. C. Barros E. R. Hruschka A. C. P. L. F. de Carvalho and J. Gama. 2013. Data stream clustering: A survey. Comput. Surv. 46 1 Article 13 (July 2013) 31 pages.DOI:http://dx.doi.org/10.1145/2522968.2522981 10.1145/2522968.2522981SOSIČ, R., & ABRAMSON, D. (1997). Guard: A Relative Debugger. Software: Practice and Experience, 27(2), 185-206. doi:10.1002/(sici)1097-024x(199702)27:23.0.co;2-dL. Sterling and E. Shapiro. 1986. The Art of Prolog: Advanced Programming Techniques. The MIT Press Cambridge MA. L. Sterling and E. Shapiro. 1986. The Art of Prolog: Advanced Programming Techniques. The MIT Press Cambridge MA.P. Kambam Sugavanam. 2013. Debugging Framework for Attribute Grammars. Ph.D. Dissertation. University of Minnesota. P. Kambam Sugavanam. 2013. Debugging Framework for Attribute Grammars. Ph.D. Dissertation. University of Minnesota.Tamarit, S., Riesco, A., Martin-Martin, E., & Caballero, R. (2016). Debugging Meets Testing in Erlang. Lecture Notes in Computer Science, 171-180. doi:10.1007/978-3-319-41135-4_10A. Tessier and G. Ferrand. 2000. Declarative diagnosis in the CLP scheme. In Analysis and Visualization Tools for Constraint Programming: Constraint Debugging Pierre Deransart Manuel V. Hermenegildo and Jan Maluszynski (Eds.). Springer-Verlag Berlin 151--174. 10.1007/10722311_6 A. Tessier and G. Ferrand. 2000. Declarative diagnosis in the CLP scheme. In Analysis and Visualization Tools for Constraint Programming: Constraint Debugging Pierre Deransart Manuel V. Hermenegildo and Jan Maluszynski (Eds.). Springer-Verlag Berlin 151--174. 10.1007/10722311_6Zinn, C. (2013). Algorithmic Debugging for Intelligent Tutoring: How to Use Multiple Models and Improve Diagnosis. Lecture Notes in Computer Science, 272-283. doi:10.1007/978-3-642-40942-4_24Zinn, C. (2014). Algorithmic Debugging and Literate Programming to Generate Feedback in Intelligent Tutoring Systems. KI 2014: Advances in Artificial Intelligence, 37-48. doi:10.1007/978-3-319-11206-0_

Provenance, Incremental Evaluation, and Debugging in Datalog

The Datalog programming language has recently found increasing traction in research and industry. Driven by its clean declarative semantics, along with its conciseness and ease of use, Datalog has been adopted for a wide range of important applications, such as program analysis, graph problems, and networking. To enable this adoption, modern Datalog engines have implemented advanced language features and high-performance evaluation of Datalog programs. Unfortunately, critical infrastructure and tooling to support Datalog users and developers are still missing. For example, there are only limited tools addressing the crucial debugging problem, where developers can spend up to 30% of their time finding and fixing bugs. This thesis addresses Datalog’s tooling gaps, with the ultimate goal of improving the productivity of Datalog programmers. The first contribution is centered around the critical problem of debugging: we develop a new debugging approach that explains the execution steps taken to produce a faulty output. Crucially, our debugging method can be applied for large-scale applications without substantially sacrificing performance. The second contribution addresses the problem of incremental evaluation, which is necessary when program inputs change slightly, and results need to be recomputed. Incremental evaluation allows this recomputation to happen more efficiently, without discarding the previous results and recomputing from scratch. Finally, the last contribution provides a new incremental debugging approach that identifies the root causes of faulty outputs that occur after an incremental evaluation. Incremental debugging focuses on the relationship between input and output and can provide debugging suggestions to amend the inputs so that faults no longer occur. These techniques, in combination, form a corpus of critical infrastructure and tooling developments for Datalog, allowing developers and users to use Datalog more productively

Extending Provenance For Deep Diagnosis Of Distributed Systems

Diagnosing and repairing problems in complex distributed systems has always been challenging. A wide variety of problems can happen in distributed systems: routers can be misconfigured, nodes can be hacked, and the control software can have bugs. This is further complicated by the complexity and scale of today’s distributed systems. Provenance is an attractive way to diagnose faults in distributed systems, because it can track the causality from a symptom to a set of root causes. Prior work on network provenance has successfully applied provenance to distributed systems. However, they cannot explain problems beyond the presence of faulty events and offer limited help with finding repairs. In this dissertation, we extend provenance to handle diagnostics problems that require deeper investigations. We propose three different extensions: negative provenance explains not just the presence but also the absence of events (such as missing packets); meta provenance can suggest repairs by tracking causality not only for data but also for code (such as bugs in control plane programs); temporal provenance tracks causality at the temporal level and aims at diagnosing timing-related faults (such as slow requests). Compared to classical network provenance, our approach tracks richer causality at runtime and applies more sophisticated reasoning and post-processing. We apply the above techniques to software-defined networking and the border gateway protocol. Evaluations with real world traffic and topology show that our systems can diagnose and repair practical problems, and that the runtime overhead as well as the query turnarounds are reasonable

First-Order Models for Configuration Analysis

Our world teems with networked devices. Their configuration exerts an ever-expanding influence on our daily lives. Yet correctly configuring systems, networks, and access-control policies is notoriously difficult, even for trained professionals. Automated static analysis techniques provide a way to both verify a configuration\u27s correctness and explore its implications. One such approach is scenario-finding: showing concrete scenarios that illustrate potential (mis-)behavior. Scenarios even have a benefit to users without technical expertise, as concrete examples can both trigger and improve users\u27 intuition about their system. This thesis describes a concerted research effort toward improving scenario-finding tools for configuration analysis. We developed Margrave, a scenario-finding tool with special features designed for security policies and configurations. Margrave is not tied to any one specific policy language; rather, it provides an intermediate input language as expressive as first-order logic. This flexibility allows Margrave to reason about many different types of policy. We show Margrave in action on Cisco IOS, a common language for configuring firewalls, demonstrating that scenario-finding with Margrave is useful for debugging and validating real-world configurations. This thesis also presents a theorem showing that, for a restricted subclass of first-order logic, if a sentence is satisfiable then there must exist a satisfying scenario no larger than a computable bound. For such sentences scenario-finding is complete: one can be certain that no scenarios are missed by the analysis, provided that one checks up to the computed bound. We demonstrate that many common configurations fall into this subclass and give algorithmic tests for both sentence membership and counting. We have implemented both in Margrave. Aluminum is a tool that eliminates superfluous information in scenarios and allows users\u27 goals to guide which scenarios are displayed. We quantitatively show that our methods of scenario-reduction and exploration are effective and quite efficient in practice. Our work on Aluminum is making its way into other scenario-finding tools. Finally, we describe FlowLog, a language for network programming that we created with analysis in mind. We show that FlowLog can express many common network programs, yet demonstrate that automated analysis and bug-finding for FlowLog are both feasible as well as complete

Scalable integration of uncertainty reasoning and semantic web technologies

In recent years formal logical standards for knowledge representation to model real world knowledge and domains and make them accessible for computers gained a lot of trac- tion. They provide an expressive logical framework for modeling, consistency checking, reasoning, and query answering, and have proven to be versatile methods to capture knowledge of various fields. Those formalisms and methods focus on specifying knowl- edge as precisely as possible. At the same time, many applications in particular on the Semantic Web have to deal with uncertainty in their data; and handling uncertain knowledge is crucial in many real- world domains. However, regular logic is unable to capture the real-world properly due to its inherent complexity and uncertainty, all the while handling uncertain or incomplete information is getting more and more important in applications like expert system, data integration or information extraction. The overall objective of this dissertation is to identify scenarios and datasets where methods that incorporate their inherent uncertainty improve results, and investigate approaches and tools that are suitable for the respective task. In summary, this work is set out to tackle the following objectives: 1. debugging uncertain knowledge bases in order to generate consistent knowledge graphs to make them accessible for logical reasoning, 2. combining probabilistic query answering and logical reasoning which in turn uses these consistent knowledge graphs to answer user queries, and 3. employing the aforementioned techniques to the problem of risk management in IT infrastructures, as a concrete real-world application. We show that in all those scenarios, users can benefit from incorporating uncertainty in the knowledge base. Furthermore, we conduct experiments that demonstrate the real- world scalability of the demonstrated approaches. Overall, we argue that integrating uncertainty and logical reasoning, despite being theoretically intractable, is feasible in real-world application and warrants further research

Approximate Assertional Reasoning Over Expressive Ontologies

In this thesis, approximate reasoning methods for scalable assertional reasoning are provided whose computational properties can be established in a well-understood way, namely in terms of soundness and completeness, and whose quality can be analyzed in terms of statistical measurements, namely recall and precision. The basic idea of these approximate reasoning methods is to speed up reasoning by trading off the quality of reasoning results against increased speed