Recently we have proposed symbolic execution techniques for the probabilistic analysis of programs. These techniques seek to quan- tify the probability of a program to satisfy a property of interest under a relevant usage profile. We describe recent advances in prob- abilistic symbolic analysis including handling of complex floating- point  constraints  and  nondeterminism,  and  the  use  of  statistical techniques for increased scalability

Filieri, A

Pasareanu, CS

Spiral - Imperial College Digital Repository

On the Probabilistic Symbolic Analysis of ProgramsAntonio FilieriUniversity of Stuttgart, Germanyfilieri@informatik.uni-stuttgart.deCorina S. PăsăreanuCarnegie Mellon Silicon Valley, NASA Ames, USAcorina.s.pasareanu@nasa.govAbstractRecently we have proposed symbolic execution techniques for theprobabilistic analysis of programs. These techniques seek to quan-tify the probability of a program to satisfy a property of interestunder a relevant usage profile. We describe recent advances in prob-abilistic symbolic analysis including handling of complex floating-point constraints and nondeterminism, and the use of statisticaltechniques for increased scalability.Categories and Subject Descriptors D.2.4 [Software Engineer-ing]: Software/Program VerificationKeywords Probabilistic Symbolic Execution, Nondeterminism1. Probabilistic Symbolic AnalysisProbabilistic software analysis aims at quantifying the probabilitythat a software system satisfies a given property. This analysis hasbeen traditionally performed through probabilistic model check-ing [1] and usually applied on architecture-level abstractions of thesoftware. This allows performing quantitative analysis during earlystages of design, but not, in general, on code artifacts. In the lastfew years, Probabilistic Symbolic Execution (PSE) [2, 5, 6, 11] hasbeen proposed to bring probabilistic analysis directly at code level,with a promising applicability scope.Symbolic Execution [8] is a program analysis technique thatexecutes programs on unspecified inputs, by using symbolic inputsinstead of concrete data. The state of a symbolically executed pro-gram is defined by the (symbolic) values of program variables, apath condition (PC), and a program counter. The PC is a (quantifier-free) boolean formula over the symbolic inputs; it accumulates con-straints that concrete inputs must satisfy for an execution to followthe associated path. The program counter identifies the next state-ment to execute. A symbolic execution tree characterizes the exe-cution paths followed during the symbolic execution of a program.The nodes of the tree represent program states and the arcs the tran-sitions between states due to the execution of an instruction. To dealwith non termination, we limit the length of executions.Probabilistic Symbolic Execution enhances symbolic executionallowing to quantify the probability for a PC to be satisfied byan input value, according to a given usage profile (a differentthough equivalent definition is given in [11]). This new featurePermission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for components of this work owned by others than ACMmust be honored. Abstracting with credit is permitted. To copy otherwise, or republish,to post on servers or to redistribute to lists, requires prior specific permission and/or afee. Request permissions from permissions@acm.org.CONF ’yy, Month d–d, 20yy, City, ST, Country.Copyright c© 20yy ACM 978-1-nnnn-nnnn-n/yy/mm. . . $15.00.http://dx.doi.org/10.1145/nnnnnnn.nnnnnnncan be exploited to perform multiple types of analysis, including:computing the probability of a program to satisfy a property φas the sum of the probabilities of satisfying the PCs associatedto execution paths leading to the satisfaction of φ; assessing therelevance of an execution path for the behavior of the softwareunder the current usage profile (for example, to evaluate the qualityof a test suite or to drive improvement/refactoring decisions).SE QuantificationPCsProbabilistic Program AnalysisProbabilityTrue/FalsePropertyProgramUsage profileFigure 1. Probabilistic program analysis chain.Figure 1 shows the basic workflow of probabilistic programanalysis, as introduced in [5]. The inputs are the source code of theprogram, the usage profile, and a target property. The program isfirst executed symbolically to collect the PCs of the execution pathsleading to the satisfaction of the target property. Then, the solutionspace of these PCs is quantified in order to either compute theprobability for the property to be satisfied under the provided usageprofile, or to verify such probability meets prescribed constraints.Notice that the usage profile is used to both prune out unfeasibleexecution paths during symbolic execution and to compute theprobability of satisfying the target property.2. The Quantification ProblemQuantifying the probability of satisfying a path condition, undera given usage profile, is in general a hard task. If the input do-main is a bounded, albeit very large, subset of the integer num-bers, and the constraints appearing in the PCs are limited to lin-ear algebraic ones (or over restricted classes of polynomials), ef-ficient counting techniques have been developed, based on Barvi-nok’s algorithm [4]. For linear algebraic constraints over floating-point numbers, a bounding algorithm has been proposed in [11] tocompute arbitrarily small intervals containing the actual probabil-ity of satisfying the target property. Arbitrarily complex finitizeddata structures can in principle be handled with Korat [5], whichperforms a “smart” enumeration of the instances satisfying com-plex constraints. However, despite its generality, such enumerationmay still be infeasible for large domains [5]. However none of thesetechniques can deal with non-linear constraints over floating-pointdomains.To deal with arbitrarily complex numerical constraints, in [2]we proposed a compositional approximate quantification method,suitable for both integers and floating-point numbers, built uponMonte Carlo integration. The latter is notoriously one of the mostbroadly applicable methods, since based on simulation. However,the complexity of simulation might be overwhelming, requiringan intractable amount of time to quantify over large domains andcomplex constraints. Furthermore, to achieve a reasonable accuracya very large number of samples might be required, especially whendealing with fairly irregular functions like the boolean functionsevaluating complex conditions. To handle these limitations, weimproved naive Monte Carlo integration in two directions.The first direction targets mostly efficiency and is a divide andconquer strategy aiming at identify independent clauses in a PC thatcan be analyzed independently, similarly to [5]. For each clause theresult of integration is estimated, as well as its accuracy in terms ofthe variance of the corresponding estimator. The local results arethen combined according to analytical composition rules so to ob-tain a global estimate for the integration of the whole PC. The localuncertainty is propagated as well, producing a conservative assess-ment of the quality of the results. The outcome is twofold. First, thelocal results for independent clauses can be heavily reused [2, 5]with a significant speedup especially for large programs. Second,the independent clauses usually predicate only on a subset of theprogram variables. This reduces the dimensionality of the problemresulting in faster and more accurate local integrations.The second targets mostly accuracy and involves the use of anInterval Constraints Solver (ICP) [7] to partition the integration do-main into a finite set of boxes. If a box contains no solutions, theresult of integration, restricted to that box, will be constantly 0.Analogously, if all the points in a box are solutions of the constraintunder analysis, the result of integration will be the volume of thebox. In these two cases the variance of the Monte Carlo estimatorswithin the boxes is 0, meaning the result has no uncertainty. For theother boxes, a non-zero variance will be returned. The results of thevarious boxes can be combined, as specified in statistic for strati-fied sampling. Depending on the ability of ICP to identify boxescontaining either only or no solutions, the accuracy of integrationmay significantly increase. This is often the case for Software [2].3. Dealing with NondeterminismThe execution of nondeterministic programs does not depend onlyon its inputs but also on the used scheduler, i.e. the componentdeciding which actions to take to resolve nondeterministic choices(e.g. thread scheduling). Fixed a scheduler, execution is completelycharacterized by its symbolic execution tree.In [5], we dealt with nondeterminism by computing the best(worst) linear schedule (i.e. ordered sequence of thread choices),which maximizes (minimizes) the probability of the program tosatisfy a target property φ, for a given usage profile. However,linear schedules are in general not enough to design completeschedulers for an optimal execution of the program and tree-likeschedulers are needed.In later work [10], we investigate probabilistic software analysisfor nondeterministic programs. In particular, we tackle the problemof synthesizing an optimal scheduler providing formal guaranteeson the probability of satisfying φ. This scheduler is a complete tree-shaped probabilistic decision maker, i.e. it assigns a probability ofbeing selected to all the valid alternatives for resolving a nondeter-ministic choice. This generalizes linear schedulers from [5] and fol-lows classical work on Markov Decision Processes (MDPs) [1, 9]but adapted to the setting of analysis of programs, not models, andtailored to the tree shape of the symbolic .In [10] we study exact and approximate algorithms for the prob-lem, combining the peculiarity of PSE, sampling, and reinforce-ment learning to effectively trade scalability and accuracy off. Weshow experimentally that we significantly improve over the state-of-the-art statistical model checking for MDPs from [3].4. Current and Future PlansWe are currently working on both introducing more efficient quan-tification procedures for common types of constraints (includingstrings and complex data structures) and on providing a better sup-port for special kinds of usage profiles, including stateful ones,needed for example to analyze reactive systems. We are also in-vestigating in depth statistical techniques as a way of improvingthe scalability of our approach.AcknowledgmentsThe research described in this work has been conducted in collab-oration with Mateus Borges, Marcelo D’Amorin, Matthew Dwyer,Kasper Luckow, and Willem Visser.References[1] C. Baier, J.-P. Katoen, et al. Principles of model checking. MIT pressCambridge, 2008.[2] M. Borges, A. Filieri, M. d’Amorim, C. S. Păsăreanu, and W. Visser.Compositional solution space quantification for probabilistic softwareanalysis. In PLDI, 2014 – to appear.[3] E. Clarke and P. Zuliani. Statistical model checking for cyber-physicalsystems. In ATVA, pages 1–12, Oct. 2011.[4] J. A. De Loera, R. Hemmecke, J. Tauzer, and R. Yoshida. Effective lat-tice point counting in rational convex polytopes. Journal of SymbolicComputation, 38(4):1273–1302, Oct. 2004.[5] A. Filieri, C. S. Păsăreanu, and W. Visser. Reliability analysis inSymbolic PathFinder. In ICSE, pages 622–631, July 2013.[6] J. Geldenhuys, M. B. Dwyer, and W. Visser. Probabilistic symbolicexecution. In ISSTA, pages 166–176, July 2012.[7] L. Granvilliers and F. Benhamou. Algorithm 852: Realpaver: an inter-val solver using constraint satisfaction techniques. ACM Transactionson Mathematical Software, 32:138–156, 2006.[8] J. C. King. Symbolic execution and program testing. Commun. ACM,19(7):385–394, July 1976.[9] A. Kolobov. Planning with Markov Decision Processes: An AI Per-spective. Morgan and Claypool, 2012.[10] K. Luckow, C. S. Păsăreanu, M. Dwyer, A. Filieri, and W. Visser.Probabilistic symbolic execution for nondeterministic programs. InCAV, 2014 – submitted.[11] S. Sankaranarayanan, A. Chakarov, and S. Gulwani. Static analysisfor probabilistic programs: inferring whole program properties fromfinitely many paths. In PLDI, pages 447–458, June 2013.

On the probabilistic symbolic analysis of programs

Abstract

Similar works

Full text

Available Versions

Spiral - Imperial College Digital Repository