Concolic Testing in Logic programming

Software testing is one of the most popular validation techniques in the software industry. Surprisingly, we can only find a few approaches to testing in the context of logic programming. In this paper, we introduce a systematic approach for dynamic testing that combines both concrete and symbolic execution. Our approach is fully automatic and guarantees full path coverage when it terminates. We prove some basic properties of our technique and illustrate its practical usefulness through a prototype implementation.This work has been partially supported by the EU (FEDER) and the Spanish Ministerio de Economia y Competitividad under grant TIN2013-44742-C4-1-R and by the Generalitat Valenciana under grant PROMETEOII/2015/013. Part of this research was done while the third author was visiting the University of Reunion; G. Vidal gratefully acknowledges their hospitality.Mesnard, F.; Payet, E.; Vidal Oriola, GF. (2015). Concolic Testing in Logic programming. Theory and Practice of Logic Programming. 15(4):711-725. https://doi.org/10.1017/S1471068415000332S711725154SCHIMPF, J., & SHEN, K. (2011). ECLiPSe – From LP to CLP. Theory and Practice of Logic Programming, 12(1-2), 127-156. doi:10.1017/s1471068411000469Martelli, A., & Montanari, U. (1982). An Efficient Unification Algorithm. ACM Transactions on Programming Languages and Systems, 4(2), 258-282. doi:10.1145/357162.357169Godefroid, P., Klarlund, N., & Sen, K. (2005). DART. Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation - PLDI ’05. doi:10.1145/1065010.1065036Mera E. , López-García P. , and Hermenegildo M. V. 2009. Integrating software testing and run-time checking in an assertion verification framework. In 25th International Conference on Logic Programming, ICLP 2009, Pasadena. 281–295.Godefroid, P., Levin, M. Y., & Molnar, D. (2012). SAGE. Communications of the ACM, 55(3), 40. doi:10.1145/2093548.2093564WIELEMAKER, J., SCHRIJVERS, T., TRISKA, M., & LAGER, T. (2011). SWI-Prolog. Theory and Practice of Logic Programming, 12(1-2), 67-96. doi:10.1017/s1471068411000494CARLSSON, M., & MILDNER, P. (2011). SICStus Prolog—The first 25 years. Theory and Practice of Logic Programming, 12(1-2), 35-66. doi:10.1017/s1471068411000482Degrave F. , Schrijvers T. , and Vanhoof W. 2008. Automatic generation of test inputs for Mercury. In Logic-Based Program Synthesis and Transformation, 18th International Symposium, LOPSTR 2008. 71–86.Somogyi, Z., Henderson, F., & Conway, T. (1996). The execution algorithm of mercury, an efficient purely declarative logic programming language. The Journal of Logic Programming, 29(1-3), 17-64. doi:10.1016/s0743-1066(96)00068-4Vidal, G. (2015). Concolic Execution and Test Case Generation in Prolog. Lecture Notes in Computer Science, 167-181. doi:10.1007/978-3-319-17822-6_10Belli F. and Jack O. 1993. Implementation-based analysis and testing of Prolog programs. In ISSTA. 70–80.King, J. C. (1976). Symbolic execution and program testing. Communications of the ACM, 19(7), 385-394. doi:10.1145/360248.360252Lloyd, J. W. (1987). Foundations of Logic Programming. doi:10.1007/978-3-642-83189-8Clarke L. 1976. A program testing system. In Proceedings of the 1976 Annual Conference (ACM'76). 488–491

Concolic Testing in CLP

[EN] Concolic testing is a popular software verification technique based on a combination of concrete and symbolic execution. Its main focus is finding bugs and generating test cases with the aim of maximizing code coverage. A previous approach to concolic testing in logic programming was not sound because it only dealt with positive constraints (by means of substitutions) but could not represent negative constraints. In this paper, we present a novel framework for concolic testing of CLP programs that generalizes the previous technique. In the CLP setting, one can represent both positive and negative constraints in a natural way, thus giving rise to a sound and (potentially) more efficient technique. Defining verification and testing techniques for CLP programs is increasingly relevant since this framework is becoming popular as an intermediate representation to analyze programs written in other programming paradigms.This author has been partially supported by EU (FEDER) and Spanish MCI/AEI under grants TIN2016-76843-C4-1-R and PID2019-104735RB-C41, and by the Generalitat Valenciana under grant Prometeo/2019/098 (DeepTrust).Mesnard, F.; Payet, E.; Vidal, G. (2020). Concolic Testing in CLP. Theory and Practice of Logic Programming. 20(5):671-686. https://doi.org/10.1017/S1471068420000216S67168620

An SMT-Based Concolic Testing Tool for Logic Programs

[EN] Concolic testing combines symbolic and concrete execution to generate test cases that achieve a good program coverage. Its benefits have been demonstrated for more than 15 years in the case of imperative programs. In this work, we present a concolic-based test generation tool for logic programs which exploits SMT-solving for constraint resolutionThird author is a research associate at FNRS that also supports this work (O05518FRG03). The last author is partially supported by the EU (FEDER) and the Spanish MCI/AEI under grants TIN2016-76843-C4-1-R/PID2019-104735RB-C41 and by the Generalitat Valenciana under grant Prometeo/2019/098 (DeepTrust)Fortz, S.; Mesnard, F.; Payet, E.; Perrouin, G.; Vanhoof, W.; Vidal, G. (2020). An SMT-Based Concolic Testing Tool for Logic Programs. Springer Nature. 215-219. https://doi.org/10.1007/978-3-030-59025-3_13S215219de Moura, L., Bjørner, N.: Z3: an efficient SMT solver. In: Ramakrishnan, C.R., Rehof, J. (eds.) TACAS 2008. LNCS, vol. 4963, pp. 337–340. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-78800-3_24Giantsios, A., Papaspyrou, N., Sagonas, K.: Concolic testing for functional languages. Sci. Comput. Program. 147, 109–134 (2017)Godefroid, P., Klarlund, N., Sen, K.: DART: directed automated random testing. In: Proceedings of PLDI 2005, pp. 213–223. ACM (2005)Mesnard, F., Payet, É., Vidal, G.: Concolic testing in logic programming. TPLP 15(4–5), 711–725 (2015). https://doi.org/10.1017/S1471068415000332Mesnard, F., Payet, É., Vidal, G.: On the completeness of selective unification in concolic testing of logic programs. In: Hermenegildo, M.V., Lopez-Garcia, P. (eds.) LOPSTR 2016. LNCS, vol. 10184, pp. 205–221. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-63139-4_12Mesnard, F., Payet, É., Vidal, G.: Selective unification in constraint logic programming. In: Vanhoof, W., Pientka, B. (eds.) PPDP, pp. 115–126. ACM (2017)Mesnard, F., Payet, É., Vidal, G.: Concolic Testing in CLP. CoRR abs/2008.00421 (2020). https://arxiv.org/abs/2008.00421Sen, K., Marinov, D., Agha, G.: CUTE: a concolic unit testing engine for C. In: ESEC/ FSE, pp. 263–272. ACM (2005)Ströder, T., Emmes, F., Schneider-Kamp, P., Giesl, J., Fuhs, C.: A linear operational semantics for termination and complexity analysis of ISO Prolog. In: Vidal, G. (ed.) LOPSTR 2011. LNCS, vol. 7225, pp. 237–252. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-32211-2_16Tikovsky, J.R.: Concolic testing of functional logic programs. In: Seipel, D., Hanus, M., Abreu, S. (eds.) WFLP/WLP/INAP -2017. LNCS (LNAI), vol. 10997, pp. 169–186. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00801-7_11Vidal, G.: Concolic execution and test case generation in prolog. In: Proietti, M., Seki, H. (eds.) LOPSTR 2014. LNCS, vol. 8981, pp. 167–181. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-17822-6_10Wielemaker, J., Schrijvers, T., Triska, M., Lager, T.: SWI-prolog. TPLP 12(1–2), 67–96 (2012). https://doi.org/10.1017/S147106841100049

Selective Unification in (Constraint) Logic Programming

[EN] Concolic testing is a well-known validation technique for imperative and object oriented programs. In a previous paper, we have introduced an adaptation of this technique to logic programming. At the heart of our framework lies a specific procedure that we call "selective unification". It is used to generate appropriate run-time goals by considering all possible ways an atom can unify with the heads of some program clauses. In this paper, we show that the existing algorithm for selective unification is not complete in the presence of non-linear atoms. We then prove soundness and completeness for a restricted version of the problem where some atoms are required to be linear. We also consider concolic testing in the context of constraint logic programming and extend the notion of selective unification accordingly.This work has been partially supported by the EU (FEDER) and the Spanish Ministerio de Ciencia, Innovacion y Universidades/AEI under grant TIN2016-76843-C4-1-R and by the Generalitat Valenciana under grant Prometeo/2019/098 (DeepTrust).Mesnard, F.; Payet, E.; Vidal, G. (2020). Selective Unification in (Constraint) Logic Programming. Fundamenta Informaticae. 177(3-4):359-383. https://doi.org/10.3233/FI-2020-1993S3593831773-

A Survey of Symbolic Execution Techniques

Many security and software testing applications require checking whether certain properties of a program hold for any possible usage scenario. For instance, a tool for identifying software vulnerabilities may need to rule out the existence of any backdoor to bypass a program's authentication. One approach would be to test the program using different, possibly random inputs. As the backdoor may only be hit for very specific program workloads, automated exploration of the space of possible inputs is of the essence. Symbolic execution provides an elegant solution to the problem, by systematically exploring many possible execution paths at the same time without necessarily requiring concrete inputs. Rather than taking on fully specified input values, the technique abstractly represents them as symbols, resorting to constraint solvers to construct actual instances that would cause property violations. Symbolic execution has been incubated in dozens of tools developed over the last four decades, leading to major practical breakthroughs in a number of prominent software reliability applications. The goal of this survey is to provide an overview of the main ideas, challenges, and solutions developed in the area, distilling them for a broad audience. The present survey has been accepted for publication at ACM Computing Surveys. If you are considering citing this survey, we would appreciate if you could use the following BibTeX entry: http://goo.gl/Hf5FvcComment: This is the authors pre-print copy. If you are considering citing this survey, we would appreciate if you could use the following BibTeX entry: http://goo.gl/Hf5Fv

Concolic Execution and Test Case Generation in Prolog

The final publication is available at Springer via http://dx.doi.org/10.1007/978-3-319-17822-6_10Symbolic execution extends concrete execution by allowing symbolic input data and then exploring all feasible execution paths. It has been defined and used in the context of many different programming languages and paradigms. A symbolic execution engine is at the heart of many program analysis and transformation techniques, like partial evaluation, test case generation or model checking, to name a few. Despite its relevance, traditional symbolic execution also suffers from several drawbacks. For instance, the search space is usually huge (often infinite) even for the simplest programs. Also, symbolic execution generally computes an overapproximation of the concrete execution space, so that false positives may occur. In this paper, we propose the use of a variant of symbolic execution, called concolic execution, for test case generation in Prolog. Our technique aims at full statement coverage. We argue that this technique computes an underapproximation of the concrete execution space (thus avoiding false positives) and scales up better to medium and large Prolog applications.This work has been partially supported by the EU (FEDER) and the Spanish Ministerio de Economía y Competitividad (Secretaría de Estado de Investigación, Desarrollo e Innovación) under grant TIN2013-44742-C4-1-R and by the Generalitat Valenciana under grant PROMETEO/2011/052.Vidal Oriola, GF. (2015). Concolic Execution and Test Case Generation in Prolog. En Logic-Based Program Synthesis and Transformation. Springer. 167-181. https://doi.org/10.1007/978-3-319-17822-6_10S167181Albert, E., Arenas, P., Gómez-Zamalloa, M., Rojas, J.M.: Test case generation by symbolic execution: basic concepts, a CLP-based instance, and actor-based concurrency. In: Bernardo, M., Damiani, F., Hähnle, R., Johnsen, E.B., Schaefer, I. (eds.) SFM 2014. LNCS, vol. 8483, pp. 263–309. Springer, Heidelberg (2014)Belli, F., Jack, O.: Implementation-based analysis and testing of Prolog programs. In: ISSTA, pp. 70–80. ACM (1993)Clarke, L.A.: A program testing system. In: Proceedings of the 1976 Annual Conference (ACM’76), Houston, pp. 488–491 (1976)De Schreye, D., Glück, R., Jørgensen, J., Leuschel, M., Martens, B., Sørensen, M.H.: Conjunctive partial deduction: foundations, control, algorithms, and experiments. J. Log. Program. 41(2&3), 231–277 (1999)Giesl, J., Ströder, T., Schneider-Kamp, P., Emmes, F., Fuhs, C.: Symbolic evaluation graphs and term rewriting: a general methodology for analyzing logic programs. In: PPDP’12, pp. 1–12. ACM (2012)Godefroid, P., Klarlund, N., Sen, K.: DART: directed automated random testing. In: Proceedings of PLDI’05, pp. 213–223. ACM (2005)Godefroid, P., Levin, M.Y., Molnar, D.A.: Sage: whitebox fuzzing for security testing. Commun. ACM 55(3), 40–44 (2012)Gómez-Zamalloa, M., Albert, E., Puebla, G.: Test case generation for object-oriented imperative languages in CLP. TPLP 10(4–6), 659–674 (2010)King, J.C.: Symbolic execution and program testing. Commun. ACM 19(7), 385–394 (1976)Leuschel, M.: The DPPD (Dozens of Problems for Partial Deduction) Library of Benchmarks. http://www.ecs.soton.ac.uk/mal/systems/dppd.html (2007)Lloyd, J.W.: Foundations of Logic Programming, 2nd edn. Springer, Berlin (1987)Lloyd, J.W., Shepherdson, J.C.: Partial evaluation in logic programming. J. Log. Program. 11, 217–242 (1991)Martens, B., Gallagher, J.: Ensuring global termination of partial deduction while allowing flexible polyvariance. In: Proceedings of ICLP’95, pp. 597–611. MIT Press (1995)Pasareanu, C.S., Rungta, N.: Symbolic PathFinder: symbolic execution of Java bytecode. In: Pecheur, C., Andrews, J., Di Nitto, E. (eds.) ASE, pp. 179–180. ACM (2010)Rojas, J.M., Gómez-Zamalloa, M.: A framework for guided test case generation in constraint logic programming. In: Albert, E. (ed.) Proceedings of LOPSTR. LNCS, vol. 7844, pp. 176–193. Springer, Heidelberg (2013)Sen, K., Marinov, D., Agha, G.: CUTE: a concolic unit testing engine for C. In: Proceedings of ESEC/SIGSOFT FSE 2005, pp. 263–272. ACM (2005)Ströder, T., Emmes, F., Schneider-Kamp, P., Giesl, J., Fuhs, C.: A linear operational semantics for termination and complexity analysis of $$\sf ISO$$ $$\sf Prolog$$ . In: Vidal, G. (ed.) LOPSTR’11. LNCS, vol. 7225, pp. 237–252. Springer, Heidelberg (2012