Concolic Testing in Logic programming

Software testing is one of the most popular validation techniques in the software industry. Surprisingly, we can only find a few approaches to testing in the context of logic programming. In this paper, we introduce a systematic approach for dynamic testing that combines both concrete and symbolic execution. Our approach is fully automatic and guarantees full path coverage when it terminates. We prove some basic properties of our technique and illustrate its practical usefulness through a prototype implementation.This work has been partially supported by the EU (FEDER) and the Spanish Ministerio de Economia y Competitividad under grant TIN2013-44742-C4-1-R and by the Generalitat Valenciana under grant PROMETEOII/2015/013. Part of this research was done while the third author was visiting the University of Reunion; G. Vidal gratefully acknowledges their hospitality.Mesnard, F.; Payet, E.; Vidal Oriola, GF. (2015). Concolic Testing in Logic programming. Theory and Practice of Logic Programming. 15(4):711-725. https://doi.org/10.1017/S1471068415000332S711725154SCHIMPF, J., & SHEN, K. (2011). ECLiPSe – From LP to CLP. Theory and Practice of Logic Programming, 12(1-2), 127-156. doi:10.1017/s1471068411000469Martelli, A., & Montanari, U. (1982). An Efficient Unification Algorithm. ACM Transactions on Programming Languages and Systems, 4(2), 258-282. doi:10.1145/357162.357169Godefroid, P., Klarlund, N., & Sen, K. (2005). DART. Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation - PLDI ’05. doi:10.1145/1065010.1065036Mera E. , López-García P. , and Hermenegildo M. V. 2009. Integrating software testing and run-time checking in an assertion verification framework. In 25th International Conference on Logic Programming, ICLP 2009, Pasadena. 281–295.Godefroid, P., Levin, M. Y., & Molnar, D. (2012). SAGE. Communications of the ACM, 55(3), 40. doi:10.1145/2093548.2093564WIELEMAKER, J., SCHRIJVERS, T., TRISKA, M., & LAGER, T. (2011). SWI-Prolog. Theory and Practice of Logic Programming, 12(1-2), 67-96. doi:10.1017/s1471068411000494CARLSSON, M., & MILDNER, P. (2011). SICStus Prolog—The first 25 years. Theory and Practice of Logic Programming, 12(1-2), 35-66. doi:10.1017/s1471068411000482Degrave F. , Schrijvers T. , and Vanhoof W. 2008. Automatic generation of test inputs for Mercury. In Logic-Based Program Synthesis and Transformation, 18th International Symposium, LOPSTR 2008. 71–86.Somogyi, Z., Henderson, F., & Conway, T. (1996). The execution algorithm of mercury, an efficient purely declarative logic programming language. The Journal of Logic Programming, 29(1-3), 17-64. doi:10.1016/s0743-1066(96)00068-4Vidal, G. (2015). Concolic Execution and Test Case Generation in Prolog. Lecture Notes in Computer Science, 167-181. doi:10.1007/978-3-319-17822-6_10Belli F. and Jack O. 1993. Implementation-based analysis and testing of Prolog programs. In ISSTA. 70–80.King, J. C. (1976). Symbolic execution and program testing. Communications of the ACM, 19(7), 385-394. doi:10.1145/360248.360252Lloyd, J. W. (1987). Foundations of Logic Programming. doi:10.1007/978-3-642-83189-8Clarke L. 1976. A program testing system. In Proceedings of the 1976 Annual Conference (ACM'76). 488–491

Concolic Testing in CLP

[EN] Concolic testing is a popular software verification technique based on a combination of concrete and symbolic execution. Its main focus is finding bugs and generating test cases with the aim of maximizing code coverage. A previous approach to concolic testing in logic programming was not sound because it only dealt with positive constraints (by means of substitutions) but could not represent negative constraints. In this paper, we present a novel framework for concolic testing of CLP programs that generalizes the previous technique. In the CLP setting, one can represent both positive and negative constraints in a natural way, thus giving rise to a sound and (potentially) more efficient technique. Defining verification and testing techniques for CLP programs is increasingly relevant since this framework is becoming popular as an intermediate representation to analyze programs written in other programming paradigms.This author has been partially supported by EU (FEDER) and Spanish MCI/AEI under grants TIN2016-76843-C4-1-R and PID2019-104735RB-C41, and by the Generalitat Valenciana under grant Prometeo/2019/098 (DeepTrust).Mesnard, F.; Payet, E.; Vidal, G. (2020). Concolic Testing in CLP. Theory and Practice of Logic Programming. 20(5):671-686. https://doi.org/10.1017/S1471068420000216S67168620

An SMT-Based Concolic Testing Tool for Logic Programs

[EN] Concolic testing combines symbolic and concrete execution to generate test cases that achieve a good program coverage. Its benefits have been demonstrated for more than 15 years in the case of imperative programs. In this work, we present a concolic-based test generation tool for logic programs which exploits SMT-solving for constraint resolutionThird author is a research associate at FNRS that also supports this work (O05518FRG03). The last author is partially supported by the EU (FEDER) and the Spanish MCI/AEI under grants TIN2016-76843-C4-1-R/PID2019-104735RB-C41 and by the Generalitat Valenciana under grant Prometeo/2019/098 (DeepTrust)Fortz, S.; Mesnard, F.; Payet, E.; Perrouin, G.; Vanhoof, W.; Vidal, G. (2020). An SMT-Based Concolic Testing Tool for Logic Programs. Springer Nature. 215-219. https://doi.org/10.1007/978-3-030-59025-3_13S215219de Moura, L., Bjørner, N.: Z3: an efficient SMT solver. In: Ramakrishnan, C.R., Rehof, J. (eds.) TACAS 2008. LNCS, vol. 4963, pp. 337–340. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-78800-3_24Giantsios, A., Papaspyrou, N., Sagonas, K.: Concolic testing for functional languages. Sci. Comput. Program. 147, 109–134 (2017)Godefroid, P., Klarlund, N., Sen, K.: DART: directed automated random testing. In: Proceedings of PLDI 2005, pp. 213–223. ACM (2005)Mesnard, F., Payet, É., Vidal, G.: Concolic testing in logic programming. TPLP 15(4–5), 711–725 (2015). https://doi.org/10.1017/S1471068415000332Mesnard, F., Payet, É., Vidal, G.: On the completeness of selective unification in concolic testing of logic programs. In: Hermenegildo, M.V., Lopez-Garcia, P. (eds.) LOPSTR 2016. LNCS, vol. 10184, pp. 205–221. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-63139-4_12Mesnard, F., Payet, É., Vidal, G.: Selective unification in constraint logic programming. In: Vanhoof, W., Pientka, B. (eds.) PPDP, pp. 115–126. ACM (2017)Mesnard, F., Payet, É., Vidal, G.: Concolic Testing in CLP. CoRR abs/2008.00421 (2020). https://arxiv.org/abs/2008.00421Sen, K., Marinov, D., Agha, G.: CUTE: a concolic unit testing engine for C. In: ESEC/ FSE, pp. 263–272. ACM (2005)Ströder, T., Emmes, F., Schneider-Kamp, P., Giesl, J., Fuhs, C.: A linear operational semantics for termination and complexity analysis of ISO Prolog. In: Vidal, G. (ed.) LOPSTR 2011. LNCS, vol. 7225, pp. 237–252. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-32211-2_16Tikovsky, J.R.: Concolic testing of functional logic programs. In: Seipel, D., Hanus, M., Abreu, S. (eds.) WFLP/WLP/INAP -2017. LNCS (LNAI), vol. 10997, pp. 169–186. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00801-7_11Vidal, G.: Concolic execution and test case generation in prolog. In: Proietti, M., Seki, H. (eds.) LOPSTR 2014. LNCS, vol. 8981, pp. 167–181. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-17822-6_10Wielemaker, J., Schrijvers, T., Triska, M., Lager, T.: SWI-prolog. TPLP 12(1–2), 67–96 (2012). https://doi.org/10.1017/S147106841100049

Selective Unification in (Constraint) Logic Programming

[EN] Concolic testing is a well-known validation technique for imperative and object oriented programs. In a previous paper, we have introduced an adaptation of this technique to logic programming. At the heart of our framework lies a specific procedure that we call "selective unification". It is used to generate appropriate run-time goals by considering all possible ways an atom can unify with the heads of some program clauses. In this paper, we show that the existing algorithm for selective unification is not complete in the presence of non-linear atoms. We then prove soundness and completeness for a restricted version of the problem where some atoms are required to be linear. We also consider concolic testing in the context of constraint logic programming and extend the notion of selective unification accordingly.This work has been partially supported by the EU (FEDER) and the Spanish Ministerio de Ciencia, Innovacion y Universidades/AEI under grant TIN2016-76843-C4-1-R and by the Generalitat Valenciana under grant Prometeo/2019/098 (DeepTrust).Mesnard, F.; Payet, E.; Vidal, G. (2020). Selective Unification in (Constraint) Logic Programming. Fundamenta Informaticae. 177(3-4):359-383. https://doi.org/10.3233/FI-2020-1993S3593831773-

Formal Methods for Constraint-Based Testing and Reversible Debugging in Erlang

Tesis por compendio[ES] Erlang es un lenguaje de programación funcional con concurrencia mediante paso de mensajes basado en el modelo de actores. Éstas y otras características lo hacen especialmente adecuado para aplicaciones distribuidas en tiempo real acrítico. En los últimos años, la popularidad de Erlang ha aumentado debido a la demanda de servicios concurrentes. No obstante, desarrollar sistemas Erlang libres de errores es un reto considerable. A pesar de que Erlang evita muchos problemas por diseño (por ejemplo, puntos muertos), algunos otros problemas pueden aparecer. En este contexto, las técnicas de testing y depuración basadas en métodos formales pueden ser útiles para detectar, localizar y arreglar errores de programación en Erlang. En esta tesis proponemos varios métodos para testing y depuración en Erlang. En particular, estos métodos están basados en modelos semánticos para concolic testing, pruebas basadas en propiedades, depuración reversible con consistencia causal y repetición reversible con consistencia causal de programas Erlang. Además, probamos formalmente las principales propiedades de nuestras propuestas y diseñamos herramientas de código abierto que implementan estos métodos.[CA] Erlang és un llenguatge de programació funcional amb concurrència mitjançant pas de missatges basat en el model d'actors. Estes i altres característiques el fan especialment adequat per a aplicacions distribuïdes en temps real acrític. En els últims anys, la popularitat d'Erlang ha augmentat degut a la demanda de servicis concurrents. No obstant, desenvolupar sistemes Erlang lliures d'errors és un repte considerable. Encara que Erlang evita molts problemes per disseny (per exemple, punts morts), alguns altres problemes poden aparéixer. En este context, les tècniques de testing y depuració basades en mètodes formals poden ser útils per a detectar, localitzar y arreglar errors de programació en Erlang. En esta tesis proposem diversos mètodes per a testing i depuració en Erlang. En particular, estos mètodes estan basats en models semàntics per a concolic testing, testing basat en propietats, depuració reversible amb consistència causal i repetició reversible amb consistència causal de programes Erlang. A més, provem formalment les principals propietats de les nostres propostes i dissenyem ferramentes de codi obert que implementen estos mètodes.[EN] Erlang is a message-passing concurrent, functional programming language based on the actor model. These and other features make it especially appropriate for distributed, soft real-time applications. In the recent years, Erlang's popularity has increased due to the demand for concurrent services. However, developing error-free systems in Erlang is quite a challenge. Although Erlang avoids many problems by design (e.g., deadlocks), some other problems may appear. Here, testing and debugging techniques based on formal methods may be helpful to detect, locate and fix programming errors in Erlang. In this thesis we propose several methods for testing and debugging in Erlang. In particular, these methods are based on semantics models for concolic testing, property-based testing, causal-consistent reversible debugging and causal-consistent replay debugging of Erlang programs. We formally prove the main properties of our proposals and design open-source tools that implement these methods.Palacios Corella, A. (2020). Formal Methods for Constraint-Based Testing and Reversible Debugging in Erlang [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/139076TESISCompendi