Concolic Testing in Logic programming

Software testing is one of the most popular validation techniques in the software industry. Surprisingly, we can only find a few approaches to testing in the context of logic programming. In this paper, we introduce a systematic approach for dynamic testing that combines both concrete and symbolic execution. Our approach is fully automatic and guarantees full path coverage when it terminates. We prove some basic properties of our technique and illustrate its practical usefulness through a prototype implementation.This work has been partially supported by the EU (FEDER) and the Spanish Ministerio de Economia y Competitividad under grant TIN2013-44742-C4-1-R and by the Generalitat Valenciana under grant PROMETEOII/2015/013. Part of this research was done while the third author was visiting the University of Reunion; G. Vidal gratefully acknowledges their hospitality.Mesnard, F.; Payet, E.; Vidal Oriola, GF. (2015). Concolic Testing in Logic programming. Theory and Practice of Logic Programming. 15(4):711-725. https://doi.org/10.1017/S1471068415000332S711725154SCHIMPF, J., & SHEN, K. (2011). ECLiPSe – From LP to CLP. Theory and Practice of Logic Programming, 12(1-2), 127-156. doi:10.1017/s1471068411000469Martelli, A., & Montanari, U. (1982). An Efficient Unification Algorithm. ACM Transactions on Programming Languages and Systems, 4(2), 258-282. doi:10.1145/357162.357169Godefroid, P., Klarlund, N., & Sen, K. (2005). DART. Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation - PLDI ’05. doi:10.1145/1065010.1065036Mera E. , López-García P. , and Hermenegildo M. V. 2009. Integrating software testing and run-time checking in an assertion verification framework. In 25th International Conference on Logic Programming, ICLP 2009, Pasadena. 281–295.Godefroid, P., Levin, M. Y., & Molnar, D. (2012). SAGE. Communications of the ACM, 55(3), 40. doi:10.1145/2093548.2093564WIELEMAKER, J., SCHRIJVERS, T., TRISKA, M., & LAGER, T. (2011). SWI-Prolog. Theory and Practice of Logic Programming, 12(1-2), 67-96. doi:10.1017/s1471068411000494CARLSSON, M., & MILDNER, P. (2011). SICStus Prolog—The first 25 years. Theory and Practice of Logic Programming, 12(1-2), 35-66. doi:10.1017/s1471068411000482Degrave F. , Schrijvers T. , and Vanhoof W. 2008. Automatic generation of test inputs for Mercury. In Logic-Based Program Synthesis and Transformation, 18th International Symposium, LOPSTR 2008. 71–86.Somogyi, Z., Henderson, F., & Conway, T. (1996). The execution algorithm of mercury, an efficient purely declarative logic programming language. The Journal of Logic Programming, 29(1-3), 17-64. doi:10.1016/s0743-1066(96)00068-4Vidal, G. (2015). Concolic Execution and Test Case Generation in Prolog. Lecture Notes in Computer Science, 167-181. doi:10.1007/978-3-319-17822-6_10Belli F. and Jack O. 1993. Implementation-based analysis and testing of Prolog programs. In ISSTA. 70–80.King, J. C. (1976). Symbolic execution and program testing. Communications of the ACM, 19(7), 385-394. doi:10.1145/360248.360252Lloyd, J. W. (1987). Foundations of Logic Programming. doi:10.1007/978-3-642-83189-8Clarke L. 1976. A program testing system. In Proceedings of the 1976 Annual Conference (ACM'76). 488–491

A Delta Debugger for ILP Query Execution

Because query execution is the most crucial part of Inductive Logic Programming (ILP) algorithms, a lot of effort is invested in developing faster execution mechanisms. These execution mechanisms typically have a low-level implementation, making them hard to debug. Moreover, other factors such as the complexity of the problems handled by ILP algorithms and size of the code base of ILP data mining systems make debugging at this level a very difficult job. In this work, we present the trace-based debugging approach currently used in the development of new execution mechanisms in hipP, the engine underlying the ACE Data Mining system. This debugger uses the delta debugging algorithm to automatically reduce the total time needed to expose bugs in ILP execution, thus making manual debugging step much lighter.Comment: Paper presented at the 16th Workshop on Logic-based Methods in Programming Environments (WLPE2006

Checking-in on Network Functions

When programming network functions, changes within a packet tend to have consequences---side effects which must be accounted for by network programmers or administrators via arbitrary logic and an innate understanding of dependencies. Examples of this include updating checksums when a packet's contents has been modified or adjusting a payload length field of a IPv6 header if another header is added or updated within a packet. While static-typing captures interface specifications and how packet contents should behave, it does not enforce precise invariants around runtime dependencies like the examples above. Instead, during the design phase of network functions, programmers should be given an easier way to specify checks up front, all without having to account for and keep track of these consequences at each and every step during the development cycle. In keeping with this view, we present a unique approach for adding and generating both static checks and dynamic contracts for specifying and checking packet processing operations. We develop our technique within an existing framework called NetBricks and demonstrate how our approach simplifies and checks common dependent packet and header processing logic that other systems take for granted, all without adding much overhead during development.Comment: ANRW 2019 ~ https://irtf.org/anrw/2019/program.htm

Maintenance of Automated Test Suites in Industry: An Empirical study on Visual GUI Testing

Context: Verification and validation (V&V) activities make up 20 to 50 percent of the total development costs of a software system in practice. Test automation is proposed to lower these V&V costs but available research only provides limited empirical data from industrial practice about the maintenance costs of automated tests and what factors affect these costs. In particular, these costs and factors are unknown for automated GUI-based testing. Objective: This paper addresses this lack of knowledge through analysis of the costs and factors associated with the maintenance of automated GUI-based tests in industrial practice. Method: An empirical study at two companies, Siemens and Saab, is reported where interviews about, and empirical work with, Visual GUI Testing is performed to acquire data about the technique's maintenance costs and feasibility. Results: 13 factors are observed that affect maintenance, e.g. tester knowledge/experience and test case complexity. Further, statistical analysis shows that developing new test scripts is costlier than maintenance but also that frequent maintenance is less costly than infrequent, big bang maintenance. In addition a cost model, based on previous work, is presented that estimates the time to positive return on investment (ROI) of test automation compared to manual testing. Conclusions: It is concluded that test automation can lower overall software development costs of a project whilst also having positive effects on software quality. However, maintenance costs can still be considerable and the less time a company currently spends on manual testing, the more time is required before positive, economic, ROI is reached after automation

Metamodel Instance Generation: A systematic literature review

Modelling and thus metamodelling have become increasingly important in Software Engineering through the use of Model Driven Engineering. In this paper we present a systematic literature review of instance generation techniques for metamodels, i.e. the process of automatically generating models from a given metamodel. We start by presenting a set of research questions that our review is intended to answer. We then identify the main topics that are related to metamodel instance generation techniques, and use these to initiate our literature search. This search resulted in the identification of 34 key papers in the area, and each of these is reviewed here and discussed in detail. The outcome is that we are able to identify a knowledge gap in this field, and we offer suggestions as to some potential directions for future research.Comment: 25 page