Supporting Unit Test Generation via Automated Isolation

“Este trabajo es una compilación de crónicas, resultado del programa Talleres de Crónica Memorias del Agua, auspiciado por el Banco de la República bajo la coordinación académica de la Pontificia Universidad Javeriana, que pretende contar una historia inédita del país inspirada por el agua. Las setenta y cuatro crónicas seleccionadas para este libro permitieron concluir que, en el nuevo milenio, en varias capitales y cabeceras municipales del país, hay miles de personas que libran una batalla diaria por acceder a este recurso básico. Estos talleres propiciaron un estimulante diálogo entre culturas regionales, generaciones, oficios, saberes, disciplinas y miradas al agua en medio de la diversidad territorial. La mayoría de los participantes jamás había escrito una crónica, y los que tenían experiencia con la escritura no habían experimentado con ese género. Otros comprendieron la desaprovechada cercanía entre la historia y el periodismo, el periodismo y la literatura. En los talleres aprendieron a encontrar su propia voz y a recoger las voces de otros para lograr esa polifonía que pide el género”. Descripción tomada de la sección de novedades de publicaciones de la Pontificia Universidad Javeriana. http://www.javeriana.edu.co/editorial/libros/pais-gota-agua- Memorias. - Lugares. - Personajes

UNIT-LEVEL ISOLATION AND TESTING OF BUGGY CODE

In real-world software development, maintenance plays a major role and developers spend 50-80% of their time in maintenance-related activities. During software maintenance, a significant amount of effort is spent on ending and fixing bugs. In some cases, the fix does not completely eliminate the buggy behavior; though it addresses the reported problem, it fails to account for conditions that could lead to similar failures. There could be many possible reasons: the conditions may have been overlooked or difficult to reproduce, e.g., when the components that invoke the code or the underlying components it interacts with can not put it in a state where latent errors appear. We posit that such latent errors can be discovered sooner if the buggy section can be tested more thoroughly in a separate environment, a strategy that is loosely analogous to the medical procedure of performing a biopsy where tissue is removed, examined and subjected to a battery of tests to determine the presence of a disease. In this thesis, we propose a process in which the buggy code is extracted and isolated in a test framework. Test drivers and stubs are added to exercise the code and observe its interactions with its dependencies. We lay the groundwork for the creation of an automated tool for isolating code by studying its feasibility and investigating existing testing technologies that can facilitate the creation of such drivers and stubs. We investigate mocking frameworks, symbolic execution and model checking tools and test their capabilities by examining real bugs from the Apache Tomcat project. We demonstrate the merits of performing unit-level symbolic execution and model checking to discover runtime exceptions and logical errors. The process is shown to have high coverage and able to uncover latent errors due to insufficient fixes

Mimicking Production Behavior with Generated Mocks

Mocking in the context of automated software tests allows testing program units in isolation. Designing realistic interactions between a unit and its environment, and understanding the expected impact of these interactions on the behavior of the unit, are two key challenges that software testers face when developing tests with mocks. In this paper, we propose to monitor an application in production to generate tests that mimic realistic execution scenarios through mocks. Our approach operates in three phases. First, we instrument a set of target methods for which we want to generate tests, as well as the methods that they invoke, which we refer to mockable method calls. Second, in production, we collect data about the context in which target methods are invoked, as well as the parameters and the returned value for each mockable method call. Third, offline, we analyze the production data to generate test cases with realistic inputs and mock interactions. The approach is automated and implemented in an open-source tool called RICK. We evaluate our approach with three real-world, open-source Java applications. RICK monitors the invocation of 128 methods in production across the three applications and captures their behavior. Next, RICK analyzes the production observations in order to generate test cases that include rich initial states and test inputs, mocks and stubs that recreate actual interactions between the method and its environment, as well as mock-based oracles. All the test cases are executable, and 52.4% of them successfully mimic the complete execution context of the target methods observed in production. We interview 5 developers from the industry who confirm the relevance of using production observations to design mocks and stubs

A Unit Test Approach for Database Schema Evolution

Context: The constant changes in today’s business requirements demand continuous database revisions. Hence, database structures, not unlike software applications, deteriorate during their lifespan and thus require refactoring in order to achieve a longer life span. Although unit tests support changes to application programs and refactoring, there is currently a lack of testing strategies for database schema evolution. Objective: This work examines the challenges for database schema evolution and explores the possibility of using various testing strategies to assist with schema evolution. Specifically, the work proposes a novel unit test approach for the application code that accesses databases with the objective of proactively evaluating the code against the altered database. Method: The approach was validated through the implementation of a testing framework in conjunction with a sample application and a relatively simple database schema. Although the database schema in this study was simple, it was nevertheless able to demonstrate the advantages of the proposed approach. Results: After changes in the database schema, the proposed approach found all SELECT statements as well as the majority of other statements requiring modifications in the application code. Due to its efficiency with SELECT statements, the proposed approach is expected to be more successful with database warehouse applications where SELECT statements are dominant. Conclusion: The unit test approach that accesses databases has proven to be successful in evaluating the application code against the evolved database. In particular, the approach is simple and straightforward to implement, which makes it easily adoptable in practice

Estimating Photometric Redshifts Using Support Vector Machines

We present a new approach to obtaining photometric redshifts using a kernel learning technique called Support Vector Machines (SVMs). Unlike traditional spectral energy distribution fitting, this technique requires a large and representative training set. When one is available, however, it is likely to produce results that are comparable to the best obtained using template fitting and artificial neural networks. Additional photometric parameters such as morphology, size and surface brightness can be easily incorporated. The technique is demonstrated using samples of galaxies from the Sloan Digital Sky Survey Data Release 2 and the hybrid galaxy formation code GalICS. The RMS error in redshift estimation is $<0.03$ for both samples. The strengths and limitations of the technique are assessed.Comment: 10 pages, 3 figures, to appear in the PASP, minor typos fixed to make consistent with published versio

Java Enterprise Edition Support in Search-Based JUnit Test Generation.

Many different techniques and tools for automated unit test generation target the Java programming languages due to its popularity. However, a lot of Java’s popularity is due to its usage to develop enterprise applications with frameworks such as Java Enterprise Edition (JEE) or Spring. These frameworks pose challenges to the automatic generation of JUnit tests. In particular, code units (“beans”) are handled by external web containers (e.g., WildFly and GlassFish). Without considering how web containers initialize these beans, automatically generated unit tests would not represent valid scenarios and would be of little use. For example, common issues of bean initialization are dependency injection, database connection, and JNDI bean lookup. In this paper, we extend the EvoSuite search-based JUnit test generation tool to provide initial support for JEE applications. Experiments on 247 classes (the JBoss EAP tutorial examples) reveal an increase in code coverage, and demonstrate that our techniques prevent the generation of useless tests (e.g., tests where dependencies are not injected)

Did You Remember to Test Your Tokens?

Authentication is a critical security feature for confirming the identity of a system's users, typically implemented with help from frameworks like Spring Security. It is a complex feature which should be robustly tested at all stages of development. Unit testing is an effective technique for fine-grained verification of feature behaviors that is not widely-used to test authentication. Part of the problem is that resources to help developers unit test security features are limited. Most security testing guides recommend test cases in a "black box" or penetration testing perspective. These resources are not easily applicable to developers writing new unit tests, or who want a security-focused perspective on coverage. In this paper, we address these issues by applying a grounded theory-based approach to identify common (unit) test cases for token authentication through analysis of 481 JUnit tests exercising Spring Security-based authentication implementations from 53 open source Java projects. The outcome of this study is a developer-friendly unit testing guide organized as a catalog of 53 test cases for token authentication, representing unique combinations of 17 scenarios, 40 conditions, and 30 expected outcomes learned from the data set in our analysis. We supplement the test guide with common test smells to avoid. To verify the accuracy and usefulness of our testing guide, we sought feedback from selected developers, some of whom authored unit tests in our dataset.Comment: In 17th International Conference on Mining Software Repositories (MSR) 2020, Technical Track, Virtual. 11 page

Tools and Language Elements for Testing, Encapsulation and Controlling Abstraction in Large Scale C++ Projects

A disszertáció új kutatási eredményeket mutat be három alapvető szoftver fejlesztési területen: tesztelés, egységbezárás és absztrakció. Az első három tézis az ún. nem-tolakodó teszteléssel foglalkozik, amely egy olyan tesztelési technika amely során nem szükséges semmilyen strukturális módosítást végrehajtanunk a termék forráskódján. Megvitatjuk a már létező nem-tolakodó tesztelési módszereket és felsoroljuk ezek előnyeit és hátrányait. Bevezetünk egy új, nem-tolakodó tesztelési módszert amely függvény hívás közbeavatkozáson alapszik és számos egyértelmű előnnyel rendelkezik a korábbi megoldásokhoz képest. Ezzel az új technikával képesek vagyunk függvényeket teszt dublőrökkel helyettesíteni még akkor is ha azok inline függvények. Továbbá bemutatunk két új kísérleti eljárást amelyek lehetővé teszik, hogy akár típusokat is helyettesítsünk teszt dublőrökkel: az egyik metódus szintaxis fa transzformációkon alapszik, a másik pedig fordítási idejű reflectionön. Demonstráljuk, hogy gyakran előfordul, hogy szükséges privát tagokhoz hozzáférni a nem-tolakodó tesztek esetében. Bemutatunk két új módszert a privát tagok eléréséhez (és ily módon támogatjuk a nem-tolakodó és fehér doboz tesztek létrehozását): egy program könyvtárat amely explicit sablon példányosításon alapszik, illetve az osztályon kívüli barát (friend) nyelvi elemet. Az egységbezárással kapcsolatosan szemléltetjük, hogy bizonyos nyelvi konstrukciók minta C++ barát (friend) túlzottan erős hozzáférést nyújthat egy osztály belső elemeihez. Ez a túlzott hozzáférés hibák forrása lehet az adott szoftverben. Javaslatot teszünk egy új nyelvi elem létrehozására amely lehetővé teszi, hogy megszorítsuk ezt a hozzáférést csupán néhány jól specifikált taghoz, ily módon erősítendő az egységbezárást és adatrejtést. Az egységbezárás mellett az absztrakció a másik alapvető szereplő ha nagy méretű szoftverek fejlesztéséről van szó. Különösen,ha többszálú programokról beszélünk. Bemutatunk egy új magas szintű C++ absztrakciót mely a read-copy-update konkurrens programozási mintán alapszik és elfogadható teljesítményt nyújt amellett, hogy kellően generikus és biztonságos használni. Az itt bemutatott új módszerek mindegyikéhez tartozik prototípus implementáció (ez alól kivételt képez a reflection alapú nem-tolakodó tesztelés ötlete)