A Critical Review of "Automatic Patch Generation Learned from Human-Written Patches": Essay on the Problem Statement and the Evaluation of Automatic Software Repair

At ICSE'2013, there was the first session ever dedicated to automatic program repair. In this session, Kim et al. presented PAR, a novel template-based approach for fixing Java bugs. We strongly disagree with key points of this paper. Our critical review has two goals. First, we aim at explaining why we disagree with Kim and colleagues and why the reasons behind this disagreement are important for research on automatic software repair in general. Second, we aim at contributing to the field with a clarification of the essential ideas behind automatic software repair. In particular we discuss the main evaluation criteria of automatic software repair: understandability, correctness and completeness. We show that depending on how one sets up the repair scenario, the evaluation goals may be contradictory. Eventually, we discuss the nature of fix acceptability and its relation to the notion of software correctness.Comment: ICSE 2014, India (2014

Tortoise: Interactive System Configuration Repair

System configuration languages provide powerful abstractions that simplify managing large-scale, networked systems. Thousands of organizations now use configuration languages, such as Puppet. However, specifications written in configuration languages can have bugs and the shell remains the simplest way to debug a misconfigured system. Unfortunately, it is unsafe to use the shell to fix problems when a system configuration language is in use: a fix applied from the shell may cause the system to drift from the state specified by the configuration language. Thus, despite their advantages, configuration languages force system administrators to give up the simplicity and familiarity of the shell. This paper presents a synthesis-based technique that allows administrators to use configuration languages and the shell in harmony. Administrators can fix errors using the shell and the technique automatically repairs the higher-level specification written in the configuration language. The approach (1) produces repairs that are consistent with the fix made using the shell; (2) produces repairs that are maintainable by minimizing edits made to the original specification; (3) ranks and presents multiple repairs when relevant; and (4) supports all shells the administrator may wish to use. We implement our technique for Puppet, a widely used system configuration language, and evaluate it on a suite of benchmarks under 42 repair scenarios. The top-ranked repair is selected by humans 76% of the time and the human-equivalent repair is ranked 1.31 on average.Comment: Published version in proceedings of IEEE/ACM International Conference on Automated Software Engineering (ASE) 201

Tailored Source Code Transformations to Synthesize Computationally Diverse Program Variants

The predictability of program execution provides attackers a rich source of knowledge who can exploit it to spy or remotely control the program. Moving target defense addresses this issue by constantly switching between many diverse variants of a program, which reduces the certainty that an attacker can have about the program execution. The effectiveness of this approach relies on the availability of a large number of software variants that exhibit different executions. However, current approaches rely on the natural diversity provided by off-the-shelf components, which is very limited. In this paper, we explore the automatic synthesis of large sets of program variants, called sosies. Sosies provide the same expected functionality as the original program, while exhibiting different executions. They are said to be computationally diverse. This work addresses two objectives: comparing different transformations for increasing the likelihood of sosie synthesis (densifying the search space for sosies); demonstrating computation diversity in synthesized sosies. We synthesized 30184 sosies in total, for 9 large, real-world, open source applications. For all these programs we identified one type of program analysis that systematically increases the density of sosies; we measured computation diversity for sosies of 3 programs and found diversity in method calls or data in more than 40% of sosies. This is a step towards controlled massive unpredictability of software

Talos: Neutralizing Vulnerabilities with Security Workarounds for Rapid Response

Considerable delays often exist between the discovery of a vulnerability and the issue of a patch. One way to mitigate this window of vulnerability is to use a configuration workaround, which prevents the vulnerable code from being executed at the cost of some lost functionality -- but only if one is available. Since program configurations are not specifically designed to mitigate software vulnerabilities, we find that they only cover 25.2% of vulnerabilities. To minimize patch delay vulnerabilities and address the limitations of configuration workarounds, we propose Security Workarounds for Rapid Response (SWRRs), which are designed to neutralize security vulnerabilities in a timely, secure, and unobtrusive manner. Similar to configuration workarounds, SWRRs neutralize vulnerabilities by preventing vulnerable code from being executed at the cost of some lost functionality. However, the key difference is that SWRRs use existing error-handling code within programs, which enables them to be mechanically inserted with minimal knowledge of the program and minimal developer effort. This allows SWRRs to achieve high coverage while still being fast and easy to deploy. We have designed and implemented Talos, a system that mechanically instruments SWRRs into a given program, and evaluate it on five popular Linux server programs. We run exploits against 11 real-world software vulnerabilities and show that SWRRs neutralize the vulnerabilities in all cases. Quantitative measurements on 320 SWRRs indicate that SWRRs instrumented by Talos can neutralize 75.1% of all potential vulnerabilities and incur a loss of functionality similar to configuration workarounds in 71.3% of those cases. Our overall conclusion is that automatically generated SWRRs can safely mitigate 2.1x more vulnerabilities, while only incurring a loss of functionality comparable to that of traditional configuration workarounds.Comment: Published in Proceedings of the 37th IEEE Symposium on Security and Privacy (Oakland 2016

Reparación eficiente de estructuras de datos en tiempo de ejecución basada en SAT

Las fallas de un programa pueden producir estados internos inconsistentes, cuya propagación puede provocar la terminación abrupta del programa, la pérdida de datos del usuario, la incapacidad de realizar alguna tarea, etc. La reparación de estructuras de datos consiste en reemplazar estos estados inconsistentes por estructuras generadas a partir de especificaciones formales, utilizando algún procedimiento de decisión, que permitan al programa continuar su ejecución sin mayores problemas. En este trabajo se presenta un enfoque para resolver este problema basado en SAT solving, cuya característica distintiva es el aprovechamiento de dos técnicas del estado del arte para mejorar su eficiencia y escalabilidad: el uso de rotura de simetrías y de cotas ajustadas. La evaluación experimental preliminar realizada muestra que nuestro enfoque es más eficiente y es capaz de lidiar con estructuras de mayor complejidad que técnicas relacionadas.VIII Workshop Innovación en Sistemas de Software (WISS).Red de Universidades con Carreras en Informática (RedUNCI

Reparación eficiente de estructuras de datos en tiempo de ejecución basada en SAT

Las fallas de un programa pueden producir estados internos inconsistentes, cuya propagación puede provocar la terminación abrupta del programa, la pérdida de datos del usuario, la incapacidad de realizar alguna tarea, etc. La reparación de estructuras de datos consiste en reemplazar estos estados inconsistentes por estructuras generadas a partir de especificaciones formales, utilizando algún procedimiento de decisión, que permitan al programa continuar su ejecución sin mayores problemas. En este trabajo se presenta un enfoque para resolver este problema basado en SAT solving, cuya característica distintiva es el aprovechamiento de dos técnicas del estado del arte para mejorar su eficiencia y escalabilidad: el uso de rotura de simetrías y de cotas ajustadas. La evaluación experimental preliminar realizada muestra que nuestro enfoque es más eficiente y es capaz de lidiar con estructuras de mayor complejidad que técnicas relacionadas.VIII Workshop Innovación en Sistemas de Software (WISS).Red de Universidades con Carreras en Informática (RedUNCI

Reparación eficiente de estructuras de datos en tiempo de ejecución basada en SAT

Las fallas de un programa pueden producir estados internos inconsistentes, cuya propagación puede provocar la terminación abrupta del programa, la pérdida de datos del usuario, la incapacidad de realizar alguna tarea, etc. La reparación de estructuras de datos consiste en reemplazar estos estados inconsistentes por estructuras generadas a partir de especificaciones formales, utilizando algún procedimiento de decisión, que permitan al programa continuar su ejecución sin mayores problemas. En este trabajo se presenta un enfoque para resolver este problema basado en SAT solving, cuya característica distintiva es el aprovechamiento de dos técnicas del estado del arte para mejorar su eficiencia y escalabilidad: el uso de rotura de simetrías y de cotas ajustadas. La evaluación experimental preliminar realizada muestra que nuestro enfoque es más eficiente y es capaz de lidiar con estructuras de mayor complejidad que técnicas relacionadas.VIII Workshop Innovación en Sistemas de Software (WISS).Red de Universidades con Carreras en Informática (RedUNCI

FixJS: A Dataset of Bug-fixing JavaScript Commits

The field of Automated Program Repair (APR) has received increasing attention in recent years both from the academic world and from leading IT companies. Its main goal is to repair software bugs automatically, thus reducing the cost of development and maintenance significantly. Recent works use state-of-the-art deep learning models to predict correct patches, for these teaching on a large amount of data is inevitable almost in every scenarios. Despite this, readily accessible data on the field is very scarce. To contribute to related research, we present \emph{FixJS}, a dataset containing bug-fixing information of \textasciitilde 2 million commits. The commits were gathered from GitHub and processed locally to have both the buggy (before bug fixing commit) and fixed (after fix) version of the same program. We focused on JavaScript functions, as it is one of the most popular programming language globally and functions are first class objects there. The data includes more than 300,000 samples of such functions, including commit information, before/after states and 3 source code representations