328,087 research outputs found
RLTF: Reinforcement Learning from Unit Test Feedback
The goal of program synthesis, or code generation, is to generate executable
code based on given descriptions. Recently, there has been an increasing number
of studies employing reinforcement learning (RL) to improve the performance of
large language models (LLMs) for code. However, these RL methods have only used
offline frameworks, limiting their exploration of new sample spaces.
Additionally, current approaches that utilize unit test signals are rather
simple, not accounting for specific error locations within the code. To address
these issues, we proposed RLTF, i.e., Reinforcement Learning from Unit Test
Feedback, a novel online RL framework with unit test feedback of
multi-granularity for refining code LLMs. Our approach generates data in
real-time during training and simultaneously utilizes fine-grained feedback
signals to guide the model towards producing higher-quality code. Extensive
experiments show that RLTF achieves state-of-the-art performance on the APPS
and the MBPP benchmarks. Our code can be found at:
https://github.com/Zyq-scut/RLTF
Validity-Guided Synthesis of Reactive Systems from Assume-Guarantee Contracts
Automated synthesis of reactive systems from specifications has been a topic
of research for decades. Recently, a variety of approaches have been proposed
to extend synthesis of reactive systems from proposi- tional specifications
towards specifications over rich theories. We propose a novel, completely
automated approach to program synthesis which reduces the problem to deciding
the validity of a set of forall-exists formulas. In spirit of IC3 / PDR, our
problem space is recursively refined by blocking out regions of unsafe states,
aiming to discover a fixpoint that describes safe reactions. If such a fixpoint
is found, we construct a witness that is directly translated into an
implementation. We implemented the algorithm on top of the JKind model checker,
and exercised it against contracts written using the Lustre specification
language. Experimental results show how the new algorithm outperforms JKinds
already existing synthesis procedure based on k-induction and addresses
soundness issues in the k-inductive approach with respect to unrealizable
results.Comment: 18 pages, 5 figures, 2 table
Parallelizing Deadlock Resolution in Symbolic Synthesis of Distributed Programs
Previous work has shown that there are two major complexity barriers in the
synthesis of fault-tolerant distributed programs: (1) generation of fault-span,
the set of states reachable in the presence of faults, and (2) resolving
deadlock states, from where the program has no outgoing transitions. Of these,
the former closely resembles with model checking and, hence, techniques for
efficient verification are directly applicable to it. Hence, we focus on
expediting the latter with the use of multi-core technology.
We present two approaches for parallelization by considering different design
choices. The first approach is based on the computation of equivalence classes
of program transitions (called group computation) that are needed due to the
issue of distribution (i.e., inability of processes to atomically read and
write all program variables). We show that in most cases the speedup of this
approach is close to the ideal speedup and in some cases it is superlinear. The
second approach uses traditional technique of partitioning deadlock states
among multiple threads. However, our experiments show that the speedup for this
approach is small. Consequently, our analysis demonstrates that a simple
approach of parallelizing the group computation is likely to be the effective
method for using multi-core computing in the context of deadlock resolution
Formal Verification of Security Protocol Implementations: A Survey
Automated formal verification of security protocols has been mostly focused on analyzing high-level abstract models which, however, are significantly different from real protocol implementations written in programming languages. Recently, some researchers have started investigating techniques that bring automated formal proofs closer to real implementations. This paper surveys these attempts, focusing on approaches that target the application code that implements protocol logic, rather than the libraries that implement cryptography. According to these approaches, libraries are assumed to correctly implement some models. The aim is to derive formal proofs that, under this assumption, give assurance about the application code that implements the protocol logic. The two main approaches of model extraction and code generation are presented, along with the main techniques adopted for each approac
- …