32 research outputs found
Tuning Models of Code with Compiler-Generated Reinforcement Learning Feedback
Large Language Models (LLMs) pre-trained on code have recently emerged as the
dominant approach to program synthesis. However, the code that these models
produce can violate basic language-level invariants, leading to lower
performance in downstream tasks. We address this issue through an approach,
called RLCF, that further trains a pre-trained LLM using feedback from a code
compiler. RLCF views the LLM as an RL agent that generates code step by step
and receives: (i) compiler-derived feedback on whether the code it generates
passes a set of correctness checks; and (ii) feedback from a different LLM on
whether the generated code is similar to a set of reference programs in the
training corpus. Together, these feedback mechanisms help the generated code
remain within the target distribution while passing all static correctness
checks. RLCF is model- and language-agnostic. We empirically evaluate it on the
MBJP and MathQA tasks for Java. Our experiments show that RLCF significantly
raises the odds that an LLM-generated program compiles, is executable, and
produces the right output on tests, often allowing LLMs to match the
performance of 2x-8x larger LLMs.Comment: 19 pages, 3 figure
PlinyCompute: A Platform for High-Performance, Distributed, Data-Intensive Tool Development
This paper describes PlinyCompute, a system for development of
high-performance, data-intensive, distributed computing tools and libraries. In
the large, PlinyCompute presents the programmer with a very high-level,
declarative interface, relying on automatic, relational-database style
optimization to figure out how to stage distributed computations. However, in
the small, PlinyCompute presents the capable systems programmer with a
persistent object data model and API (the "PC object model") and associated
memory management system that has been designed from the ground-up for high
performance, distributed, data-intensive computing. This contrasts with most
other Big Data systems, which are constructed on top of the Java Virtual
Machine (JVM), and hence must at least partially cede performance-critical
concerns such as memory management (including layout and de/allocation) and
virtual method/function dispatch to the JVM. This hybrid approach---declarative
in the large, trusting the programmer's ability to utilize PC object model
efficiently in the small---results in a system that is ideal for the
development of reusable, data-intensive tools and libraries. Through extensive
benchmarking, we show that implementing complex objects manipulation and
non-trivial, library-style computations on top of PlinyCompute can result in a
speedup of 2x to more than 50x or more compared to equivalent implementations
on Spark.Comment: 48 pages, including references and Appendi
Auto-Differentiation of Relational Computations for Very Large Scale Machine Learning
The relational data model was designed to facilitate large-scale data
management and analytics. We consider the problem of how to differentiate
computations expressed relationally. We show experimentally that a relational
engine running an auto-differentiated relational algorithm can easily scale to
very large datasets, and is competitive with state-of-the-art, special-purpose
systems for large-scale distributed machine learning.Comment: ICML 202