15 research outputs found
User Interaction Models for Disambiguation in Programming by Example
Programming by Examples (PBE) has the potential to revolutionize end-user programming by enabling end users, most of whom are non-programmers, to create small scripts for automating repetitive tasks. However, examples, though often easy to provide, are an ambiguous specification of the user's intent. Because of that, a key impedance in adoption of PBE systems is the lack of user confidence in the correctness of the program that was synthesized by the system. We present two novel user interaction models that communicate actionable information to the user to help resolve ambiguity in the examples. One of these models allows the user to effectively navigate between the huge set of programs that are consistent with the examples provided by the user. The other model uses active learning to ask directed example-based questions to the user on the test input data over which the user intends to run the synthesized program. Our user studies show that each of these models significantly reduces the number of errors in the performed task without any difference in completion time. Moreover, both models are perceived as useful, and the proactive active-learning based model has a slightly higher preference regarding the users' confidence in the result
The Usability of Pragmatic Communication in Regular Expression Synthesis
Programming-by-example (PBE) systems aim to alleviate the burden of
programming. However, user-specified examples are often ambiguous, leaving
multiple programs to satisfy the specification. Consequently, in most prior
work, users have had to provide additional examples, particularly negative
ones, to further constrain the search over compatible programs. Recent work
resolves additional ambiguity by modeling program synthesis tasks as pragmatic
communication, showing promising results on a graphics domain using a
rudimentary user-study. We adapt pragmatic reasoning to a sub-domain of regular
expressions and rigorously study its usability as a means of communication both
with and without the ability to provide negative examples. Our user study
(N=30) demonstrates that, with a pragmatic synthesizer, end-users can more
successfully communicate a target regex using positive examples alone (95%)
compared to using a non-pragmatic synthesizer (51%). Further, users can
communicate more efficiently (57% fewer examples) with a pragmatic synthesizer
compared to a non-pragmatic one
FlashProfile: A Framework for Synthesizing Data Profiles
We address the problem of learning a syntactic profile for a collection of
strings, i.e. a set of regex-like patterns that succinctly describe the
syntactic variations in the strings. Real-world datasets, typically curated
from multiple sources, often contain data in various syntactic formats. Thus,
any data processing task is preceded by the critical step of data format
identification. However, manual inspection of data to identify the different
formats is infeasible in standard big-data scenarios.
Prior techniques are restricted to a small set of pre-defined patterns (e.g.
digits, letters, words, etc.), and provide no control over granularity of
profiles. We define syntactic profiling as a problem of clustering strings
based on syntactic similarity, followed by identifying patterns that succinctly
describe each cluster. We present a technique for synthesizing such profiles
over a given language of patterns, that also allows for interactive refinement
by requesting a desired number of clusters.
Using a state-of-the-art inductive synthesis framework, PROSE, we have
implemented our technique as FlashProfile. Across tasks over large
real datasets, we observe a median profiling time of only s.
Furthermore, we show that access to syntactic profiles may allow for more
accurate synthesis of programs, i.e. using fewer examples, in
programming-by-example (PBE) workflows such as FlashFill.Comment: 28 pages, SPLASH (OOPSLA) 201
Sketch-n-Sketch: Output-Directed Programming for SVG
For creative tasks, programmers face a choice: Use a GUI and sacrifice
flexibility, or write code and sacrifice ergonomics?
To obtain both flexibility and ease of use, a number of systems have explored
a workflow that we call output-directed programming. In this paradigm, direct
manipulation of the program's graphical output corresponds to writing code in a
general-purpose programming language, and edits not possible with the mouse can
still be enacted through ordinary text edits to the program. Such capabilities
provide hope for integrating graphical user interfaces into what are currently
text-centric programming environments.
To further advance this vision, we present a variety of new output-directed
techniques that extend the expressive power of Sketch-n-Sketch, an
output-directed programming system for creating programs that generate vector
graphics. To enable output-directed interaction at more stages of program
construction, we expose intermediate execution products for manipulation and we
present a mechanism for contextual drawing. Looking forward to output-directed
programming beyond vector graphics, we also offer generic refactorings through
the GUI, and our techniques employ a domain-agnostic provenance tracing scheme.
To demonstrate the improved expressiveness, we implement a dozen new
parametric designs in Sketch-n-Sketch without text-based edits. Among these is
the first demonstration of building a recursive function in an output-directed
programming setting.Comment: UIST 2019 Paper + Appendi
CUBES: A Parallel Synthesizer for SQL Using Examples
In recent years, more people have seen their work depend on data manipulation
tasks. However, many of these users do not have the background in programming
required to write complex programs, particularly SQL queries. One way of
helping these users is automatically synthesizing the SQL query given a small
set of examples. Several program synthesizers for SQL have been recently
proposed, but they do not leverage multicore architectures.
This paper proposes CUBES, a parallel program synthesizer for the domain of
SQL queries using input-output examples. Since input-output examples are an
under-specification of the desired SQL query, sometimes, the synthesized query
does not match the user's intent. CUBES incorporates a new disambiguation
procedure based on fuzzing techniques that interacts with the user and
increases the confidence that the returned query matches the user intent.
We perform an extensive evaluation on around 4000 SQL queries from different
domains. Experimental results show that our sequential version can solve more
instances than other state-of-the-art SQL synthesizers. Moreover, the parallel
approach can scale up to 16 processes with super-linear speedups for many hard
instances. Our disambiguation approach is critical to achieving an accuracy of
around 60%, significantly larger than other SQL synthesizers