Programming Not Only by Example

In recent years, there has been tremendous progress in automated synthesis techniques that are able to automatically generate code based on some intent expressed by the programmer. A major challenge for the adoption of synthesis remains in having the programmer communicate their intent. When the expressed intent is coarse-grained (for example, restriction on the expected type of an expression), the synthesizer often produces a long list of results for the programmer to choose from, shifting the heavy-lifting to the user. An alternative approach, successfully used in end-user synthesis is programming by example (PBE), where the user leverages examples to interactively and iteratively refine the intent. However, using only examples is not expressive enough for programmers, who can observe the generated program and refine the intent by directly relating to parts of the generated program. We present a novel approach to interacting with a synthesizer using a granular interaction model. Our approach employs a rich interaction model where (i) the synthesizer decorates a candidate program with debug information that assists in understanding the program and identifying good or bad parts, and (ii) the user is allowed to provide feedback not only on the expected output of a program, but also on the underlying program itself. That is, when the user identifies a program as (partially) correct or incorrect, they can also explicitly indicate the good or bad parts, to allow the synthesizer to accept or discard parts of the program instead of discarding the program as a whole. We show the value of our approach in a controlled user study. Our study shows that participants have strong preference to using granular feedback instead of examples, and are able to provide granular feedback much faster

Characterizing lab instructors' self-reported learning goals to inform development of an experimental modeling skills assessment

The ability to develop, use, and refine models of experimental systems is a nationally recognized learning outcome for undergraduate physics lab courses. However, no assessments of students' model-based reasoning exist for upper-division labs. This study is the first step toward development of modeling assessments for optics and electronics labs. In order to identify test objectives that are likely relevant across many institutional contexts, we interviewed 35 lab instructors about the ways they incorporate modeling in their course learning goals and activities. The study design was informed by the Modeling Framework for Experimental Physics. This framework conceptualizes modeling as consisting of multiple subtasks: making measurements, constructing system models, comparing data to predictions, proposing causes for discrepancies, and enacting revisions to models or apparatus. We found that each modeling subtask was identified by multiple instructors as an important learning outcome for their course. Based on these results, we argue that test objectives should include probing students' competence with most modeling subtasks, and test items should be designed to elicit students' justifications for choosing particular modeling pathways. In addition to discussing these and other implications for assessment, we also identify future areas of research related to the role of modeling in optics and electronics labs.Comment: 24 pages, 2 figures, 5 tables; submitted to Phys. Rev. PE

Scoping a vision for formative e-assessment: a project report for JISC

Assessment is an integral part of teaching and learning. If the relationship between teaching and learning were causal, i. e. if students always mastered the intended learning outcomes of a particular sequence of instruction, assessment would be superfluous. Experience and research suggest this is not the case: what is learnt can often be quite different from what is taught. Formative assessment is motivated by a concern with the elicitation of relevant information about student understanding and / or achievement, its interpretation and an exploration of how it can lead to actions that result in better learning. In the context of a policy drive towards technology-enhanced approaches to teaching and learning, the question of the role of digital technologies is key and it is the latter on which this project particularly focuses. The project and its deliverables have been informed by recent and relevant literature, in particular recent work by Black andIn this work, they put forward a framework which suggests that assessment for learning their term for formative assessment can be conceptualised as consisting of a number of aspects and five keystrategies. The key aspects revolve around the where the learner is going, where the learner is right now and how she can get there and examines the role played by the teacher, peers and the learner. Language: English Keywords: assessments, case studies, design patterns, e-assessmen

Sketched Answer Set Programming

Answer Set Programming (ASP) is a powerful modeling formalism for combinatorial problems. However, writing ASP models is not trivial. We propose a novel method, called Sketched Answer Set Programming (SkASP), aiming at supporting the user in resolving this issue. The user writes an ASP program while marking uncertain parts open with question marks. In addition, the user provides a number of positive and negative examples of the desired program behaviour. The sketched model is rewritten into another ASP program, which is solved by traditional methods. As a result, the user obtains a functional and reusable ASP program modelling her problem. We evaluate our approach on 21 well known puzzles and combinatorial problems inspired by Karp's 21 NP-complete problems and demonstrate a use-case for a database application based on ASP.Comment: 15 pages, 11 figures; to appear in ICTAI 201

Defining Youth Outcomes for STEM Learning in Afterschool

This report presents the results of a study to obtain consensus from afterschool experts on appropriate and feasible youth outcomes for STEM learning in afterschool. It presents a compelling set of developmental outcomes, indicators of progress toward these outcomes, and types of evidence that could be collected to demonstrate the impact of STEM programming in afterschool