Investigating the Essential of Meaningful Automated Formative Feedback for Programming Assignments

This study investigated the essential of meaningful automated feedback for programming assignments. Three different types of feedback were tested, including (a) What's wrong - what test cases were testing and which failed, (b) Gap - comparisons between expected and actual outputs, and (c) Hint - hints on how to fix problems if test cases failed. 46 students taking a CS2 participated in this study. They were divided into three groups, and the feedback configurations for each group were different: (1) Group One - What's wrong, (2) Group Two - What's wrong + Gap, (3) Group Three - What's wrong + Gap + Hint. This study found that simply knowing what failed did not help students sufficiently, and might stimulate system gaming behavior. Hints were not found to be impactful on student performance or their usage of automated feedback. Based on the findings, this study provides practical guidance on the design of automated feedback

EqFix: Fixing LaTeX Equation Errors by Examples

LaTeX is a widely-used document preparation system. Its powerful ability in mathematical equation editing is perhaps the main reason for its popularity in academia. Sometimes, however, even an expert user may spend much time on fixing an erroneous equation. In this paper, we present EqFix, a synthesis-based repairing system for LaTeX equations. It employs a set of fixing rules, and can suggest possible repairs for common errors in LaTeX equations. A domain specific language is proposed for formally expressing the fixing rules. The fixing rules can be automatically synthesized from a set of input-output examples. An extension of relaxer is also introduced to enhance the practicality of EqFix. We evaluate EqFix on real-world examples and find that it can synthesize rules with high generalization ability. Compared with a state-of-the-art string transformation synthesizer, EqFix solved 37% more cases and spent only one third of their synthesis time

The Effects of Mixed-Initiative Visualization Systems on Exploratory Data Analysis

The primary purpose of information visualization is to act as a window between a user and the data. Historically, this has been accomplished via a single-agent framework: the only decision-maker in the relationship between visualization system and analyst is the analyst herself. Yet this framework arose not from first principles, but a necessity. Before this decade, computers were limited in their decision-making capabilities, especially in the face of large, complex datasets and visualization systems. This paper aims to present the design and evaluation of a mixed-initiative system that aids the user in handling large, complex datasets and dense visualization systems. We demonstrate this system with a between-groups, two-by-two study measuring the effects of this mixed-initiative system on user interactions and system usability. We find little to no evidence that the adaptive system designed here has a statistically significant impact on user interactions or system usability. We discuss the implications of this lack of evidence and examine how the data suggests a promising avenue for further research

The Effects of Mixed-Initiative Visualization Systems on Exploratory Data Analysis

The main purpose of information visualization is to act as a window between a user and data. Historically, this has been accomplished via a single-agent framework: the only decisionmaker in the relationship between visualization system and analyst is the analyst herself. Yet this framework arose not from first principles, but from necessity: prior to this decade, computers were limited in their decision-making capabilities, especially in the face of large, complex datasets and visualization systems. This thesis aims to present the design and evaluation of a mixed-initiative system that aids the user in handling large, complex datasets and dense visualization systems. We demonstrate this system with a between-groups, two-by-two study measuring the effects of this mixed-initiative system on user interactions and system usability. We find little to no evidence that the adaptive system designed here has a statistically-significant effect on user interactions or system usability. We discuss the implications of this lack of evidence, and examine how the data suggests a promising avenue of further research

Hubble Spacer Telescope

Visualizing a model checker’s run on a model can be useful when trying to gain a deeper understanding of the verification of the particular model. However, it can be difficult to formalize the problem that visualization solves as it varies from person to person. Having a visualized form of a model checker’s run allows a user to pinpoint sections of the run without having to look through the entire log multiple times or having to know what to look for. This thesis presents the Hubble Spacer Telescope (HST), a visualizer for Spacer, an SMT horn clause based solver. HST combines multiple exploration graph views along with customizable lemma transformations. HST offers a variety of ways to transform lemmas so that a user can pick and choose how they want lemmas to be presented. HST’s lemma transformations allow a user to change variable names, rearrange terms in a literal, and rearrange the placement of literals within the lemma through programming by example. HST allows users to not only visually depict a Spacer exploration log but it allows users to transform lemmas produced, in a way that the user hopes, will make understanding a Spacer model checking run, easier. Given a Spacer exploration log, HST creates a raw exploration graph where clicking on a node produces the state of the model as well as the lemmas learned from said state. In addition, there is a second graph view which summarizes the exploration into its proof obligations. HST uses programming by example to simplify lemma transformations so that users only have to modify a few lemmas to transform all lemmas in an exploration log. Users can also choose between multiple transformations to better suit their needs. This thesis presents an evaluation of HST through a case study. The case study is used to demonstrate the extent of the grammar created for lemma transformations. Users have the opportunity to transform disjunctions of literals produced by Spacer into a conditional statement, customized by the contents of the predicate. Since lemma transformations are completely customizable, HST can be viewed as per each individual user’s preferences