Feedback Generation for Performance Problems in Introductory Programming Assignments

Providing feedback on programming assignments manually is a tedious, error prone, and time-consuming task. In this paper, we motivate and address the problem of generating feedback on performance aspects in introductory programming assignments. We studied a large number of functionally correct student solutions to introductory programming assignments and observed: (1) There are different algorithmic strategies, with varying levels of efficiency, for solving a given problem. These different strategies merit different feedback. (2) The same algorithmic strategy can be implemented in countless different ways, which are not relevant for reporting feedback on the student program. We propose a light-weight programming language extension that allows a teacher to define an algorithmic strategy by specifying certain key values that should occur during the execution of an implementation. We describe a dynamic analysis based approach to test whether a student's program matches a teacher's specification. Our experimental results illustrate the effectiveness of both our specification language and our dynamic analysis. On one of our benchmarks consisting of 2316 functionally correct implementations to 3 programming problems, we identified 16 strategies that we were able to describe using our specification language (in 95 minutes after inspecting 66, i.e., around 3%, implementations). Our dynamic analysis correctly matched each implementation with its corresponding specification, thereby automatically producing the intended feedback.Comment: Tech report/extended version of FSE 2014 pape

On the Concept of Variable Roles and its Use in Software Analysis

Human written source code in imperative programming languages exhibits typical patterns for variable use such as flags, loop iterators, counters, indices, bitvectors etc. Although it is widely understood by practitioners that these variable roles are important for automated software analysis tools, they are not systematically studied by the formal methods community, and not well documented in the research literature. In this paper, we study the notion of variable roles on the example of basic types (int, float, char) in C. We propose a classification of the variables in a program by variable roles, and demonstrate that classical data flow analysis lends itself naturally both as a specification formalism and an analysis paradigm for this classification problem. We demonstrate the practical applicability of our method by predicting membership of source files to the different categories of the software verification competition SVCOMP 2013

Monadic second order finite satisfiability and unbounded tree-width

The finite satisfiability problem of monadic second order logic is decidable only on classes of structures of bounded tree-width by the classic result of Seese (1991). We prove the following problem is decidable: Input: (i) A monadic second order logic sentence $\alpha$, and (ii) a sentence $\beta$ in the two-variable fragment of first order logic extended with counting quantifiers. The vocabularies of $\alpha$ and $\beta$ may intersect. Output: Is there a finite structure which satisfies $\alpha\land\beta$ such that the restriction of the structure to the vocabulary of $\alpha$ has bounded tree-width? (The tree-width of the desired structure is not bounded.) As a consequence, we prove the decidability of the satisfiability problem by a finite structure of bounded tree-width of a logic extending monadic second order logic with linear cardinality constraints of the form $|X_{1}|+\cdots+|X_{r}|<|Y_{1}|+\cdots+|Y_{s}|$, where the $X_{i}$ and $Y_{j}$ are monadic second order variables. We prove the decidability of a similar extension of WS1S

A Simple and Scalable Static Analysis for Bound Analysis and Amortized Complexity Analysis

We present the first scalable bound analysis that achieves amortized complexity analysis. In contrast to earlier work, our bound analysis is not based on general purpose reasoners such as abstract interpreters, software model checkers or computer algebra tools. Rather, we derive bounds directly from abstract program models, which we obtain from programs by comparatively simple invariant generation and symbolic execution techniques. As a result, we obtain an analysis that is more predictable and more scalable than earlier approaches. Our experiments demonstrate that our analysis is fast and at the same time able to compute bounds for challenging loops in a large real-world benchmark. Technically, our approach is based on lossy vector addition systems (VASS). Our bound analysis first computes a lexicographic ranking function that proves the termination of a VASS, and then derives a bound from this ranking function. Our methodology achieves amortized analysis based on a new insight how lexicographic ranking functions can be used for bound analysis

Identifying impediments of succession planning in credit unions

Credit unions, the cooperatives started by the people to serve the people, have experienced tremendous growth, success, and challenge since their inception in the mid-19th century. While the overall number of members and assets are growing, the physical number of credit unions is decreasing due to mergers or insolvency, keeping market share stagnant for the past 20 years. As with all organizations, succession planning is essential to ensure a future. Considering a conceptual foundation including stakeholder theory and succession planning, the purpose of this qualitative study was to better understand how succession planning is utilized by CEOs of credit unions today, what impedes credit unions from succession planning and leadership development, as well as what tools or resources are needed within the industry to either build or enhance the succession planning efforts. Based on in-depth interviews with eight current CEO’s, findings revealed that succession planning is happening more frequently in large credit unions than previous research noted. CEOs are committed to their stakeholders and to the industry and are driving this process in their credit unions, they believe developing leaders is their main role, and they see succession planning as a competitive advantage because of the results it generates. To make the process successful, the CEOs are utilizing consultants, incorporating a variety of activities, focusing on innovation and technology, and challenging the talent management status quo. The CEOs did not believe suggested impediments from previous research were accurate. Rather, they believed that intrinsic factors got in the way including excuses, basic human nature and egos. To truly revolutionize the industry and gain market share, the CEOs shared that many strategies including hiring practices must change. It is recommended that credit union leaders and directors become knowledgeable on succession planning and its benefits, connect strategic planning with talent management, and remove intrinsic obstacles to most effectively give back to their stakeholders. Additional research on smaller credit unions and their succession planning efforts, recruiting and hiring tactics for credit union CEOs, and the prioritization of succession planning, strategic planning, and financial results is needed

On the Expressiveness of a Logic of Separated Relations

We compare the model-theoretic expressiveness of the existential fragment of Separation Logic over unrestricted relational signatures (SLR) -- with only separating conjunction as logical connective and higher-order inductive definitions, traditionally known as the symbolic heap fragment -- with the expressiveness of (Monadic) Second Order Logic ((M)SO). While SLR and MSO are incomparable on structures of unbounded treewidth, it turns out that SLR can be embedded in SO, in general, and that MSO becomes a strict subset of SLR, when the treewidth of the models is bounded by a parameter given as input. We also discuss the problem of defining a fragment of SLR that is equivalent to MSO over models of bounded treewidth. Such a fragment would then become the most general Separation Logic with a decidable entailment problem, a key ingredient of practical verification methods for self-adapting (reconfigurable) component-based and distributed systems