1 research outputs found
Describing Console I/O Behavior for Testing Student Submissions in Haskell
We present a small, formal language for specifying the behavior of simple
console I/O programs. The design is driven by the concrete application case of
testing interactive Haskell programs written by students. Specifications are
structurally similar to lexical analysis regular expressions, but are augmented
with features like global variables that track state and history of program
runs, enabling expression of an interesting range of dynamic behavior. We give
a semantics for our specification language based on acceptance of execution
traces. From this semantics we derive a definition of the set of all traces
valid for a given specification. Sampling that set enables us to mechanically
check program behavior against specifications in a probabilistic fashion.
Beyond testing, other possible uses of the specification language in an
education context include related activities like providing more helpful
feedback, generating sample solutions, and even generating random exercise
tasks.Comment: In Proceedings TFPIE 2019 and 2020, arXiv:2008.0892