Interactive tool for iterative test suite construction

We can only test software effectively if we understand how it is intended to behave. For some categories of programs, such as scientific models, it is not obvious what the output of the software should be. New techniques are needed to help domain experts, such as scientists, gather the knowledge they need to construct suitable tests and oracles. This paper introduces a new interactive tool for iterative test suite construction that is based upon the scientific method paradigm that scientists are familiar with. We apply our technique to a deterministic mathematical model, used to predict the spread of disease, and show how it helps scientists uncover situations they had not yet considered. Of the 15 hypotheses originally created by modellers, our technique found discrepancies in all but one, allowing us to refine them into a more rigorous test suite.University of Cambridge/Wellcome Trust Junior Interdisciplinary FellowshipThis is the author accepted manuscript. It is currently under an indefinite embargo pending publication by KSI Research Inc

Finding The Lazy Programmer's Bugs

Traditionally developers and testers created huge numbers of explicit tests, enumerating interesting cases, perhaps biased by what they believe to be the current boundary conditions of the function being tested. Or at least, they were supposed to. A major step forward was the development of property testing. Property testing requires the user to write a few functional properties that are used to generate tests, and requires an external library or tool to create test data for the tests. As such many thousands of tests can be created for a single property. For the purely functional programming language Haskell there are several such libraries; for example QuickCheck [CH00], SmallCheck and Lazy SmallCheck [RNL08]. Unfortunately, property testing still requires the user to write explicit tests. Fortunately, we note there are already many implicit tests present in programs. Developers may throw assertion errors, or the compiler may silently insert runtime exceptions for incomplete pattern matches. We attempt to automate the testing process using these implicit tests. Our contributions are in four main areas: (1) We have developed algorithms to automatically infer appropriate constructors and functions needed to generate test data without requiring additional programmer work or annotations. (2) To combine the constructors and functions into test expressions we take advantage of Haskell's lazy evaluation semantics by applying the techniques of needed narrowing and lazy instantiation to guide generation. (3) We keep the type of test data at its most general, in order to prevent committing too early to monomorphic types that cause needless wasted tests. (4) We have developed novel ways of creating Haskell case expressions to inspect elements inside returned data structures, in order to discover exceptions that may be hidden by laziness, and to make our test data generation algorithm more expressive. In order to validate our claims, we have implemented these techniques in Irulan, a fully automatic tool for generating systematic black-box unit tests for Haskell library code. We have designed Irulan to generate high coverage test suites and detect common programming errors in the process

High-Tech Tools for Teaching Physics: the Physics Education Technology Project

This article appeared in the Journal of Online Teaching and Learning September 15, 2006.This paper introduces a new suite of computer simulations from the Physics Education Technology (PhET) project, identifies features of these educational tools, and demonstrates their utility. We compare the use of PhET simulations to the use of more traditional educational resources in lecture, laboratory, recitation and informal settings of introductory college physics. In each case we demonstrate that simulations are as productive, or more productive, for developing student conceptual understanding as real equipment, reading resources, or chalk-talk lectures. We further identify six key characteristic features of these simulations that begin to delineate why these are productive tools. The simulations: support an interactive approach, employ dynamic feedback, follow a constructivist approach, provide a creative workplace, make explicit otherwise inaccessible models or phenomena, and constrain students productively

IDR : a participatory methodology for interdisciplinary design in technology enhanced learning

One of the important themes that emerged from the CAL’07 conference was the failure of technology to bring about the expected disruptive effect to learning and teaching. We identify one of the causes as an inherent weakness in prevalent development methodologies. While the problem of designing technology for learning is irreducibly multi-dimensional, design processes often lack true interdisciplinarity. To address this problem we present IDR, a participatory methodology for interdisciplinary techno-pedagogical design, drawing on the design patterns tradition (Alexander, Silverstein & Ishikawa, 1977) and the design research paradigm (DiSessa & Cobb, 2004). We discuss the iterative development and use of our methodology by a pan-European project team of educational researchers, software developers and teachers. We reflect on our experiences of the participatory nature of pattern design and discuss how, as a distributed team, we developed a set of over 120 design patterns, created using our freely available open source web toolkit. Furthermore, we detail how our methodology is applicable to the wider community through a workshop model, which has been run and iteratively refined at five major international conferences, involving over 200 participants

Rating scale development: a multistage exploratory sequential design

The project chosen to showcase the application of the exploratory sequential design in second/ foreign (L2) language assessment comes from the context of rating scale development and focuses on the development of a set of scales for a suite of high-stakes L2 speaking tests. The assessment of speaking requires assigning scores to a speech sample in a systematic fashion by focusing on explicitly defined criteria which describe different levels of performance (Ginther 2013). Rating scales are the instruments used in this evaluation process, and they can be either holistic (i.e. providing a global overall assessment) or analytic (i.e. providing an independent evaluations for a number of assessment criteria, e.g. Grammar, Vocabulary, Organisation, etc.). The discussion in this chapter is framed within the context of rating scales in speaking assessment. However, it is worth noting that the principles espoused, stages employed and decisions taken during the development process have wider applicability to performance assessment in general

Problems and Promises of Using LMS Learner Analytics for Assessment: Case Study of a First-Year English Program

Learning management systems (LMS) are widely used in education. They offer the potential for assessing student learning, but the reality of using them for this is problematic. This case study chronicles efforts by librarians at Marquette University to use LMS data to assess students’ information literacy knowledge in Marquette’s first-year English program

Building the Scientific Modeling Assistant: An interactive environment for specialized software design

The construction of scientific software models is an integral part of doing science, both within NASA and within the scientific community at large. Typically, model-building is a time-intensive and painstaking process, involving the design of very large, complex computer programs. Despite the considerable expenditure of resources involved, completed scientific models cannot easily be distributed and shared with the larger scientific community due to the low-level, idiosyncratic nature of the implemented code. To address this problem, we have initiated a research project aimed at constructing a software tool called the Scientific Modeling Assistant. This tool provides automated assistance to the scientist in developing, using, and sharing software models. We describe the Scientific Modeling Assistant, and also touch on some human-machine interaction issues relevant to building a successful tool of this type

The 1990 progress report and future plans

This document describes the progress and plans of the Artificial Intelligence Research Branch (RIA) at ARC in 1990. Activities span a range from basic scientific research to engineering development and to fielded NASA applications, particularly those applications that are enabled by basic research carried out at RIA. Work is conducted in-house and through collaborative partners in academia and industry. Our major focus is on a limited number of research themes with a dual commitment to technical excellence and proven applicability to NASA short, medium, and long-term problems. RIA acts as the Agency's lead organization for research aspects of artificial intelligence, working closely with a second research laboratory at JPL and AI applications groups at all NASA centers