A Model of Purpose-driven Analogy and Skill Acquisition In Programming

X is a production system model of the acquisition of programming skill. Skilled programming is modelled by the goal-driven application of production rules (productions). Knowledge compilation mechanisms produce new productions that summarize successful problem solving experiences. Analogical problem solving mechanisms use representations of example solutions to overcome problem solving impasses. The interaction of these two mechanisms yields productions that generalize over example and target problem solutions.Simulations of subjects learning to program recursive functions are presented to illustrate the operation of X

Empirical Analyses of Self-Explanation and Transfer in Learning to Program

Building upon recent work on production system models of transfer and analysis-based generalization techniques, we present analyses of three studies of learning to program recursion. In Experiment 1, a production system model was used to identify problem solving that involved previously acquired skills or required novel solutions. A mathematical model based on this analysis accounts for inter-problem transfer. Programming performance was also affected by particular examples presented in instruction. Experiment 2 examined these example effects in finer detail. Using a production system analysis, examples were found to affect the initial error rates, but not the learning rates on cognitive skills. Experiment 3 examined relations between the ways in which people explain examples to themselves and subsequent learning. Results suggest that good learners engage in more metacognition, generate more domain-specific elaborations of examples, make connections between examples and abstract text, and focus on the semantics of programs rather than syntax

Assisting People to Become Independent Learners in the Analysis of Intelligence

Section 1: What Makes Intelligence Analysis Difficult? A Cognitive Task Analysis of Intelligence Analysts by Susan G. Hutchins, Peter L. Pirolli, and Stuart K. Card; Section 2: Evaluation of a Computer Support Tool for Analysis of Competing Hypotheses by Peter Pirolli, Lance Good, Julie Heiser, Jeff Shrager, and Susan Huthins; Section 3: Collaborative Intelligence Analysis with CACHE and its Effects on Information Gathering and Cognitive Bias by Dorrit Billman, Gregorio Convertino, Jeff Shrager, J.P. Massar, Peter PirolliThe purpose of this project was to conduct applied research with exemplary technology to support post-graduate instruction in intelligence analysis. The first phase of research used Cognitive Task Analysis (CTA) to understand the nature of subject matter expertise for this domain, as well as leverage points for technology support. Results from the CTA and advice from intelligence analysis instructors at the Naval Postgraduate School lead us to focus on the development of a collaborative computer tool (CACHE) to support a method called the Analysis of Competing Hypotheses (ACH). We first evaluated a non-collaborative version of an ACH tool in an NPS intelligence classroom setting, followed by an evaluation of the collaborative tool, CACHE at NPS. These evaluations, along with similar studies conducted in coordination with NIST and MITRE, suggested that ACH and CACHE can support intelligence activities and mitigate confirmation bias. However, collaborative analysis has a number of trade-offs: it incurs overhead costs, and can mitigate or exacerbate confirmation bias, depending on the mixture of predisposing biases of collaborators.Office of Naval Researc

Collaborative Explanations and Metacognition : Identifying Successful Learning Activities in the Acquisition of Cognitive Skills

Individual differences in collaborative explanations during learning were analyzed to detennine effects on problem solving. Twenty-five university students with no prior programming experience worked through a sequence of programming lessons. For the Target lesson, subjects studied instructional texts and examples in either mixed performance-level dyads (collaborative dyad group) or individually (individual group) prior to individual programming activities. The collaborative dyad subjects were divided into equal sized groups of high-benefit and low-benefit dyad subjects based on Target lesson programming performance. Betweengroup analyses of the characteristics of the explanations generated by high-benefit and lowbenefit dyad subjects were investigated, including (a) explanation and metacognitive strategies, (b) content of elaborations, and (c) manner of generating elaborations. High-benefit dyad subjects were found to generate both a higher quantity and higher quality of elaborations. These results are compared to findings from prior researc

Effects of Sensemaking Translucence on Distributed Collaborative Analysis

Collaborative sensemaking requires that analysts share their information and insights with each other, but this process of sharing runs the risks of prematurely focusing the investigation on specific suspects. To address this tension, we propose and test an interface for collaborative crime analysis that aims to make analysts more aware of their sensemaking processes. We compare our sensemaking translucence interface to a standard interface without special sensemaking features in a controlled laboratory study. We found that the sensemaking translucence interface significantly improved clue finding and crime solving performance, but that analysts rated the interface lower on subjective measures than the standard interface. We conclude that designing for distributed sensemaking requires balancing task performance vs. user experience and real-time information sharing vs. data accuracy.Comment: ACM SIGCHI CSCW 201

Sense-making strategies in explorative intelligence analysis of network evolutions

Visualising how social networks evolve is important in intelligence analysis in order to detect and monitor issues, such as emerging crime patterns or rapidly growing groups of offenders. It remains an open research question how this type of information should be presented for visual exploration. To get a sense of how users work with different types of visualisations, we evaluate a matrix and a node-link diagram in a controlled thinking aloud study. We describe the sense-making strategies that users adopted during explorative and realistic tasks. Thereby, we focus on the user behaviour in switching between the two visualisations and propose a set of nine strategies. Based on a qualitative and quantitative content analysis we show which visualisation supports which strategy better. We find that the two visualisations clearly support intelligence tasks and that for some tasks the combined use is more advantageous than the use of an individual visualisation