Rose garden promises of intelligent tutoring systems: Blossom or thorn

Intelligent tutoring systems (ITS) have been in existence for over a decade. However, few controlled evaluation studies have been conducted comparing the effectiveness of these systems to more traditional instruction methods. Two main promises of ITSs are examined: (1) Engender more effective and efficient learning in relation to traditional formats; and (2) Reduce the range of learning outcome measures where a majority of individuals are elevated to high performance levels. Bloom (1984) has referred to these as the two sigma problem; to achieve two standard deviation improvements with tutoring over traditional instruction methods. Four ITSs are discussed in relation to the two promises. These tutors have undergone systematic, controlled evaluations: (1) The LISP tutor (Anderson Farrell and Sauers, 1984); (2) Smithtown (Shute and Glaser, in press); (3) Sherlock (Lesgold, Lajoie, Bunzo and Eggan, 1990); and (4) The Pascal ITS (Bonar, Cunningham, Beatty and Well, 1988). Results show that these four tutors do accelerate learning with no degradation in final outcome. Suggestions for improvements to the design and evaluation of ITSs are discussed

Intelligent tutoring systems as tools for investigating individual differences in learning

The ultimate goal of this research is to build an improved model-based selection and classification system for the United States Air Force. Researchers are developing innovative approaches to ability testing. The Learning Abilities Measurement Program (LAMP) examines individual differences in learning abilities, seeking answers to the questions of why some people learn more and better than others and whether there are basic cognitive processes applicable across tasks and domains that are predictive of successful performance (or whether there are more complex problem solving behaviors involved)

Practice Effects and Learning Control on Acquisition, Outcome, and Efficiency

This paper presents results from a study that attempted to replicate unexpected findings from a previous study (Shute & Gawlick, 1995) which investigated the effects of differential practice opportunities on skill acquisition, outcome, efficiency, and retention. These same variables were examined in a new study (N= 380), and the following results were replicated: (1) Learners receiving fewer practice opportimities completed the curriculum significantly faster than the other practice conditions, but at the expense of greater errors; and (2) Despite acquisition differences, all groups performed comparably on the outcome measure. This study also examines the effects of learner control (LC) on these same parameters. We included a condition where students chose their degree of practice, per problem set. Overall, this group completed the curriculum faster, and showed the highest outcome efficiencies, relative to the other conditions. Preliminary results from the retention part of this study (n = 76) continue to show an overall LC advantage, as well as a significant condition x gender interaction. That is, the LC condition is optimal for males, while the extended practice condition is best for females. We discuss the implications of these findings in relation to the design of efficacious instruction

Barriers to the uptake and use of feedback in the context of summative assessment

Despite calls for feedback to be incorporated in all assessments, a dichotomy exists between formative and summative assessments. When feedback is provided in a summative context, it is not always used effectively by learners. In this study we explored the reasons for this. We conducted individual interviews with 17 students who had recently received web based feedback following a summative assessment. Constant comparative analysis was conducted for recurring themes. The summative assessment culture, with a focus on avoiding failure, was a dominant and negative influence on the use of feedback. Strong emotions were prevalent throughout the period of assessment and feedback, which reinforced the focus on the need to pass, rather than excel. These affective factors were heightened by interactions with others. The influence of prior learning experiences affected expectations about achievement and the need to use feedback. The summative assessment and subsequent feedback appeared disconnected from future clinical workplace learning. Socio-cultural influences and barriers to feedback need to be understood before attempting to provide feedback after all assessments. A move away from the summative assessment culture may be needed in order to maximise the learning potential of assessments

Formative Evaluation of Students at Quest to Learn

Abstract Quest to Learn (Q2L) is a public school that opened in 2009 to respond to the needs of kids who are growing up in a highly networked, globally competitive and complex world. The school was designed to include a game-based pedagogy that provides a dynamic, interdisciplinary curriculum, as well as immersive, challenge -based learning experiences. The present article reports the findings from a 20-month evaluation of Q2L. This research was launched when the school opened, with the main goal to identify and assess a set of key student competencies that are relevant and important to Q2L. This student-level evaluation focused on students' development of three core skills (i.e., systems thinking, time management, and teamwork) over the 20-month time period. We found that students significantly improved their systems thinking skills over the duration of the study, and improved (albeit, not significantly) on their time management an d teamwork skills as well