A Study of Online Assessment Tools to Practice Programming and Their Effect on Students Grades

“Practice makes perfect” is an old phrase that proves truth in many aspects of the life of a computer engineer. Students in programming courses are reminded constantly by their instructors to practice in order to become better developers. Traditionally, book exercises have been used or assigned to students for practicing programming. However unless these exercises are counted for credit, some students will lack the motivation to do them. On the other hand, assigning too many problems for credit can become a time consuming activity for both students and faculty as well as a grading burden for instructors. It is also known that there are a lot of online resources for practicing programming, but students can get overwhelmed with so many tools. In this work in progress paper we present our preliminary results of how using online assessment tools can help student practice and improve their programming skills. The tools used provide immediate feedback and automatic grading. The hypothesis is that these tools help students to practice more and by giving them immediate feedback and quick grading, they get better at programming and consequently get better test scores. Preliminary data collected shows this to be the case. In this paper we present different scenarios how these tools were used and their effect in the final exam results in different semesters

What Cognitive and Affective States Should Technology Monitor to Support Learning?

This paper discusses self-efficacy, curiosity, and reflectivity as cognitive and affective states that are critical to learning but are overlooked in the context of affect-aware technology for learning. This discussion sits within the opportunities offered by the weDRAW project aiming at an embodied approach to the design of technology to support exploration and learning of mathematical concepts. We first review existing literature to clarify how the three states facilitate learning and how, if not supported, they may instead hinder learning. We then review the literature to understand how bodily expressions communicate these states and how technology could be used to monitor them. We conclude by presenting initial movement cues currently explored in the context of weDRAW

Computing education theories : what are they and how are they used?

In order to mature as a research field, computing education research (CER) seeks to build a better theoretical understanding of how students learn computing concepts and processes. Progress in this area depends on the development of computing-specific theories of learning to complement the general theoretical understanding of learning processes. In this paper we analyze the CER literature in three central publication venues -- ICER, ACM Transactions of Computing Education, and Computer Science Education -- over the period 2005--2015. Our findings identify new theoretical constructs of learning computing that have been published, and the research approaches that have been used in formulating these constructs. We identify 65 novel theoretical constructs in areas such as learning/understanding, learning behaviour/strategies, study choice/orientation, and performance/progression/retention. The most common research methods used to devise new constructs include grounded theory, phenomenography, and various statistical models. We further analyze how a number of these constructs, which arose in computing education, have been used in subsequent research, and present several examples to illustrate how theoretical constructs can guide and enrich further research. We discuss the implications for the whole field

Sensor-Free or Sensor-Full: A Comparison of Data Modalities in Multi-Channel Affect Detection

ABSTRACT Computational models that automatically detect learners' affective states are powerful tools for investigating the interplay of affect and learning. Over the past decade, affect detectors-which recognize learners' affective states at run-time using behavior logs and sensor data-have advanced substantially across a range of K-12 and postsecondary education settings. Machine learningbased affect detectors can be developed to utilize several types of data, including software logs, video/audio recordings, tutorial dialogues, and physical sensors. However, there has been limited research on how different data modalities combine and complement one another, particularly across different contexts, domains, and populations. In this paper, we describe work using the Generalized Intelligent Framework for Tutoring (GIFT) to build multi-channel affect detection models for a serious game on tactical combat casualty care. We compare the creation and predictive performance of models developed for two different data modalities: 1) software logs of learner interactions with the serious game, and 2) posture data from a Microsoft Kinect sensor. We find that interaction-based detectors outperform posture-based detectors for our population, but show high variability in predictive performance across different affect. Notably, our posture-based detectors largely utilize predictor features drawn from the research literature, but do not replicate prior findings that these features lead to accurate detectors of learner affect

Evidencing the value of inquiry based, constructionist learning for student coders

For the last decade, there has been growing interest in the STEAM approach (essentially combining methods and practices in arts, humanities and social sciences into STEM teaching and research) with its potential to deliver better research and education, and to enable us to produce students who can work more effectively in the current and developing marketplace. However, despite this interest, there seems to be little quantitative evidence of the true power of STEAM learning, especially describing how it compares and performs with respect to more established approaches. To address this, we present a comparative, quantitative study of two distinct approaches to teaching programming, one based on STEAM (with an open ended inquirydriven, inductive approach), the other based on a more traditional, non-STEAM approach (where constrained problems are set and solved deductively). Our key results evidence how students exhibit different styles of programming in different types of lessons and, crucially, that students who tend to exhibit more of the style of programming observed in our STEAM lessons also tend to achieve higher grades. We present our claims through a range of visualisations and statistical validations which clearly show the significance of the results, despite the small scale of the study. We believe that this work provides clear evidence for the advantages of STEAM over non-STEAM, and provides a strong theoretical and technological framework for future, larger studies

Biosignals reflect pair-dynamics in collaborative work : EDA and ECG study of pair-programming in a classroom environment

Collaboration is a complex phenomenon, where intersubjective dynamics can greatly affect the productive outcome. Evaluation of collaboration is thus of great interest, and can potentially help achieve better outcomes and performance. However, quantitative measurement of collaboration is difficult, because much of the interaction occurs in the intersubjective space between collaborators. Manual observation and/or self-reports are subjective, laborious, and have a poor temporal resolution. The problem is compounded in natural settings where task-activity and response-compliance cannot be controlled. Physiological signals provide an objective mean to quantify intersubjective rapport (as synchrony), but require novel methods to support broad deployment outside the lab. We studied 28 student dyads during a self-directed classroom pair-programming exercise. Sympathetic and parasympathetic nervous system activation was measured during task performance using electrodermal activity and electrocardiography. Results suggest that (a) we can isolate cognitive processes (mental workload) from confounding environmental effects, and (b) electrodermal signals show role-specific but correlated affective response profiles. We demonstrate the potential for social physiological compliance to quantify pair-work in natural settings, with no experimental manipulation of participants required. Our objective approach has a high temporal resolution, is scalable, non-intrusive, and robust.Peer reviewe

Effects of Error Messages on a Student’s Ability to Understand and Fix Programming Errors

abstract: Assemblers and compilers provide feedback to a programmer in the form of error messages. These error messages become input to the debugging model of the programmer. For the programmer to fix an error, they should first locate the error in the program, understand what is causing that error, and finally resolve that error. Error messages play an important role in all three stages of fixing of errors. This thesis studies the effects of error messages in the context of teaching programming. Given an error message, this work investigates how it effects student’s way of 1) understanding the error, and 2) fixing the error. As part of the study, three error message types were developed – Default, Link and Example, to better understand the effects of error messages. The Default type provides an assembler-centric single line error message, the Link type provides a program-centric detailed error description with a hyperlink for more information, and the Example type provides a program centric detailed error description with a relevant example. All these error message types were developed for assembly language programming. A think aloud programming exercise was conducted as part of the study to capture the student programmer’s knowledge model. Different codes were developed to analyze the data collected as part of think aloud exercise. After transcribing, coding, and analyzing the data, it was found that the Link type of error message helped to fix the error in less time and with fewer steps. Among the three types, the Link type of error message also resulted in a significantly higher ratio of correct to incorrect steps taken by the programmer to fix the error.Dissertation/ThesisMasters Thesis Software Engineering 201

Coarse-grained detection of student frustration in an introductory programming course

We attempt to automatically detect student frustration, at a coarse-grained level, using measures distilled from student behavior within a learning environment for introductory programming. We find that each student\u27s average level of frustration across five lab exercises can be detected based on the number of pairs of consecutive compilations with the same edit location, the number of pairs of consecutive compilations with the same error, the average time between compilations and the total number of errors. Attempts to detect frustration at a finer grain-size, identifying individual students\u27 fluctuations in frustration between labs, were less successful. These results indicate that it is possible to detect frustration at a coarse-grained level, solely from coarse-grained data about students\u27 behavior within a learning environment

Modeling Student Affective State Patterns during Self-Regulated Learning in Physics Playground

This dissertation research focuses on investigating the incidence of student self-regulated learning behavior, and examines patterns in student affective states that accompany such self-regulated behavior. This dissertation leverages prediction models of student affective states in the Physics Playground educational game platform to identify common patterns in student affective states during use of self-regulated learning behavior. In Study 1, prediction models of student affective states are developed in the context of the educational game environment Physics Playground, using affective state observations and computer log data that had already been collected as part of a larger project. The performances of student affective state prediction models generated using a combination of the computer log and observational data are then compared against those of similar prediction models generated using video data collected at the same time. In Study 2, I apply these affective state prediction models to generate predictions of student affective states on a broader set of data collected from students participants playing Physics Playground. In parallel, I define aggregated behavioral features that represent the self-observation and strategic planning components of self-regulated learning. Affective state predictions are then mapped to playground level attempts that contain these self-regulated learning behavioral features, and sequential pattern mining is applied to the affective state predictions to identify the most common patterns in student emotions. Findings from Study 1 demonstrate that both video data and interaction log data can be used to predict student affective states with significant accuracy. Since the video data is a direct measure of student emotions, it shows better performance across most affective states. However, the interaction log data can be collected natively by Physics Playground and is able to be generalized more easily to other learning environments. Findings from Study 2 suggest that self-regulatory behavior is closely associated with sustained periods of engaged concentration and .self-regulated learning behaviors are associated with transitions from negative affective states (confusion, frustration, and boredom) to the positive engaged concentration state. The results of this dissertation project demonstrate the power of measuring student affective states in real time and examining the temporal relationship to self-regulated learning behavior within an unstructured educational game platform. These results thus provide a building block for future research on the real-time assessment of student emotions and its relationship with self-regulated learning behaviors, particularly within online student-centered and self-directed learning contexts