Teaching Introductory Programming Concepts through a Gesture-Based Interface

Computer programming is an integral part of a technology driven society, so there is a tremendous need to teach programming to a wider audience. One of the challenges in meeting this demand for programmers is that most traditional computer programming classes are targeted to university/college students with strong math backgrounds. To expand the computer programming workforce, we need to encourage a wider range of students to learn about programming. The goal of this research is to design and implement a gesture-driven interface to teach computer programming to young and non-traditional students. We designed our user interface based on the feedback from students attending the College of Engineering summer camps at the University of Arkansas. Our system uses the Microsoft Xbox Kinect to capture the movements of new programmers as they use our system. Our software then tracks and interprets student hand movements in order to recognize specific gestures which correspond to different programming constructs, and uses this information to create and execute programs using the Google Blockly visual programming framework. We focus on various gesture recognition algorithms to interpret user data as specific gestures, including template matching, sector quantization, and supervised machine learning clustering algorithms

A Study of Educational Simulations Part I - Engagement and Learning

Interactive computer simulations with complex representations and sophisticated graphics are a relatively new addition to the classroom, and research in this area is limited. We have conducted over 200 individual student interviews during which the students described what they were thinking as they interacted with simulations. These interviews were conducted as part of the research and design of simulations for the Physics Education Technology (PhET) project. PhET is an ongoing project that has developed over 60 simulations for use in teaching physics, chemistry, and physical science. These interviews are a rich source of information about how students interact with computer simulations and what makes an educationally effective simulation. We have observed that simulations can be highly engaging and educationally effective, but only if the student's interaction with the simulation is directed by the student's own questioning. Here we describe our design process, what features are effective for engaging students in educationally productive interactions and the underlying principles which support our empirically developed guidelines. In a companion paper we describe in detail the design features used to create an intuitive simulation for students to use

The State Of Play: A Notional Machine for Learning Programming

Comprehension of programming and programs is known to be a difficult task for many beginning students, with many computing courses showing significant drop out and failure rates. In this paper, we present a new notional machine de- sign and implementation to help with understanding of pro- gramming and its dynamics for beginning learners. The no- tional machine offers an abstraction of the physical machine designed for comprehension and learning purposes. We in- troduce the notional machine and a graphical notation for its representation. We also present Novis, an implementation of a dynamic real-time visualiser of this notional machine, integrated into BlueJ

Novis: A notional machine implementation for teaching introductory programming

Comprehension of programming and programs is known to be a difficult task for many beginning students, with many computing courses showing significant drop out and failure rates. In this paper, we present a notional machine imple- mentation, Novis, to help with understanding of program- ming and its dynamics for beginning learners. The notional machine offers an abstraction of the physical machine de- signed for comprehension and learning purposes. Novis pro- vides a real-time visualisation of this notional machine, and is integrated into BlueJ

Board games as a teaching tool for technology classes in Compulsory Secondary Education

Aquest treball estudia la tècnica coneguda com game-based learning, és a dir, l’ús dels jocs com a eina didàctica. Primer que res, es fa recerca sobre els treballs ja existents i es veu que, tot i haver-hi articles sobre game-based learning, és difícil trobar-ne de relacionats amb la tecnologia, més enllà d’ensenyar a programar. A continuació, es revisen els continguts curriculars i les competències de secundària i es relacionen amb alguns jocs de taula ja existents, dels quals es detallen breument les regles de joc. Es veu que hi ha continguts curriculars, pels quals es difícil trobar un joc que hi encaixi. A més a més, es desenvolupa la idea d’un nou joc de taula, basat en el ja existent Party & Co., per treballar alguns dels continguts curriculars pels quals no s’ha trobat cap joc existent que s’hi escaigui. Finalment, s’explica una experiència duta a terme durant el període de pràctiques en el centre escolar al curs de 3r d’ESO. Es disposava de tres grups i en tots tres es va seguir la mateixa programació: classe introductòria expositiva, una sessió de muntatge de robots LEGO, 4 sessions de programació i un petit test. En un dels tres grups, però, es va fer una classe prèvia extra on es va jugar a un joc de taula anomenat RoboRally. Els objectius eren dobles: que aprenguessin la importància de l’algorísmica i que s’ho passessin bé. Els resultats mostren que aquest grup va treballar més i millor. En el treball s’analitzen els resultats obtinguts

Spatial Programming for Industrial Robots through Task Demonstration

We present an intuitive system for the programming of industrial robots using markerless gesture recognition and mobile augmented reality in terms of programming by demonstration. The approach covers gesture-based task definition and adaption by human demonstration, as well as task evaluation through augmented reality. A 3D motion tracking system and a handheld device establish the basis for the presented spatial programming system. In this publication, we present a prototype toward the programming of an assembly sequence consisting of several pick-and-place tasks. A scene reconstruction provides pose estimation of known objects with the help of the 2D camera of the handheld. Therefore, the programmer is able to define the program through natural bare-hand manipulation of these objects with the help of direct visual feedback in the augmented reality application. The program can be adapted by gestures and transmitted subsequently to an arbitrary industrial robot controller using a unified interface. Finally, we discuss an application of the presented spatial programming approach toward robot-based welding tasks

Agents for educational games and simulations

This book consists mainly of revised papers that were presented at the Agents for Educational Games and Simulation (AEGS) workshop held on May 2, 2011, as part of the Autonomous Agents and MultiAgent Systems (AAMAS) conference in Taipei, Taiwan. The 12 full papers presented were carefully reviewed and selected from various submissions. The papers are organized topical sections on middleware applications, dialogues and learning, adaption and convergence, and agent applications