Addressing drop-out and sustained effort issues with large practical groups using an automated delivery and assessment system

The acquisition of programming skills specially in introductory programming courses poses an important challenge for freshmen students of engineering programs. These courses require students to devote a sustained effort during the whole course and a failure to do so may contribute to not passing the course. However, it is difficult for the teaching staff to deploy measures to enforce a pattern of continuous work without significantly increasing the required management tasks. A significant reduction of this workload can be achieved with the automation of time consuming tasks such as the delivery of activities or submission grading. This paper presents a case of study where a technology based orchestration of learning scripts was applied in a large enrollment course to promote student sustained effort through the course and keep the workload on teaching staff within reasonable margins. The orchestration system, based on IMS Learning Design and Generic Service Integration, automatically evaluated the student submissions and gradually unlocked the following activities depending on the received results. Such system was used during a semester supporting 425 students and 8 instructors. The analysis of the case of study followed a mixed method based on qualitative and quantitative data, and revealed a significant reduction of the orchestration workload on the teaching staff, allowing the strategy of continuous work to be applied in a course with high enrollment. Additionally, the application of these techniques show statistically significant differences in the drop out rate with respect to previous editions of the course.Work partially funded by the EEE project, “Plan Nacional de I+D+I TIN2011-28308-C03-01” and the “Emadrid: Investigación y desarrollo de tecnologías para el e-learning en la Comunidad de Madrid” project (S2009/TIC-1650)

Improving Student Enjoyment, Time Management, and Performance Using a Secure, Immersive Gamified Learning Platform

We analyze the thesis that a fully immersive gamified learning platform benefits students, improving their enjoyment, time management, and performance by engaging them with the subject matter. We assert that an immersive experience centered around a true game produces better results than exercises with just a few gamified elements (e.g., points, badges, and leaderboards), which are the most common forms of gamification. To this end, we present Riposte, an online gamified learning platform with a 2D action game at its core. To analyze student behavior and support less gamified assignments (to serve as contrasting examples), Riposte is designed to be a single, unified experience for students, with an integrated development environment (Jupyter Notebook) built in. In outlining this system, we give particular emphasis to its security, as it was developed for use in the "Introduction to Computer Security" course at our university. The topic of cybersecurity presents a number of interesting challenges for an online gamified learning platform – aspects of the framework must be insecure enough to be hackable, but secure enough not to be abused. We evaluate the system over three semesters, highlighting many pitfalls and lessons learned but, in the end, demonstrating that it successfully improved student enjoyment, time management, and performance. Furthermore, based on our experiences with the platform, we offer a number of suggestions and recommendations that we hope will help those considering gamification as a winning educational strategy in computer science, cybersecurity, and beyond.Doctor of Philosoph

Cotextual Android Education

With advances in mobile phone hardware, the demand for mobile applica- tions has risen drastically. This has resulted in mobile phones becoming a pop- ular new medium for application development. However, the body of knowledge for contextual examples and tutorials leaves much to be desired. As of January 2010, California Polytechnic State University has offered a mobile development class that teaches students how to write applications for phones running Google’s Android platform. This class aims at taking advantage of students’ current in- terest in mobile applications to teach them about difficult computer science topics. As a corollary, the class hopes to foster and encourage a sense of inde- pendence and entrepreneurship through having students design, implement, and publish their own applications to the Android Application Marketplace. The main contribution of this thesis project comes in the form of a series of detailed educational laboratory exercises and a system for grading student submissions in an automated fashion. These labs are designed to supplement the Android documentation by providing contextual examples, activities, and tutorials. It is therefore the goal of this thesis project to aid in transforming the class of mobile development students into a group of successful, practicing, mobile developers

Ripple: Concept-Based Interpretation for Raw Time Series Models in Education

Time series is the most prevalent form of input data for educational prediction tasks. The vast majority of research using time series data focuses on hand-crafted features, designed by experts for predictive performance and interpretability. However, extracting these features is labor-intensive for humans and computers. In this paper, we propose an approach that utilizes irregular multivariate time series modeling with graph neural networks to achieve comparable or better accuracy with raw time series clickstreams in comparison to hand-crafted features. Furthermore, we extend concept activation vectors for interpretability in raw time series models. We analyze these advances in the education domain, addressing the task of early student performance prediction for downstream targeted interventions and instructional support. Our experimental analysis on 23 MOOCs with millions of combined interactions over six behavioral dimensions show that models designed with our approach can (i) beat state-of-the-art educational time series baselines with no feature extraction and (ii) provide interpretable insights for personalized interventions. Source code: https://github.com/epfl-ml4ed/ripple/.Comment: Accepted as a full paper at AAAI 2023: 37th AAAI Conference on Artificial Intelligence (EAAI: AI for Education Special Track), 7-14 of February 2023, Washington DC, US

MOOCs: Expectations and Reality

This comprehensive study of MOOCs from the perspective of institutions of higher education includes an investigation of definitions and characteristics of MOOCs, their origins, institutional goals for developing and delivering MOOCs, how MOOC data is being used, a review of MOOC resource requirements and costs, and a compilation of ideas from 83 interviewees about MOOCs and the future of higher education. We identify six major goals for MOOC initiatives and assess the evidence regarding whether these goals are being met, or are likely to be in the future

Enhancing Student Assignment Completion Through Gamified Mobile Applications: A Motivational Approach

This thesis explores the design of digital tools to encourage motivation for students in higher education. A design is proposed for an original prototype app that makes use of gamification techniques. Theories on improving motivation are discussed in the literature and those most suited to be implemented within a digital prototype are selected. To measure how well the prototype performs, we specifically use the delivery for coursework to be handed in a timely manner as a metric. User consultation in the form of a questionnaire is used to create part of the specification used in the prototype design. The prototype app is tested on a limited number of participants and the results reported. Results show that students using a mobile application with gamification features would be motivated to start their assignments earlier and be motivated to complete them on time however results also show that specific gamification features have more split opinions than others for e.g., the extrinsic rewards that users can earn when completing assignments and challenges. These rewards have very little tangible value to them so when earned by users, it doesn’t provide much nor does it motivate them to want to achieve more, users would prefer rewards that have a direct impact on them for e.g., being able to extend assignment deadlines. The Thesis offers theoretical and practical contributions to research and design of motivation and the use of gamification in a mobile application

Value-Based Delivery Of Education: Moocs As Messengers

Value-based delivery of healthcare has been discussed in the literature for almost a decade.  The concept focuses on the patient and defines value as the improvement of patient outcomes divided by healthcare costs.  Further refinements, called the Triple Aim model, focus on improving patient outcomes, reducing treatment costs, and improving patient satisfaction.  Inspired by the MOOC (Massive Open Online Courses) movement in education, which should be seen as a wake-up message for educators, this paper attempts to reframe MOOCs, and related educational initiatives, within a Value-Based Delivery of Education (VBDE) model.  The VBDE model focuses on the student learner and defines value as the improvement of learning outcomes, reduction of educational costs, and improvement of student stakeholder satisfaction.   VBDE model elements are operationally defined, quantified, and used to identify and assess key components of MOOC and MOOC-related initiatives.  Implications for the successful delivery of value-based education in the future are discussed as part one of a two-paper series on the topic

Automated assessment of programming assignments : visual feedback, assignment mobility, and assessment of students' testing skills

The main objective of this thesis is to improve the automated assessment of programming assignments from the perspective of assessment tool developers. We have developed visual feedback on functionality of students' programs and explored methods to control the level of detail in visual feedback. We have found that visual feedback does not require major changes to existing assessment platforms. Most modern platforms are web based, creating an opportunity to describe visualizations in JavaScript and HTML embedded into textual feedback. Our preliminary results on the effectiveness of automatic visual feedback indicate that students perform equally well with visual and textual feedback. However, visual feedback based on automatically extracted object graphs can take less time to prepare than textual feedback of good quality. We have also developed programming assignments that are easier to port from one server environment to another by performing assessment on the client-side. This not only makes it easier to use the same assignments in different server environments but also removes the need for sandboxing the execution of students' programs. The approach will likely become more important in the future together with interactive study materials becoming more popular. Client-side assessment is more suitable for self-studying material than for grading because assessment results sent by a client are often too easy to falsify. Testing is an important part of programming and automated assessment should also cover students' self-written tests. We have analyzed how students behave when they are rewarded for structural test coverage (e.g. line coverage) and found that this can lead students to write tests with good coverage but with poor ability to detect faulty programs. Mutation analysis, where a large number of (faulty) programs are automatically derived from the program under test, turns out to be an effective way to detect tests otherwise fooling our assessment systems. Applying mutation analysis directly for grading is problematic because some of the derived programs are equivalent with the original and some assignments or solution strategies generate more equivalent mutants than others

An automated marking system for graphical user interfaces

This research investigates the feasibility and effectiveness of assessing students programming solutions to Graphical User Interface exercises in an automated fashion. Automated marking systems ease the burden on the staff involved in running a course and allow students to get results and feedback in a timely fashion. Several automated marking systems exist but are currently unable to mark GUIs. The inherent complexity of GUIs and the need for aesthetic analysis has rendered GUIs beyond the scope of most marking systems. The marking approach described in this thesis implements a number of novel concepts. By exploiting language design properties such as the hierarchical relationship between components, it was possible to develop a framework capable of testing and marking students' GUI programs. Introspectively analysing the interface enables the marking system to obtain access to the intrinsic elements contained within the GUI. Once access has been obtained, the tests can be performed on the actual interface components themselves rather than a mere representation. GUI assessment is more than functional testing, aesthetics play a major role in the creation of an interface. Existing aesthetic metrics do not provide the analytical capabilities required due to their failure to include colour. The distractive effects that colours have were quantified and incorporated into the metrics. The results of the dynamic and aesthetic testing show that through the implementation of the novel components detailed, the creation of a GUI marking system is feasible and its marking both consistent and effective. The design enables the system to return results in a timely fashion and the effects that colour has can be seen in the results of basic aesthetic testing