Enhancing university student engagement using online multiple choice questions and answers

For many education providers, student engagement can be a major issue. Given the positive correlation between engagement and good performance, providers are continually looking for ways to engage students in the learning process. The growth of student digital literacy, the wide proliferation of online tools and the understanding of why online gaming can be addictive have combined to create a set of tools that providers can leverage to enhance engagement. One such tool is Peerwise, https://peerwise.cs.auckland.ac.nz/, an online, multiple choice question (MCQ) and answer tool in which students create questions that are answered by other students. Why use MCQs? Using MCQs tests knowledge, provides reassurance of learning, identifies gaps and makes this data available to student and provider. Students use this information to focus their time on areas requiring additional work [1], benefiting from the early feedback provided. Formative assess- ments using MCQs are beneficial in preparing students for summative testing and are appreciated and liked by students [2]. Providers can use this information to determine how the material is being received and react accordingly. Students use Peerwise to create MCQs that are answered, rated and commented on by their peers. Students’ engagement in Peerwise earns trophies for contributing regular use and for providing feedback, all of which act to stimulate further engagement, using the principles of gamification. Bournemouth University, a public university in the UK with over 18,000 students, has been embedding Peerwise in under-graduate and post-graduate units since 2014. The results experienced by Bournemouth University have been beneficial and correlate with other studies of using Peerwise [3] [4]. A statistically significant improvement was seen by one cohort of students compared to the previous year where Peerwise was not used. However, no correlation was found between Peerwise participation and a student’s unit mark. The processes followed by Bournemouth University and the advantages and disadvantages, backed by qualitative and quantitative data, will be presented so that other institutions can gain an informed view of the merits of Peerwise for their own teaching and learning environments

Using Peerwise to improve engagement and learning.

This paper assesses the experiences of Bournemouth University in using the online multiple choice question (MCQ) tool, Peerwise, in student learning and engagement. MCQs are excellent for developing and testing knowledge, providing reassurance and identifying development needs. The creation of MCQs reinforces learning by tasking students to generate challenging questions. Peerwise supports self-direction and flexibility, which is embraced by students. Bournemouth University started embedding Peerwise within teaching units in 2014. The intention was to transform the approach of students towards the non-assessed elements of the unit. Peerwise was used in an undergraduate business unit consisting of 50 students over at 15 week period. 804 questions were created and 3,345 answers were recorded. 10% of the unit marks were allocated to Peerwise use. Qualitative feedback from students was very positive. Correlation analysis showed a very weak relationship, 0.120, between the number of questions answered and the overall unit mark. Self-assessment of the change in learning was statistically significantly better for students who used Peerwise compared to those who did not. Overall, the evaluation of the Peerwise was positive with many lessons learnt. Six recommendations for the further use of Peerwise were developed, including improving the scaffolding to students, refining the way quality is assessed and developing evaluation criteria

Comparing Code Explanations Created by Students and Large Language Models

Reasoning about code and explaining its purpose are fundamental skills for computer scientists. There has been extensive research in the field of computing education on the relationship between a student's ability to explain code and other skills such as writing and tracing code. In particular, the ability to describe at a high-level of abstraction how code will behave over all possible inputs correlates strongly with code writing skills. However, developing the expertise to comprehend and explain code accurately and succinctly is a challenge for many students. Existing pedagogical approaches that scaffold the ability to explain code, such as producing exemplar code explanations on demand, do not currently scale well to large classrooms. The recent emergence of powerful large language models (LLMs) may offer a solution. In this paper, we explore the potential of LLMs in generating explanations that can serve as examples to scaffold students' ability to understand and explain code. To evaluate LLM-created explanations, we compare them with explanations created by students in a large course ($n \approx 1000$) with respect to accuracy, understandability and length. We find that LLM-created explanations, which can be produced automatically on demand, are rated as being significantly easier to understand and more accurate summaries of code than student-created explanations. We discuss the significance of this finding, and suggest how such models can be incorporated into introductory programming education.Comment: 8 pages, 3 figures. To be published in Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V.

Cryptography in Grade 10: Core Ideas with Snap! and Unplugged

International audienceWe report our experience of an extracurricular online intervention on cryptography in Grade 10. Our first goal is to describe how we taught some fundamental cryptography ideas by making students encounter a progression of representative cryptosystems, from classical to modern, and discover their characteristics and limitations. We used Snap! (a visual programming language) to realize hands-on activities: block-programming playgrounds (a form of task-specific programming languages) to experiment with cryptosystems, and an interactive app to support an unplugged (albeit remote) Diffie-Hellman key agreement. After experimenting with each system, the students were involved in a Socratic discussion on how to overcome the discovered limitations, motivating the introduction of the following system in our path. Our second goal is to evaluate the students' perceptions and learning of cryptography core ideas. They appreciated the course and felt that, despite being remote, it was fun and engaging. According to the students, the course helped them understand the role of cryptography, CS, and Math in society and sparked their interest in cryptography and CS. The final assessment showed that the students well understood the cryptography ideas addressed. Our third goal is to discuss what worked and areas of improvement. The "remote-unplugged" Diffie-Hellman, where the meeting chat was a metaphor for the public channel, engaged the students in understanding this groundbreaking protocol. Overall, they praised the activities as engaging, even when challenging. However, a strong "instructor blindness" induced by remote teaching often prevented us from giving the students the right amount of guidance during the exploration activities

Mobile Scaffolding Application to Support Novice Learners of Computer Programming

Support for novice learners of computer programming can be provided by scaffolding the construction of programs. The ubiquity of mobile phones allows us to support learners whenever they wish to work on a program outside the classroom. This paper describes the development of an application that scaffolds the construction of programs on a mobile phone. The application was designed based on a five-level scaffolding framework and implemented on the Android platform.The application scaffolds the construction of programs on a mobile device by: (i) representing a program in parts; (ii) restricting a learner to complete the program in a certain order; (iii) enabling construction of a program one part at a time; (iv) providing instructions, steps, default code to be edited, hints, and error prompts where appropriate; and (v) fading the scaffolds as the learner progresses from one successfully completed and compiled program, to the next.Experiments are currently ongoing to test and evaluate the mobile application

"It's Weird That it Knows What I Want": Usability and Interactions with Copilot for Novice Programmers

Recent developments in deep learning have resulted in code-generation models that produce source code from natural language and code-based prompts with high accuracy. This is likely to have profound effects in the classroom, where novices learning to code can now use free tools to automatically suggest solutions to programming exercises and assignments. However, little is currently known about how novices interact with these tools in practice. We present the first study that observes students at the introductory level using one such code auto-generating tool, Github Copilot, on a typical introductory programming (CS1) assignment. Through observations and interviews we explore student perceptions of the benefits and pitfalls of this technology for learning, present new observed interaction patterns, and discuss cognitive and metacognitive difficulties faced by students. We consider design implications of these findings, specifically in terms of how tools like Copilot can better support and scaffold the novice programming experience.Comment: 26 pages, 2 figures, TOCH

Scaffolding Java programming on a mobile phone for novice learners

The ubiquity of mobile phones provides an opportunity to use them as a resource for construction of programs beyond the classroom. However, limitations of mobile phones impede their use as typical programming environments. This research proposes that programming environments on mobile phones should include scaffolding techniques specifically designed for mobile phones, and designed based on learners’ needs. This paper discusses the effectiveness of theoretically-derived scaffolding techniques to construct Java programs on a mobile phone. The results indicate that even though scaffolding techniques could support learners to program on a mobile phone, further modifications of the designed scaffolding techniques may be necessary in order to more effectively support programming on a mobile phone

The effectiveness of the Webtable-Datatable Conversion approach

Informatics education in Hungary is based on the National Base Curriculum (NAT) and the Frame Curricula. These documents contain the subjects (sciences), the number of classes for each subject and the requirements for each grade. According to the NAT2012, Informatics as a compulsory school subject is introduced in Grade 6. The filemanagement is among the first topics that students must learn according to the Frame Curricula. However, this is not their first encounter with filemanagent, since by the age of 12 most of the students are already active users of digital tools, and associated with the false assumptions of digital natives. Due to the late introduction, the filemanagement is one of the most neglected topics in informatics education. Nevertheless, this is one of the most important topics, since it is essential for further development in handling digital products. Our research group developed the Webtable-Datatable Conversion (WDC) highmathability method to teach filemanagement. This approach not only focuses on the main file operations but handles real world problems which require firm algorithm construction and datamanagement. The aim of the present study is to measure the effectiveness of the WDC approach with Grade 9 students, where the comparison of groups studying with the traditional and the WDC methods was carried out

Initial Evaluation of a Mobile Scaffolding Application that seeks to Support Novice Learners of Programming

The aim of this paper is to explore the use of an application that scaffolds the constructions of programs on a mobile device. The application was developed to support novice learners of programming outside the classroom. This paper reports on results of a first experiment conducted to evaluate the mobile application. The main research questions are: (i) whether the use of the application is effective in supporting construction of programs on a mobile device; and (ii) how the learners experienced the use of the mobile application. Data was collected by task completion, video and audio recording, and a questionnaire. A total of 18 first-year learners of programming from two African universities took part in the experiment by participating in focus groups. Almost two thirds of the learners completed two out of three programming exercises using the mobile application, with all the learners completing the first program. The results of the study suggest that the students found the mobile application useful, as evident from high rating of its features. The results also consisted of feedback from the learners on features that would make the application more usable. The findings suggest that the use of a mobile scaffolding application may support novice learners of programming outside the classroom. The outcomes of these results lead to a clearer understanding of how to design a mobile application that scaffolds the construction of programs on a mobile device