Using design-based research to develop a Mobile Learning Framework for Assessment Feedback

Students’ lack of engagement with their assessment feedback and the lack of dialogue and communication for feedback are some of the issues that affect educational institutions. Despite the affordance that mobile technologies could bring in terms of assessment feedback, research in this area is scarce. The main obstacle for research on mobile learning assessment feedback is the lack of a cohesive and unified mobile learning framework. This paper thus presents a Mobile Learning Framework for Assessment Feedback (MLFAF), developed using a design-based research approach. The framework emerged from the observation of, and reflection upon, the different stages of a research project that investigated the use of a mobile web application for summative and formative assessment feedback. MLFAF can be used as a foundation to study the requirements when developing and implementing wide-scale mobile learning initiatives that underpin longitudinal practices, as opposed to short-term practices. The paper also provides design considerations and implementation guidelines for the use of mobile technology in assessment feedback to increase student engagement and foster dialogic feedback communication channels

Teaching Algorithms and Data Structures: A Tale of Two Approaches

In Computing Science, Algorithms and Data Structures (ADS) are fundamental as they give learners a solid foundation in programming. ADS require abstract thinking and problem-solving abilities from students instead of memorisation – of formula or procedures, for example - without a clear understanding of the concepts. This paper reveals how master’s students (<i>n</i> = 35) perceived and experienced the different activities used to teach ADS in traditional lecture and flipped classroom environments. Results showed that the individual activities that scored higher were problem-solving and peer discussion in the flipped classroom. Regardless of the teaching approach, many students did not access the prior preparation teaching material mainly due to the lack of time. Findings also highlight the importance of communicating the intentions and pedagogical goals better. Findings have wider implication for practitioners facing the same challenges. It is hoped that the experiences reported in this paper can help practitioners to understand better the benefits and pitfalls that could emerge when converting an ADS class or similar to the flipped model

Evaluation of a mobile web application for assessment feedback

This paper reports an evaluation of a mobile web application, “MyFeedBack”, that can deliver both feedback and marks on assignments to students from their lecturer. It enables them to use any device anywhere, any time to check on, and receive their feedback. It keeps the feedback private to the individual student. It enables and successfully fosters dialogue about the feedback between the students and the educator. Feedback and marks were already being delivered using the institution’s learning environment/management system “Moodle”. The study used a sequential explanatory mixed-method approach. Two hundred thirty-nine (239) participants were reported on their experiences of receiving feedback and divided among several groups: (a) feedback delivered in “Moodle”, (b) formative feedback in “MyFeedBack”, and (c) summative feedback in “MyFeedBack”. Overall, results showed a statistically significant more positive attitude towards “MyFeedBack” than “Moodle”, with the summative assessment subgroup being more positive than the formative subgroup. There was an unprecedented increase in communication and feedback dialogue between the lecturer and the students. Qualitative results enriched and complemented the findings. The paper provides guidelines for an enabling technology for assessment feedback. These offer insight into the extent to which any of the new apps and functionalities that have become available since this study might likely be favourably viewed by learners and help achieve the desired pedagogical outcomes. These include: (1) accessible using any device, making feedback accessible anywhere, anytime; (2) display feedback first (before the grade/mark); (3) enable personalisation of group feedback by the teacher; (4) provide privacy for each student; (5) facilitate dialogue and communication about the feedback; and (6) include a monitoring feature. Three goals already put forward in the literature—(1) making the feedback feel more personal, (2) getting a quicker turnround by making it easier for the teachers to achieve this, and (3) prompting more dialogue between the educators and students—are advanced by this study which shows how they can be supported by software, and that when they are achieved then users strongly approve them

Educators’ pre-Covid19 lived experience of the assessment and feedback policy at a UK higher education institution

This study was conducted before the Covid19 pandemic. It used an interpretative thematic analysis approach to inquire about educators’ experiences of assessment and feedback policy and their views on assessment feedback. Eighteen teaching staff from a UK-based higher education institution expressed their opinions on various factors that facilitated or hindered the adoption of the assessment and feedback policy, along with their perceptions of assessment and feedback. The policy required that teaching staff provide feedback using technology such as Turnitin, with a two-week turnaround. Three themes were developed from the interpretative thematic analysis: 1) assessment and feedback policy as a personal and professional challenge, 2) mixed perceptions on the effectiveness of feedback, and 3) facilitating conditions for a successful assessment and feedback policy implementation. Workload and time constraints appeared to be the most prominent issues across all the interviews, affecting educators at every level of their professional and personal experiences. Insufficient time for attending meetings, providing timely and quality feedback, conducting research, pursuing professional development, and engaging in innovative strategies created tension between educators’ engagement and commitment to enhancing students learning and the pressure imposed by assessment and feedback policy demands. This research has implications for both educators and their professional development. Additionally, it can inform the university’s internal development of assessment practices and possibly broaden the understanding of assessment and feedback, particularly regarding the potential mismatch between the assessment and feedback policy and educators’ experiences and expectations

Developing a Dual Dashboard Early Detection System

This paper describes the development of `StudentsAtRisk', a prototype early detection system. It is based on engagement with course materials and can be used to identify students who are falling behind by automatically flagging them. The system also allows instructors to flag these students manually. On their dashboard, students can flag themselves, as engagement with the material might not reveal all those who struggle

Using Students' Affective State as a Measure of CS Lab Risk in an Early Detection System

This paper presents a dual dashboard early warning system which uses students’ affective state as a measure of risk. Affective state has been shown to influence CS1 performance, and specific states such as frustration have been linked to attrition. The software administers affective surveys to students using a series of 2-dimensional grids. Students then complete a qualitative journal entry. Risk weights are assigned to students based on the journal response’s sentiment analysis and whether student’s 2-dimensional grid responses fall within configurable 'danger zone' bounds. The early warning system automatically flags students as needing support if the responses’ combined risk weights exceed configurable thresholds. Additionally, flags can be assigned manually, either by instructors or by students themselves

Feedback on Teaching: Non-standard Minute Paper Methods

The importance of feedback in the learning and teaching context is widely recognised. In recent years, its primary focus has been on the provision of feedback to students, a unidirectional flow of information from educators to students on their formative and summative assignments. Feedback on teaching makes learning visible; however, this visibility depends on the teacher awareness of their impact on students. The uptake of end of class student feedback is relatively small in large classes. This paper reports on three lecturers’ account in collecting and responding to regular feedback on teaching in the School of Computing Science at the University of Glasgow. The lecturers’ accounts of their experience may be used as a starting point for educators willing to implement regular routine feedback on their teaching. Based on our experience, we propose guidelines emphasising structure and regularity in the collection of feedback on teaching

Do Current Online Coding Tutorial Systems Address Novice Programmer Difficulties?

Mastering programming skills is a multifaceted challenge, particularly for novice learners. While abundant existing literature examines students’ learning obstacles and proposes potential resolutions—predominantly through questionnaire-based methodologies—this study presents a fresh perspective. This paper focuses on identifying programming learning impediments as presented in the literature, paired with an exploration of the supportive features intrinsic to online coding platforms that could potentially mitigate these difficulties. The findings reveal that several of these online coding systems lack crucial features that could effectively address the learning difficulties experienced by novice programmers. This lack stresses an urgent call to create more robust and learner-centric environments that better facilitate the acquisition of programming skills for beginners, thus bridging the identified gap in the existing pedagogical tools

Experience Report: Thinkathon -- Countering an "I Got It Working" Mentality with Pencil-and-Paper Exercises

Goal-directed problem-solving labs can lead a student to believe that the most important achievement in a first programming course is to get programs working. This is counter to research indicating that code comprehension is an important developmental step for novice programmers. We observed this in our own CS-0 introductory programming course, and furthermore, that students weren't making the connection between code comprehension in labs and a final examination that required solutions to pencil-and-paper comprehension and writing exercises, where sound understanding of programming concepts is essential. Realising these deficiencies late in our course, we put on three 3-hour optional revision evenings just days before the exam. Based on a mastery learning philosophy, students were expected to work through a bank of around 200 pencil-and-paper exercises. By comparison with a machine-based hackathon, we called this a Thinkathon. Students completed a pre and post questionnaire about their experience of the Thinkathon. While we find that Thinkathon attendance positively influences final grades, we believe our reflection on the overall experience is of greater value. We report that: respected methods for developing code comprehension may not be enough on their own; novices must exercise their developing skills away from machines; and there are social learning outcomes in programming courses, currently implicit, that we should make explicit

Traditional vs non-traditional assessment activities as learning indicators of student learning : teachers' perceptions

In online settings, some teachers express reservations about relying only on traditional assessments (e.g., tests, assignments, exams, etc.) as trustworthy instruments to evaluate students' understanding of the content accurately. A previous qualitative study revealed that the richness of online environments has allowed teachers to use traditional assessments (anything contributing to the final grade) and non-traditional assessment-based activities (not factored into the final grade but useful in gauging student knowledge) to assess their students' learning status. This study aims to compare the perceived accuracy of both types of assessment activities as indicators of student learning. A total of 124 participants engaged in online teaching completed a self-report instrument. The results revealed a significant difference in teachers' perceptions of the accuracy of traditional assessment activities (M = 3.16; SD =. 442) compared to non-traditional assessment activities (M = 3.05, SD =. 521), t (122) = -2.64, p =. 009 with small effect size (eta =. 02). No significant gender differences were observed in the perceptions of the accuracy of either assessment activities type. The most commonly employed traditional assessment activities were “final exams” (85.5%) and “individual assignments” (83.9%). In comparison, the most common non-traditional assessment methods to evaluate students' knowledge were “questions on previously taught content” (79.8%) and “asking students questions about current content during the lecture” (79%). A one-way analysis of variance revealed no significant differences in perceptions of the accuracy of traditional and non-traditional assessment activities among teachers with varying years of experience (up to 10 years, 11–15 years, and 16+ years). The findings suggest that certain non-traditional assessment activities can also be as accurate as traditional learning activities. Moreover, non-assessment-related activities are perceived to be effective learning indicators. This study has implications for academic institutions and educators interested in supplementing traditional approaches to assessing student learning with non-traditional methods