Scalable Two-Minute Feedback: Digital, Lecture-Accompanying Survey as a Continuous Feedback Instrument

Detailed feedback on courses and lecture content is essential for their improvement and also serves as a tool for reflection. However, feedback methods are often only used sporadically, especially in mass courses, because collecting and analyzing feedback in a timely manner is often a challenge for teachers. Moreover, the current situation of the students or the changing workload during the semester are usually not taken into account either. For a holistic investigation, the article used a digital survey format as formative feedback which attempts to measure student stress in a quantitative part and to address the participants' reflection in a qualitative part, as well as to collect general suggestions for improvement (based on the so-called One-Minute Paper) at two educational institutions. The feedback during the semester is evaluated qualitatively and discussed on a meta-level and special features (e.g. reflections on student work ethic or other courses) are addressed. The results show a low, but constant rate of feedback. Responses mostly cover topics of the lecture content or organizational aspects and were intensively used to report issues within the lecture. In addition, artificial intelligence (AI) support in the form of a large language model was tested and showed promising results in summarizing the open-ended responses for the teacher. Finally, the experiences from the lecturers are reflected upon and the results as well as possibilities for improvement are discussed.Comment: accepted to the International Journal of Information and Education Technology (IJIET

AI-Enhanced Auto-Correction of Programming Exercises: How Effective is GPT-3.5?

Timely formative feedback is considered as one of the most important drivers for effective learning. Delivering timely and individualized feedback is particularly challenging in large classes in higher education. Recently Large Language Models such as GPT-3 became available to the public that showed promising results on various tasks such as code generation and code explanation. This paper investigates the potential of AI in providing personalized code correction and generating feedback. Based on existing student submissions of two different real-world assignments, the correctness of the AI-aided e-assessment as well as the characteristics such as fault localization, correctness of hints, and code style suggestions of the generated feedback are investigated. The results show that 73% of the submissions were correctly identified as either correct or incorrect. In 59% of these cases, GPT-3.5 also successfully generated effective and high-quality feedback. Additionally, GPT-3.5 exhibited weaknesses in its evaluation, including localization of errors that were not the actual errors, or even hallucinated errors. Implications and potential new usage scenarios are discussed

Developing Open Source Educational Resources for Machine Learning and Data Science

Education should not be a privilege but a common good. It should be openly accessible to everyone, with as few barriers as possible; even more so for key technologies such as Machine Learning (ML) and Data Science (DS). Open Educational Resources (OER) are a crucial factor for greater educational equity. In this paper, we describe the specific requirements for OER in ML and DS and argue that it is especially important for these fields to make source files publicly available, leading to Open Source Educational Resources (OSER). We present our view on the collaborative development of OSER, the challenges this poses, and first steps towards their solutions. We outline how OSER can be used for blended learning scenarios and share our experiences in university education. Finally, we discuss additional challenges such as credit assignment or granting certificates.Comment: 6 page

PLATON: Developing a Graphical Lesson Planning System for Prospective Teachers

Lesson planning is both an important and demanding task—especially as part of teacher training. This paper presents the requirements for a lesson planning system and evaluates existing systems regarding these requirements. One major drawback of existing software tools is that most are limited to a text- or form-based representation of the lesson designs. In this article, a new approach with a graphical, time-based representation with (automatic) analyses methods is proposed and the system architecture and domain model are described in detail. The approach is implemented in an interactive, web-based prototype called PLATON, which additionally supports the management of lessons in units as well as the modelling of teacher and student-generated resources. The prototype was evaluated in a study with 61 prospective teachers (bachelor’s and master’s preservice teachers as well as teacher trainees in post-university teacher training) in Berlin, Germany, with a focus on usability. The results show that this approach proofed usable for lesson planning and offers positive effects for the perception of time and self-reflection

Unterstützungsmöglichkeiten für die computerbasierte Planung von Unterricht

Das Erstellen von Unterrichtsentwürfen wird als Voraussetzung für effektiven, zielgerichteten Unterricht gesehen. Dies gilt insbesondere für angehende Lehrpersonen. Da viele Aspekte, wie z.B. Methoden, Fachinhalte, Standards und die Charakteristik der Lerngruppe, zur gleichen Zeit berücksichtigt und sinnvoll aufeinander abgestimmt werden müssen, handelt es sich um eine kreative und zugleich anspruchsvolle Aufgabe. Es liegt nahe, diesen aufwändigen Prozess der Planung durch spezialisierte Softwaresysteme zu unterstützen, die nicht nur Routineaufgaben erleichtern, sondern auch zur Reflexion anregen. Basierend auf einem Literatur-Review und Interviews mit Dozierenden, die Lehrpersonen ausbilden, werden Anforderungen an Unterrichtsentwürfe analysiert und diskutiert. Es werden dabei nicht nur die relevanten theoretischen Grundlagen betrachtet, sondern ebenfalls das Vorgehen bei der Planung, genutzte Hilfsmittel und besonders unterstützenswerte Aspekte aus der Praxis. Darauf aufbauend werden Anwendungsfälle und Anforderungen an ein Planungsunterstützungssystem systematisch abgeleitet. Existierende Planungssysteme werden vorgestellt und untersucht, wobei viele bereits elementare Anforderungen nicht erfüllen. Insbesondere gibt es kein System, das adaptives Feedback bereitstellt. Um diese Lücke zu schließen, wird eine graphische, zeitbasierte Notation zur Unterrichtsplanung vorgeschlagen. Weiterhin sind ein Ressourcen-Management, Analyse-Möglichkeiten und automatisches Feedback zentrale Bestandteile des Ansatzes. Der entwickelte Prototyp wurde mit über 100 Personen in mehreren Studien systematisch empirisch evaluiert. Im Fokus stand dabei nicht nur die Untersuchung der Gebrauchstauglichkeit des Ansatzes und des Prototyps, sondern auch wie sich die Nutzung des Prototyps auf die Qualität der Unterrichtsentwürfe auswirkt. Es wird aufgezeigt, dass der beschriebene Ansatz für die Planung von Unterricht geeignet ist und angehende Lehrpersonen auch zur Reflexion angeregt hat.Developing a lesson plan is considered as a prerequisite for effective, targeted instruction. This is particularly true for prospective teachers. Since many aspects such as methods, subject content, standards, and the characteristics of the learner group have to be taken into account and reasonably compiled, it is a creative as well as a demanding task. It stands to reason that this complex process of planning can be supported by specialized software systems that do not only facilitate routine tasks, but also stimulate reflection. Based on a literature-review and interviews with instructors of prospective teachers, requirements for lesson plans are analyzed and discussed. Not only relevant theoretical principles of lesson planning are considered, but also individual planning steps, tools used and practical aspects that would benefit particularly from further support. Building upon this, use cases and requirements for a lesson planning support system are derived systematically. Existing planning systems are presented and analyzed. It is argued that many systems do not meet even basic requirements. Especially there is no system that offers adaptive feedback. In order to fill this gap, a graphical time-based representation is proposed for lesson planning. Further central aspects of this approach are resource management, analysis capabilities and automatic feedback. A software architecture for a planning support system is presented that implements the aforementioned approach and meets the requirements. The developed prototype was empirically evaluated in several systematic studies involving over 100 participants. The focus to study was the usability of the approach and the prototype on the one hand, and how the use of the prototype affects the quality of the lesson plans on the other hand. In this thesis it is shown that the described approach for lesson planning is adequate and suitable to promote reflective thinking about one''s own planning activities

Engineering Hybrid Learning Communities: The Case of a Regional Parent Community

We present an approach (and a corresponding system design) for supporting regionally bound hybrid learning communities (i.e., communities which combine traditional face-to-face elements with web based media such as online community platforms, e-mail and SMS newsletters). The goal of the example community used to illustrate the approach was to support and motivate (especially hard-to-reach underprivileged) parents in the education of their young children. The article describes the design process used and the challenges faced during the socio-technical system design. An analysis of the community over more than one year indicates that the hybrid approach works better than the two separated “traditional” approaches separately. Synergy effects like advertising effects from the offline trainings for the online platform and vice versa occurred and regular newsletters turned out to have a noticeable effect on the community

Building a Corpus of Task-Based Grading and Feedback Systems for Learning and Teaching Programming

Using grading and feedback systems in the context of learning and teaching programming is quite common. During the last 20 to 40 years research results on several hundred systems and approaches have been published. Existing papers may tell researchers what works well in terms of educational support and how to make a grading and feedback system stable, extensible, secure, or sustainable. However, finding a solid basis for such kind of research is hard due to the vast amount of publications from a very diverse community. Hardly any recent systematic review includes data from more than 100 systems (most include less than 30). Hence, the authors started an endeavor to build a corpus of all task-based grading and feedback systems for learning and teaching programming that deal with source code and have been published in recent years. The intention is to provide the community with a solid basis for their research. The corpus is also designed to be updated and extended by the community with future systems. This paper describes the process of building the corpus and presents some meta-analysis that shed light on the involved research communities

Towards Open Science at the DELFI Conference

Despite the increasing awareness of Open Science within the educational technology community, conferences, such as DELFI, do not yet foster the publication of research data including software. To address this, we conducted a survey eliciting the community's needs, perspectives, and publication preferences. The analysis of 24 valid responses reveals a variety of research data formats used, and several uncertainties, e.\,g., regarding data ownership. Associated barriers comprise legal concerns and lacking resources to publish data. Nonetheless, researchers seem open for new publication formats. Moreover, we analyzed author's intentions to publish data related to their DELFI submissions in 2023 (n=66). Many researchers assume not to have data to share (n=28), or no intention to publish data in the future (n=16). Overall, the results imply a lack of awareness and recognition of data publications, so that further efforts and incentives are required to move toward Open Science practices in the DELFI community. The following article is linked to this dataset: Kiesler, Natalie; Röpke, René; Schiffner, Daniel; Schulz, Sandra; Strickroth, Sven; Ehlenz, Matthias; Heinemann, Birte; Wilhelm-Weidner, Arno (2024). Towards Open Science at the DELFI Conference. In: 22. Fachtagung Bildungstechnologien (DELFI). Bonn: Gesellschaft für Informatik e.V

ProFormA: An XML-based exchange format for programming tasks

Unterstützungssysteme für die Programmierausbildung sind weit verbreitet, doch gängige Standards für den Austausch von allgemeinen (Lern-) Inhalten und Tests erfüllen nicht die speziellen Anforderungen von Programmieraufgaben wie z. B. den Umgang mit komplexen Einreichungen aus mehreren Dateien oder die Kombination verschiedener (automatischer) Bewertungsverfahren. Dadurch können Aufgaben nicht zwischen Systemen ausgetauscht werden, was aufgrund des hohen Aufwands für die Entwicklung guter Aufgaben jedoch wünschenswert wäre. In diesem Beitrag wird ein erweiterbares XML-basiertes Format zum Austausch von Programmieraufgaben vorgestellt, das bereits von mehreren Systemen prototypisch genutzt wird. Die Spezifikation des Austauschformats ist online verfügbar [PFMA].Support systems for programming education are in widespread use. However common standards for the exchange of general (learning) content and tests do not meet the special requirements of programming tasks, e.g. dealing with complex submissions consisting of multiple files or the combination of different (automatic) evaluation and assessment procedures. Thus, due to missing interoperability, programming tasks cannot be exchanged between systems easily despite the fact that this would be desirable due to the high cost of the development of good tasks. In this paper, an extensible XML-based format for the exchange of programming tasks is presented, which is already used in multiple systems. The XML-format is available on-line [PFMA]