Technology-enhanced Assessment of Thinking Skills in Engineering Sciences

Assessment gilt allgemein als eines der wichtigsten Elemente in der Aus und Weiterbildung. Mit dem Einzug digitaler Technologien in Assessment-Prozesse (auch als E-Assessment bezeichnet), wurden neue Möglichkeiten für personalisierte, unmittelbare und eindrucksvolle Erfahrungen beim Assessment eröffnet. In Zeiten, in denen hochentwickelte, digitale Lernplattformen die Art und Weise verändern, was, wann und wie gelernt werden kann, ist es umso verwunderlicher, wie eingeschränkt die vorhandenen Methoden für die technologie-gestützte Bewertung des Lernens sind. Deutlich wird dies durch die Tatsache, dass aktuelle E-Assessment-Systeme sich größtenteils auf das Replizieren von traditionellen Tests mit Stift und Papier beschränken. Folglich bedarf es neuer Lösungen für die Identifikation, Sammlung, Analyse und Interpretation von Informationen über das individuelle Lernen. Die Berücksichtigung der Anforderungen an das Lernen im 21. Jahrhundert spielt dabei eine entscheidende Rolle. In Erkenntnis dieser Notwendigkeit präsentiert diese Arbeit ein neuartiges Architekturmodell für personalisierte und interaktive E-Assessment-Systeme und -Werkzeuge. Es erlaubt die Integration und Nutzung von interaktiven und immersiven Werkzeugen (z.B. Simulationen oder Animationen) innerhalb von Fragen und Tests, und ermöglicht diesen, sich an die individuellen Charakteristiken der Prüflinge (z.B. Vorwissen, Kontext und Vorlieben) anzupassen. Während das erste Hauptmerkmal (Didaktische Interaktivität) der Annahme gerecht wird, dass Lernen ein Ergebnis von Interaktionen sowie der aktiven Auseinandersetzung mit der jeweiligen Thematik ist, adressiert das zweite Hauptmerkmal (Personalisierung) die bei vielen E-Assessment-Systemen vorherrschende one-size-fits-all Strategie. Die Arbeit beschreibt die Struktur der grundlegenden Komponenten des Architekturmodells. Ein konsistentes Nutzermodell, ein generisches Domänenmodell sowie ein flexibles Adaptionsmodell bilden den zentralen Kern des Gesamtmodells und repräsentieren die Basis für das adaptive Verhalten. Komplettiert wird das Architekturmodell durch eine Komponente für die Modellierung von Fragen sowie einer Komponente für die Durchführung der spezifiierten Adaptionen. Darüber hinaus präsentiert die Arbeit die Implementierung des Architekturmodells durch das webbasierte E-Assessment-System askMe! sowie deren Erprobung und Evaluation nach pädagogischen (Lernunterstützung) sowie technischen (Gebrauchstauglichkeit und Nutzungserlebnis) Gesichtspunkten. Die Ergebnisse dieser Arbeit eröffnen neue Möglichkeiten für zukunftsweisende (E-Assessment-)Systeme, welche in der Lage sind, die Bedürfnisse und Charakteristiken Einzelner zu berücksichtigen sowie mehr Kreativität bei der Beantwortung durch Interaktion mit digitalen Werkzeugen ermöglichen.Assessment is generally recognized as one of the most important elements of an educational experience. Since digital technologies found their way into assessment processes (referred to as technology-enhanced assessment or eassessment), newpossibilities for more personalized, immediate and engaging assessment experiences were opened up. However, especially in current times when sophisticated digital learning environments, mostly enriched by multimedia, virtual/augmented reality technologies, change the way what can be learned, when and how, the methods of assessing students’ learning that have so far been developed are surprisingly limited. This can be demonstrated by the fact that current e-assessment practices simply imitate or replicate traditional pen-and-paper assessments. Consequently, new solutions are needed to identify, gather, analyze and interpret information about students’ learning, especially considering the requirements of the 21st century. In recognizing this need, this thesis proposes a novel architectural model for personalized and interactive e-assessment systems and tools. It allows integrating and using interactive and immersive tools (e.g., simulations or animations) into questions and tests, and enables tailoring them to students’ individual characteristics (e.g., prior knowledge, context and preferences). While the former key feature (didactic interactivity) takes into account the assumption that learning is the result of interaction and the active engagement with the subject matter, respectively, the latter one (personalization) tackles the one-size-fits-all approach mostly applied in traditional e-assessment settings. Furthermore, the thesis describes the structure and the constituent components of the architectural model. A consistent user model, a generic domain model and a flexible adaptation model build up the central part of the overall model and represent the fundamental basis for the adaptive behavior. Each model is managed by an own component and has well-defined interfaces to each other. Additionally, the architectural model is complemented by a question modeling component responsible for representing (interactive) questions, responses, etc. and finally, an adaptive testing engine component that performs the actual adaptations. Moreover, this thesis presents the implementation of the architectural model by the web-based e-assessment system askMe!. It also describes how this system was trialed and evaluated from a pedagogical (learning support) and technical (usability and user experience) point of view. The research and development performed in this thesis open up new opportunities for advanced e-assessment systems, which are able to consider the needs and characteristics of students and allow for more creativity in answering by interacting with digital tools in a variety of ways

Learning flexibility: the environment and a case study

This paper outlines the flexible student learning environment in the Faculty of Engineering and Surveying, before concentrating on evaluating one online learning option. This Faculty provides a variety of high quality on-campus, distance education and on-line academic programmes and various learning strategies for the heterogeneous student cohort (national and international). By accessing appropriate flexible learning and different learning experiences, students are empowered to determine learning opportunities and methodologies to suit their personal needs. The off-campus mode study may disadvantage students since they don’t have the benefit of face-to-face instructions or to participate in formative assessments delivered informally in lectures. This may lead to feelings of remoteness and isolation leading to poorer learning, lower results in assessments, and may also contribute to drop-out rates, particularly in first year courses. To overcome this inequity, the usual training materials presented for a first year course in 2005 were supplemented with PowerPoint lectures, enhanced with synchronous audio, and a series of quizzes to be used as formative assessments. The lectures and quizzes were presented online via a course web site and were designed to become an integral part of the learning experience. An evaluation of the effectiveness of these strategy demonstrated improved students' learning, a positive contribution to the learning experience, increased enjoyment of the course, and a strong learning motivator. Students reported feeling less disenfranchised with the university and having a greater affinity with the lecturer

The artisan and the artist. Innovation enables transformation

Technologies Excellence Group, for theCurriculum for Excellence Group for SG (commissioned by/for Mike Russell-Cabinet Secy on Education

Framework to Enhance Teaching and Learning in System Analysis and Unified Modelling Language

Cowling, MA ORCiD: 0000-0003-1444-1563; Munoz Carpio, JC ORCiD: 0000-0003-0251-5510Systems Analysis modelling is considered foundational for Information and Communication Technology (ICT) students, with introductory and advanced units included in nearly all ICT and computer science degrees. Yet despite this, novice systems analysts (learners) find modelling and systems thinking quite difficult to learn and master. This makes the process of teaching the fundamentals frustrating and time intensive. This paper will discuss the foundational problems that learners face when learning Systems Analysis modelling. Through a systematic literature review, a framework will be proposed based on the key problems that novice learners experience. In this proposed framework, a sequence of activities has been developed to facilitate understanding of the requirements, solutions and incremental modelling. An example is provided illustrating how the framework could be used to incorporate visualization and gaming elements into a Systems Analysis classroom; therefore, improving motivation and learning. Through this work, a greater understanding of the approach to teaching modelling within the computer science classroom will be provided, as well as a framework to guide future teaching activities

Integrating Technology With Student-Centered Learning

Reviews research on technology's role in personalizing learning, its integration into curriculum-based and school- or district-wide initiatives, and the potential of emerging digital technologies to expand student-centered learning. Outlines implications

Aligning Best Practices in Student Success and Career Preparedness: An Exploratory Study to Establish Pathways to STEM Careers for Undergraduate Minority Students

Undergraduate minority retention and graduation rates in STEM disciplines is a nationally recognized challenge for workforce growth and diversification. The Benjamin Banneker Scholars Program (BBSP) was a five-year undergraduate study developed to increase minority student retention and graduation rates at an HBCU. The program structure utilized a family model as a vehicle to orient students to the demands of college. Program activities integrated best K-12 practices and workforce skillsets to increase academic preparedness and career readiness. Findings revealed that a familial atmosphere improved academic performance, increased undergraduate research, and generated positive perceptions of faculty mentoring. Retention rates among BBSP participants averaged 88% compared to 39% among non-participant STEM peers. The BBSP graduation rate averaged 93% compared to 20% for non-participants. BBSP participants were more likely to gain employment in a STEM field or enter into a professional study. This paper furthers the body of research on STEM workforce diversity and presents a transferrable model for other institutions

A Working Model for Evaluating Academic Excellence in Geoscience Education, Undergraduate, and K-12

This article discusses a model for academic excellence in the geosciences which can be used for K-12 and undergraduate education. The model combines criteria for excellence derived from studies of the National Research Council, the National Academy of Sciences, and other national professional organizations. It consists of two major components: opportunity for students to learn, and attainment at the highest structural and multidimensional levels. The author concludes that this model will be most useful as a working model for evaluation, as a reference for comparison, and as a guide for reaching excellence in geoscience education efforts. Educational levels: Graduate or professional

Responsible research and innovation in science education: insights from evaluating the impact of using digital media and arts-based methods on RRI values

The European Commission policy approach of Responsible Research and Innovation (RRI) is gaining momentum in European research planning and development as a strategy to align scientific and technological progress with socially desirable and acceptable ends. One of the RRI agendas is science education, aiming to foster future generations' acquisition of skills and values needed to engage in society responsibly. To this end, it is argued that RRI-based science education can benefit from more interdisciplinary methods such as those based on arts and digital technologies. However, the evidence existing on the impact of science education activities using digital media and arts-based methods on RRI values remains underexplored. This article comparatively reviews previous evidence on the evaluation of these activities, from primary to higher education, to examine whether and how RRI-related learning outcomes are evaluated and how these activities impact on students' learning. Forty academic publications were selected and its content analysed according to five RRI values: creative and critical thinking, engagement, inclusiveness, gender equality and integration of ethical issues. When evaluating the impact of digital and arts-based methods in science education activities, creative and critical thinking, engagement and partly inclusiveness are the RRI values mainly addressed. In contrast, gender equality and ethics integration are neglected. Digital-based methods seem to be more focused on students' questioning and inquiry skills, whereas those using arts often examine imagination, curiosity and autonomy. Differences in the evaluation focus between studies on digital media and those on arts partly explain differences in their impact on RRI values, but also result in non-documented outcomes and undermine their potential. Further developments in interdisciplinary approaches to science education following the RRI policy agenda should reinforce the design of the activities as well as procedural aspects of the evaluation research