51 research outputs found
A Trustworthy Automated Short-Answer Scoring System Using a New Dataset and Hybrid Transfer Learning Method
To measure the quality of student learning, teachers must conduct evaluations. One of the most efficient modes of evaluation is the short answer question. However, there can be inconsistencies in teacher-performed manual evaluations due to an excessive number of students, time demands, fatigue, etc. Consequently, teachers require a trustworthy system capable of autonomously and accurately evaluating student answers. Using hybrid transfer learning and student answer dataset, we aim to create a reliable automated short answer scoring system called Hybrid Transfer Learning for Automated Short Answer Scoring (HTL-ASAS). HTL-ASAS combines multiple tokenizers from a pretrained model with the bidirectional encoder representations from transformers. Based on our evaluation of the training model, we determined that HTL-ASAS has a higher evaluation accuracy than models used in previous studies. The accuracy of HTL-ASAS for datasets containing responses to questions pertaining to introductory information technology courses reaches 99.6%. With an accuracy close to one hundred percent, the developed model can undoubtedly serve as the foundation for a trustworthy ASAS system
Balancing Test Accuracy and Security in Computerized Adaptive Testing
Computerized adaptive testing (CAT) is a form of personalized testing that
accurately measures students' knowledge levels while reducing test length.
Bilevel optimization-based CAT (BOBCAT) is a recent framework that learns a
data-driven question selection algorithm to effectively reduce test length and
improve test accuracy. However, it suffers from high question exposure and test
overlap rates, which potentially affects test security. This paper introduces a
constrained version of BOBCAT to address these problems by changing its
optimization setup and enabling us to trade off test accuracy for question
exposure and test overlap rates. We show that C-BOBCAT is effective through
extensive experiments on two real-world adult testing datasets.Comment: The 24th International Conference on Artificial Intelligence in
Education (AIED 2023
Catalyzing Equity in STEM Teams: Harnessing Generative AI for Inclusion and Diversity
Collaboration is key to STEM, where multidisciplinary team research can solve
complex problems. However, inequality in STEM fields hinders their full
potential, due to persistent psychological barriers in underrepresented
students' experience. This paper documents teamwork in STEM and explores the
transformative potential of computational modeling and generative AI in
promoting STEM-team diversity and inclusion. Leveraging generative AI, this
paper outlines two primary areas for advancing diversity, equity, and
inclusion. First, formalizing collaboration assessment with inclusive analytics
can capture fine-grained learner behavior. Second, adaptive, personalized AI
systems can support diversity and inclusion in STEM teams. Four policy
recommendations highlight AI's capacity: formalized collaborative skill
assessment, inclusive analytics, funding for socio-cognitive research, human-AI
teaming for inclusion training. Researchers, educators, policymakers can build
an equitable STEM ecosystem. This roadmap advances AI-enhanced collaboration,
offering a vision for the future of STEM where diverse voices are actively
encouraged and heard within collaborative scientific endeavors.Comment: 21 pages, 0 figure, to be published in Policy Insights from
Behavioral and Brain Science
THE ROLE OF SIMULATION IN SUPPORTING LONGER-TERM LEARNING AND MENTORING WITH TECHNOLOGY
Mentoring is an important part of professional development and longer-term learning. The nature of longer-term mentoring contexts means that designing, developing, and testing adaptive learning sys-tems for use in this kind of context would be very costly as it would require substantial amounts of fi-nancial, human, and time resources. Simulation is a cheaper and quicker approach for evaluating the impact of various design and development decisions. Within the Artificial Intelligence in Education (AIED) research community, however, surprisingly little attention has been paid to how to design, de-velop, and use simulations in longer-term learning contexts. The central challenge is that adaptive learning system designers and educational practitioners have limited guidance on what steps to consider when designing simulations for supporting longer-term mentoring system design and development deci-sions.
My research work takes as a starting point VanLehn et al.’s [1] introduction to applications of simulated students and Erickson et al.’s [2] suggested approach to creating simulated learning envi-ronments. My dissertation presents four research directions using a real-world longer-term mentoring context, a doctoral program, for illustrative purposes. The first direction outlines a framework for guid-ing system designers as to what factors to consider when building pedagogical simulations, fundamen-tally to answer the question: how can a system designer capture a representation of a target learning context in a pedagogical simulation model? To illustrate the feasibility of this framework, this disserta-tion describes how to build, the SimDoc model, a pedagogical model of a longer-term mentoring learn-ing environment – a doctoral program. The second direction builds on the first, and considers the issue of model fidelity, essentially to answer the question: how can a system designer determine a simulation model’s fidelity to the desired granularity level? This dissertation shows how data from a target learning environment, the research literature, and common sense are combined to achieve SimDoc’s medium fidelity model. The third research direction explores calibration and validation issues to answer the question: how many simulation runs does it take for a practitioner to have confidence in the simulation model’s output? This dissertation describes the steps taken to calibrate and validate the SimDoc model, so its output statistically matches data from the target doctoral program, the one at the university of Saskatchewan. The fourth direction is to demonstrate the applicability of the resulting pedagogical model. This dissertation presents two experiments using SimDoc to illustrate how to explore pedagogi-cal questions concerning personalization strategies and to determine the effectiveness of different men-toring strategies in a target learning context.
Overall, this dissertation shows that simulation is an important tool in the AIED system design-ers’ toolkit as AIED moves towards designing, building, and evaluating AIED systems meant to support learners in longer-term learning and mentoring contexts. Simulation allows a system designer to exper-iment with various design and implementation decisions in a cost-effective and timely manner before committing to these decisions in the real world
Multimodality of AI for Education: Towards Artificial General Intelligence
This paper presents a comprehensive examination of how multimodal artificial
intelligence (AI) approaches are paving the way towards the realization of
Artificial General Intelligence (AGI) in educational contexts. It scrutinizes
the evolution and integration of AI in educational systems, emphasizing the
crucial role of multimodality, which encompasses auditory, visual, kinesthetic,
and linguistic modes of learning. This research delves deeply into the key
facets of AGI, including cognitive frameworks, advanced knowledge
representation, adaptive learning mechanisms, strategic planning, sophisticated
language processing, and the integration of diverse multimodal data sources. It
critically assesses AGI's transformative potential in reshaping educational
paradigms, focusing on enhancing teaching and learning effectiveness, filling
gaps in existing methodologies, and addressing ethical considerations and
responsible usage of AGI in educational settings. The paper also discusses the
implications of multimodal AI's role in education, offering insights into
future directions and challenges in AGI development. This exploration aims to
provide a nuanced understanding of the intersection between AI, multimodality,
and education, setting a foundation for future research and development in AGI
AI in Learning: Designing the Future
AI (Artificial Intelligence) is predicted to radically change teaching and learning in both schools and industry causing radical disruption of work. AI can support well-being initiatives and lifelong learning but educational institutions and companies need to take the changing technology into account. Moving towards AI supported by digital tools requires a dramatic shift in the concept of learning, expertise and the businesses built off of it. Based on the latest research on AI and how it is changing learning and education, this book will focus on the enormous opportunities to expand educational settings with AI for learning in and beyond the traditional classroom. This open access book also introduces ethical challenges related to learning and education, while connecting human learning and machine learning. This book will be of use to a variety of readers, including researchers, AI users, companies and policy makers
AI in Learning: Designing the Future
AI (Artificial Intelligence) is predicted to radically change teaching and learning in both schools and industry causing radical disruption of work. AI can support well-being initiatives and lifelong learning but educational institutions and companies need to take the changing technology into account. Moving towards AI supported by digital tools requires a dramatic shift in the concept of learning, expertise and the businesses built off of it. Based on the latest research on AI and how it is changing learning and education, this book will focus on the enormous opportunities to expand educational settings with AI for learning in and beyond the traditional classroom. This open access book also introduces ethical challenges related to learning and education, while connecting human learning and machine learning. This book will be of use to a variety of readers, including researchers, AI users, companies and policy makers
A Closer Look into Recent Video-based Learning Research: A Comprehensive Review of Video Characteristics, Tools, Technologies, and Learning Effectiveness
People increasingly use videos on the Web as a source for learning. To
support this way of learning, researchers and developers are continuously
developing tools, proposing guidelines, analyzing data, and conducting
experiments. However, it is still not clear what characteristics a video should
have to be an effective learning medium. In this paper, we present a
comprehensive review of 257 articles on video-based learning for the period
from 2016 to 2021. One of the aims of the review is to identify the video
characteristics that have been explored by previous work. Based on our
analysis, we suggest a taxonomy which organizes the video characteristics and
contextual aspects into eight categories: (1) audio features, (2) visual
features, (3) textual features, (4) instructor behavior, (5) learners
activities, (6) interactive features (quizzes, etc.), (7) production style, and
(8) instructional design. Also, we identify four representative research
directions: (1) proposals of tools to support video-based learning, (2) studies
with controlled experiments, (3) data analysis studies, and (4) proposals of
design guidelines for learning videos. We find that the most explored
characteristics are textual features followed by visual features, learner
activities, and interactive features. Text of transcripts, video frames, and
images (figures and illustrations) are most frequently used by tools that
support learning through videos. The learner activity is heavily explored
through log files in data analysis studies, and interactive features have been
frequently scrutinized in controlled experiments. We complement our review by
contrasting research findings that investigate the impact of video
characteristics on the learning effectiveness, report on tasks and technologies
used to develop tools that support learning, and summarize trends of design
guidelines to produce learning video
More than Model Documentation: Uncovering Teachers' Bespoke Information Needs for Informed Classroom Integration of ChatGPT
ChatGPT has entered classrooms, but not via the typical route of other
educational technology, which includes comprehensive training, documentation,
and vetting. Consequently, teachers are urgently tasked to assess its
capabilities to determine potential effects on student learning and instruct
their use of ChatGPT. However, it is unclear what support teachers have and
need and whether existing documentation, such as model cards, provides adequate
direction for educators in this new paradigm. By interviewing 22 middle- and
high-school teachers, we connect the discourse on AI transparency and
documentation with educational technology integration, highlighting the
critical information needs of teachers. Our findings reveal that teachers
confront significant information gaps, lacking clarity on exploring ChatGPT's
capabilities for bespoke learning tasks and ensuring its fit with the needs of
diverse learners. As a solution, we propose a framework for interactive model
documentation that empowers teachers to navigate the interplay between
pedagogical and technical knowledge
Deep learning based Arabic short answer grading in serious games
Automatic short answer grading (ASAG) has become part of natural language processing problems. Modern ASAG systems start with natural language preprocessing and end with grading. Researchers started experimenting with machine learning in the preprocessing stage and deep learning techniques in automatic grading for English. However, little research is available on automatic grading for Arabic. Datasets are important to ASAG, and limited datasets are available in Arabic. In this research, we have collected a set of questions, answers, and associated grades in Arabic. We have made this dataset publicly available. We have extended to Arabic the solutions used for English ASAG. We have tested how automatic grading works on answers in Arabic provided by schoolchildren in 6th grade in the context of serious games. We found out those schoolchildren providing answers that are 5.6 words long on average. On such answers, deep learning-based grading has achieved high accuracy even with limited training data. We have tested three different recurrent neural networks for grading. With a transformer, we have achieved an accuracy of 95.67%. ASAG for school children will help detect children with learning problems early. When detected early, teachers can solve learning problems easily. This is the main purpose of this research
- …