Practical and Ethical Challenges of Large Language Models in Education: A Systematic Scoping Review

Educational technology innovations leveraging large language models (LLMs) have shown the potential to automate the laborious process of generating and analysing textual content. While various innovations have been developed to automate a range of educational tasks (e.g., question generation, feedback provision, and essay grading), there are concerns regarding the practicality and ethicality of these innovations. Such concerns may hinder future research and the adoption of LLMs-based innovations in authentic educational contexts. To address this, we conducted a systematic scoping review of 118 peer-reviewed papers published since 2017 to pinpoint the current state of research on using LLMs to automate and support educational tasks. The findings revealed 53 use cases for LLMs in automating education tasks, categorised into nine main categories: profiling/labelling, detection, grading, teaching support, prediction, knowledge representation, feedback, content generation, and recommendation. Additionally, we also identified several practical and ethical challenges, including low technological readiness, lack of replicability and transparency, and insufficient privacy and beneficence considerations. The findings were summarised into three recommendations for future studies, including updating existing innovations with state-of-the-art models (e.g., GPT-3/4), embracing the initiative of open-sourcing models/systems, and adopting a human-centred approach throughout the developmental process. As the intersection of AI and education is continuously evolving, the findings of this study can serve as an essential reference point for researchers, allowing them to leverage the strengths, learn from the limitations, and uncover potential research opportunities enabled by ChatGPT and other generative AI models

A Knowledge Based Educational (KBEd) framework for enhancing practical skills in engineering distance learners through an augmented reality environment

The technology advancement has changed distance learning teaching and learning approaches, for example, virtual laboratories are increasingly used to deliver engineering courses. These advancements enhance the distance learners practical experience of engineering courses. While most of these efforts emphasise the importance of the technology, few have sought to understand the techniques for capturing, modelling and automating the on-campus laboratory tutors’ knowledge. The lack of automation of tutors’ knowledge has also affected the practical learning outcomes of engineering distance learners. Hence, there is a need to explore further on how to integrate the tutor's knowledge, which is necessary for imparting and assessing practical skills through current technological advances in distance learning. One approach to address this concern is through the use of Knowledge Based Engineering (KBE) principles. These KBE principles facilitate the utilisation of standardised methods for capturing, modelling and embedding experts’ knowledge into engineering design applications for the automation of product design. Hence, utilising such principles could facilitate, automating engineering laboratory tutors’ knowledge for teaching and assessing practical skills. However, there is limited research in the application of KBE principles in the educational domain. Therefore, this research explores the use of KBE principles to automate instructional design in engineering distance learning technologies. As a result, a Knowledge Based Educational (KBEd) framework that facilitates the capturing, modelling and automating on-campus tutors’ knowledge and introduces it to distance learning and teaching approaches. This study used a four-stage experimental approach, which involved rapid prototyping method to design and develop the proposed KBEd framework to a functional prototype. The developed prototype was further refined through internal and external expert group using face validity methods such as questionnaire, observation and discussion. The refined prototype was then evaluated through welding task use-case. The use cases were assessed by first year engineering undergraduate students with no prior experience of welding from Birmingham City University. The participants were randomly separated into two groups (N = 46). One group learned and practised basic welding in the proposed KBEd system, while the other learned and practised in the conventional on-campus environment. A concurrent validity assessment was used in determining the usefulness of the proposed system in learning hands-on practical engineering skills through proposed KBEd system. The results of the evaluation indicate that students who trained with the proposed KBEd system successfully gained the practical skills equivalent to those in the real laboratory environment. Although there was little performance variation between the two groups, it was rooted in the limitations of the system’s hardware. The learning outcomes achieved also demonstrated the successful application of KBE principles in capturing, modelling and transforming the knowledge from the real tutor to the AI tutor for automating the teaching and assessing of the practical skills for distance learners. Further the data analysis has shown the potential of KBEd to be extendable to other taught distance-learning courses involving practical skills

Privaatsust säilitava raalnägemise meetodi arendamine kehalise aktiivsuse automaatseks jälgimiseks koolis

Väitekirja elektrooniline versioon ei sisalda publikatsiooneKuidas vaadelda inimesi ilma neid nägemata? Öeldakse, et ei ole viisakas jõllitada. Õigus privaatsusele on lausa inimõigus. Siiski on inimkäitumises palju sellist, mida teadlased tahaksid uurida inimesi vaadeldes. Näiteks tahame teada, kas lapsed hakkavad vahetunnis rohkem liikuma, kui koolis keelatakse nutitelefonid? Selle välja selgitamiseks peaks teadlane küsima lapsevanematelt nõusolekut võsukeste vaatlemiseks. Eeldusel, et lapsevanemad annavad loa, oleks klassikaliseks vaatluseks vaja tohutult palju tööjõudu – mitu vaatlejat koolimajas iga päev piisavalt pikal perioodil enne ja pärast nutitelefoni keelu kehtestamist. Doktoritööga püüdsin lahendada korraga privaatsuse probleemi ja tööjõu probleemi, asendades inimvaatleja tehisaruga. Kaasaegsed masinõppe meetodid võimaldavad luua mudeleid, mis tuvastavad automaatselt pildil või videos kujutatud objekte ja nende omadusi. Kui tahame tehisaru, mis tunneb pildil ära inimese, tuleb moodustada masinõppe andmestik, kus on pilte inimestest ja pilte ilma inimesteta. Kui tahame tehisaru, mis eristaks videos madalat ja kõrget kehalist aktiivsust, on vaja vastavat videoandmestikku. Doktoritöös kogusingi andmestiku, kus video laste liikumisest on sünkroniseeritud puusal kantavate aktseleromeetritega, et treenida mudel, mis eristaks videopikslites madalamat ja kõrgemat liikumise intensiivsust. Koostöös Tehonoloogiainstituudi iCV laboriga arendasime välja videoanalüüsi sensori prototüübi, mis suudab reaalaja kiirusel hinnata kaamera vaateväljas olevate inimeste kehalise aktiivsuse taset. Just see, et tehisaru suudab tuletada videost kehalise aktiivsuse informatsiooni ilma neid videokaadreid salvestamata ega inimestele üldsegi näitamata, võimaldab vaadelda inimesi ilma neid nägemata. Väljatöötatud meetod on mõeldud kehalise aktiivsuse mõõtmiseks koolipõhistes teadusuuringutes ning seetõttu on arenduses rõhutatud privaatsuse kaitsmist ja teaduseetikat. Laiemalt vaadates illustreerib doktoritöö aga raalnägemistehnoloogiate potentsiaali töötlemaks visuaalset infot linnaruumis ja töökohtadel ning mitte ainult kehalise aktiivsuse mõõtmiseks kõrgete teaduseetika kriteerimitega. Siin ongi koht avalikuks aruteluks – millistel tingimustel või kas üldse on OK, kui sind jõllitab robot?  How to observe people without seeing them? They say it's not polite to stare. The right to privacy is considered a human right. However, there is much in human behavior that scientists would like to study via observation. For example, we want to know whether children will start moving more during recess if smartphones are banned at school? To figure this out, scientists would have to ask parental consent to carry out the observation. Assuming parents grant permission, a huge amount of labour would be needed for classical observation - several observers in the schoolhouse every day for a sufficiently long period before and after the smartphone ban. With my doctoral thesis, I tried to solve both the problem of privacy and of labor by replacing the human observer with artificial intelligence (AI). Modern machine learning methods allow training models that automatically detect objects and their properties in images or video. If we want an AI that recognizes people in images, we need to form a machine learning dataset with pictures of people and pictures without people. If we want an AI that differentiates between low and high physical activity in video, we need a corresponding video dataset. In my doctoral thesis, I collected a dataset where video of children's movement is synchronized with hip-worn accelerometers to train a model that could differentiate between lower and higher levels of physical activity in video. In collaboration with the ICV lab at the Institute of Technology, we developed a prototype video analysis sensor that can estimate the level of physical activity of people in the camera's field of view at real-time speed. The fact that AI can derive information about physical activity from the video without recording the footage or showing it to anyone at all, makes it possible to observe without seeing. The method is designed for measuring physical activity in school-based research and therefore highly prioritizes privacy protection and research ethics. But more broadly, the thesis illustrates the potential of computer vision technologies for processing visual information in urban spaces and workplaces, and not only for measuring physical activity or adhering to high ethical standards. This warrants wider public discussion – under what conditions or whether at all is it OK to have a robot staring at you?https://www.ester.ee/record=b555972

SEPEC conference proceedings: Hypermedia and Information Reconstruction. Aerospace applications and research directions

Papers presented at the conference on hypermedia and information reconstruction are compiled. The following subject areas are covered: real-world hypermedia projects, aerospace applications, and future directions in hypermedia research and development

“No powers, man!”: A student perspective on designing university smart building interactions

Smart buildings offer an opportunity for better performance and enhanced experience by contextualising services and interactions to the needs and practices of occupants. Yet, this vision is limited by established approaches to building management, delivered top-down through professional facilities management teams, opening up an interaction-gap between occupants and the spaces they inhabit. To address the challenge of how smart buildings might be more inclusively managed, we present the results of a qualitative study with student occupants of a smart building, with design workshops including building walks and speculative futuring. We develop new understandings of how student occupants conceptualise and evaluate spaces as they experience them, and of how building management practices might evolve with new sociotechnical systems that better leverage occupant agency. Our findings point to important directions for HCI research in this nascent area, including the need for HBI (Human-Building Interaction) design to challenge entrenched roles in building management

Applications of Artificial Intelligence in Military Training Simulation

This report is a survey of Artificial Intelligence (AI) technology contributions to military training. It provides an overview of military training simulation and a review of instructional problems and challenges which can be addressed by AI. The survey includes current as well as potential applications of AI, with particular emphasis on design and system integration issues. Applications include knowledge and skills training in strategic planning and decision making, tactical warfare operations, electronics maintenance and repair, as well as computer-aided design of training systems. The report describes research contributions in the application of AI technology to the training world, and it concludes with an assessment of future research directions in this area