Tell Me and I Forget, Involve Me and I Learn: Design and Evaluation of a Multimodal Conversational Agent for Supporting Distance Learning

The COVID-19 pandemic has shifted children’s learning routines from schools to their own homes, necessitating learning support solutions. This paper reports on a design science research project that combines augmented reality with a conversational agent to assist schoolchildren in learning complex subjects by providing verbal descriptions and interactive animations. Drawing on the theoretical foundations of multimedia learning, we derive three design principles to resolve seven issues associated with distance learning. The instantiated artifact augments text-based learning resources and facilitates learning in a contextsensitive manner through multimodal output. The proof-of-concept evaluation with 11 experienced teachers and researchers in the field of didactics confirms the usefulness of these design principles and suggests refinements of the artifact

It's all in the mime: Actions speak louder than words when teaching the cranial nerves

Cranial nerve (CN) knowledge is essential for students in health professions. Gestures and body movements (e.g., mime) have been shown to improve cognition and satisfaction with anatomy teaching. The aim of this pilot study was to compare the effectiveness of didactic lecturing with that of miming lecturing for student learning of the CNs. The research design involved exposure of the same group of students to didactic followed by miming lecturing of CNs. The effectiveness of each lecturing strategy was measured via pre‐ and post‐testing. Student perceptions of these strategies were measured by a survey. As an example of miming, gestures for CN VII included funny faces for muscles of facial expression, kangaroo vocalization for taste, spitting action for saliva production, and crying for lacrimal gland production. Accounting for extra duration of the miming lecture, it was shown that pre‐ to post‐test improvement was higher for the miming presentation than for the didactic (0.47 ± 0.03 marks/minute versus 0.33 ± 0.03, n = 39, P < 0.005). Students perceived that the miming lecture was more interactive, engaging, effective, and motivating to attend (mean on five‐point Likert scale: 4.62, 4.64, 4.56, 4.31, respectively) than the didactic lecture. In the final examination, performance was better (P < 0.001, n = 39) on the CN than on the non‐CN questions—particularly for students scoring ≤60%. While mediating factors need elucidation (e.g., learning due to repetition of content), this study's findings support the theory that gestures and body movements help learners to acquire anatomical knowledge. Anat Sci Educ 8: 584–592. © 2015 The Authors. Published by Wiley Periodicals, Inc. on behalf of the American Association of Anatomists

Teaching Neuroanatomy Virtually: Integrating an Interactive 3D E-Learning Resource for Enhanced Neuroanatomy Education

An interactive 3D e-learning module was developed to complement neuroanatomy instruction in both an undergraduate medicine neuroanatomy laboratory course, and an undergraduate systemic human anatomy course. The 3D e-learning resource provided students the opportunity to manipulate a dynamic 3D model to view structures from any desired angle, view deep cortical structures at high magnification, and add interactive structural labels. The study utilized a cross-over design, to separate participants into two groups. Each group completed baseline anatomy knowledge and spatial ability knowledge assessments, followed by access to either the 3D e-learning module or conventional learning resources. Participants completed a post-module anatomy knowledge assessment prior to accessing to the other learning modality. A final post-module knowledge assessment was administered following student exposure to the second learning modality. Students who initially accessed the 3D module scored significantly higher on the post-module knowledge assessment than the students who initially accessed the conventional anatomy resources. Participants who accessed the 3D learning resources following gross anatomy resources, significantly improved on the final post-module knowledge assessment. A negative correlation was observed between spatial ability and change in assessment score following access to the 3D module suggesting that students with low spatial ability experienced a greater positive effect on their learning of neuroanatomy following the use of the 3D learning module than students with higher spatial ability. A novel virtual syncretion assessment was also developed that assessed participants’ ability to place neuroanatomical structures in a partial 3D neuroanatomical model, rather than a conventional nominal response. Participants who initially utilized the 3D e-learning resource performed significantly better on the virtual syncretion assessment than participants who initially utilized the 2D e-learning resource. Participants who accessed the 3D e-learning resource subsequent to the 2D e-learning resource significantly improved their performance on the final virtual syncretion assessment. Results of this study could be used to inform the effective development and implementation of 3D e-learning resources to improve neuroanatomy instruction, particularly for students with low spatial ability

An investigation of mobile augmented reality-based learning features in cognitive and affective environments

This research focuses on the effectiveness of using mobile Augmented Reality (mAR) for learning. Prior research has focused primarily on developing virtual contents for Augmented Reality (AR) and has largely ignored AR in the mobile context. Herein, this research primarily aims to examine the effectiveness of learning through two modes: mobile Augmented Reality (mAR) and the Current Learning Mode (CLM). This research is extended to the development stage of a theoretical model, to evaluate the ability of mAR in improving the learning outcomes that guide a further consideration of growth in learning. The first phase of this thesis is to examine the impact of how mAR influences the learning outcomes in cognitive ability and affective learning outcomes. The cognitive outcome was measured by the experimental method of using pre/ post-test performance achievement, while the affective learning outcome was measured by perceived usefulness, self-efficacy and satisfaction. This research contributes to cognitive ability and affective learning by investigating the differences in the learning outcomes and performance achievements of mAR within a self-centred learning environment, a classroom. The findings show that students’ performance achievement, learning outcomes, perceived learning effectiveness and self-efficacy were greater in the mAR group, as compared to the CLM group. Second, a theoretical model was developed and analysed using Structural Equation Modelling (SEM). SEM examines significant relationships between the determinants that integrate and facilitate effective mAR-based learning environments. SEM produces a feasible alternative in measuring the causal relationship amongst the constructs. This model evaluates to implement mAR as a learning aid in student-centred learning and to evaluate the motivation among students through the features of mAR, due to the absence of an in-depth understanding of the motivation of mAR-based learning from the current literature. This model also provides an insight into the causal factors amongst the dimensions of mAR. Finally, in the model, the moderating effects of students’ characteristics, which include their experience and age, are investigated to determine the factors influencing mAR. The findings of this research will help to verify the learning effectiveness of mAR, to improve the learning experiences, learning outcomes and performance achievements of students. Based on the results, it is confirmed that mAR can be leveraged upon and used as an optimum learning tool, exemplifying the use of technology within an educational context. In the aspects of information retention and learning outcome enhancement, mAR is significant in education as it facilitates students’ understanding by supporting abstract ideas throughout the course, enabling the students to learn in a limited period. Based on the results, it can be concluded that mAR is a technology that aids students with a better understanding of the subject matter and hence, resulting in greater motivation. With regards to the model fitness via the analysis of goodness-of-fit, all the results are confirmed as appropriate and good fit. Also, the model also shows a positive causal path from the mAR features’ determinant. The thesis can also assist educational administrators and educational policy makers in gauging the importance of mAR as a learning tool. This helps mainly to overcome the issue of educators being criticised for the lack of real-life experience that is being exposed to students at the university level. Furthermore, academia can use the model’s findings as appropriate groundwork to initiate other related studies, and this will help to fill the gap in the mAR learning area

Teaching Science Lab Safety: Are Virtual Simulations Effective?

abstract: The purpose of this study was to investigate the impact of immersion on knowledge, cognitive load, and presence in a simulation designed to deliver a lesson on science lab safety training. 108 participants were randomly assigned to one of three conditions: high immersion (played an interactive simulation about lab safety in a VR headset), medium immersion (played the same interactive simulation on the computer), or low immersion (watched a video and read about lab safety procedures). Participants completed a pretest, a science lab safety training, a posttest (same as the pretest), a questionnaire with subjective presence questions, and a questionnaire with subjective cognitive load questions. Participants were again asked to complete a follow-up test (same as the pretest and posttest) a week later. The results revealed three significant findings: (a) Participants in the high and medium immersion conditions had significantly higher knowledge scores at posttest and follow-up than their peers in the low immersion condition, (b) Participants in the high and medium immersion conditions reported higher presence scores than participants in the low immersion conditions. (c) Correlation coefficients suggested that the higher the immersion and presence, the higher the knowledge scores are at posttest and follow-up. In addition, multiple hierarchical linear regression models were conducted out of which one was significant.Dissertation/ThesisDoctoral Dissertation Educational Technology 201

Meaningful Hand Gestures for Learning with Touch-based I.C.T.

The role of technology in educational contexts is becoming increasingly ubiquitous, with very few students and teachers able to engage in classroom learning activities without using some sort of Information Communication Technology (ICT). Touch-based computing devices in particular, such as tablets and smartphones, provide an intuitive interface where control and manipulation of content is possible using hand and finger gestures such as taps, swipes and pinches. Whilst these touch-based technologies are being increasingly adopted for classroom use, little is known about how the use of such gestures can support learning. The purpose of this study was to investigate how finger gestures used on a touch-based device could support learning

INTERFACE DESIGN FOR A VIRTUAL REALITY-ENHANCED IMAGE-GUIDED SURGERY PLATFORM USING SURGEON-CONTROLLED VIEWING TECHNIQUES

Initiative has been taken to develop a VR-guided cardiac interface that will display and deliver information without affecting the surgeons’ natural workflow while yielding better accuracy and task completion time than the existing setup. This paper discusses the design process, the development of comparable user interface prototypes as well as an evaluation methodology that can measure user performance and workload for each of the suggested display concepts. User-based studies and expert recommendations are used in conjunction to es­ tablish design guidelines for our VR-guided surgical platform. As a result, a better understanding of autonomous view control, depth display, and use of virtual context, is attained. In addition, three proposed interfaces have been developed to allow a surgeon to control the view of the virtual environment intra-operatively. Comparative evaluation of the three implemented interface prototypes in a simulated surgical task scenario, revealed performance advantages for stereoscopic and monoscopic biplanar display conditions, as well as the differences between three types of control modalities. One particular interface prototype demonstrated significant improvement in task performance. Design recommendations are made for this interface as well as the others as we prepare for prospective development iterations

Examination and Assessment of Commercial Anatomical E-Learning Tools: Software Usability, Dual-Task Paradigms and Learning

Technological innovation is changing the landscape of higher education, and the competing interests and responsibilities of today’s learners have propelled the movement of post-secondary courses into the online environment. In the anatomical sciences, commercialized e-learning tools have become a critical component for teaching the intricacies of the human body when physical classroom space and cadaveric resources are limited. This dissertation comparatively assessed the impact of two commercial anatomical e-learning tools (1) a simple 2-dimensional e-learning tool (A.D.A.M. Interactive Anatomy) and (2) a complex tool that allows for a 3-dimensional perspective (Netter’s 3D Interactive Anatomy). The comparison was then extended to include a traditional visual-kinesthetic method of studying anatomy (i.e. a physical skeleton). Applying cognitive load theory and working memory limitations as guiding principles, a dual-task assessment with cross over design was used to evaluate cognitive load. Students were assessed using baseline knowledge tests, observation task reaction times (a measure of cognitive load), mental rotation test scores (a measure of spatial ability) and anatomy post-tests (a measure of knowledge recall). Results from experiments carried out in this thesis suggest that the value of commercial anatomical e-learning tools cannot be assessed adequately on the basis of an educator’s, or a software developer’s, intuition alone. Despite the delivery benefits offered by e-learning tools and the positive feedback they often receive, this research demonstrates that neither commercial e-learning tool conferred any instructional advantage over textbook images. In fact, later results showed that the visual-kinesthetic experience of physically manipulating a skeleton yielded major positive impacts on knowledge recall that A.D.A.M. Interactive Anatomy, as a visual only tool, failed to deliver. The results of this dissertation also suggest that the design of e-learning tools can differentially influence students based on their spatial ability. Moreover our results suggest that learners with low spatial ability may also struggle to relate anatomical knowledge if they are examined on contralateral images. By objectively assessing commercial anatomical e-learning tools against traditional, visual-kinesthetic modalities, educators can be confident that the learning tool they select will give their students the best chance to acquire an understanding of human anatomy

Recent Advancements in Augmented Reality for Robotic Applications: A Survey

Robots are expanding from industrial applications to daily life, in areas such as medical robotics, rehabilitative robotics, social robotics, and mobile/aerial robotics systems. In recent years, augmented reality (AR) has been integrated into many robotic applications, including medical, industrial, human–robot interactions, and collaboration scenarios. In this work, AR for both medical and industrial robot applications is reviewed and summarized. For medical robot applications, we investigated the integration of AR in (1) preoperative and surgical task planning; (2) image-guided robotic surgery; (3) surgical training and simulation; and (4) telesurgery. AR for industrial scenarios is reviewed in (1) human–robot interactions and collaborations; (2) path planning and task allocation; (3) training and simulation; and (4) teleoperation control/assistance. In addition, the limitations and challenges are discussed. Overall, this article serves as a valuable resource for working in the field of AR and robotic research, offering insights into the recent state of the art and prospects for improvement

Multisensory learning in adaptive interactive systems

The main purpose of my work is to investigate multisensory perceptual learning and sensory integration in the design and development of adaptive user interfaces for educational purposes. To this aim, starting from renewed understanding from neuroscience and cognitive science on multisensory perceptual learning and sensory integration, I developed a theoretical computational model for designing multimodal learning technologies that take into account these results. Main theoretical foundations of my research are multisensory perceptual learning theories and the research on sensory processing and integration, embodied cognition theories, computational models of non-verbal and emotion communication in full-body movement, and human-computer interaction models. Finally, a computational model was applied in two case studies, based on two EU ICT-H2020 Projects, "weDRAW" and "TELMI", on which I worked during the PhD