Investigating Real-time Touchless Hand Interaction and Machine Learning Agents in Immersive Learning Environments

The recent surge in the adoption of new technologies and innovations in connectivity, interaction technology, and artificial realities can fundamentally change the digital world. eXtended Reality (XR), with its potential to bridge the virtual and real environments, creates new possibilities to develop more engaging and productive learning experiences. Evidence is emerging that thissophisticated technology offers new ways to improve the learning process for better student interaction and engagement. Recently, immersive technology has garnered much attention as an interactive technology that facilitates direct interaction with virtual objects in the real world. Furthermore, these virtual objects can be surrogates for real-world teaching resources, allowing for virtual labs. Thus XR could enable learning experiences that would not bepossible in impoverished educational systems worldwide. Interestingly, concepts such as virtual hand interaction and techniques such as machine learning are still not widely investigated in immersive learning. Hand interaction technologies in virtual environments can support the kinesthetic learning pedagogical approach, and the need for its touchless interaction nature hasincreased exceptionally in the post-COVID world. By implementing and evaluating real-time hand interaction technology for kinesthetic learning and machine learning agents for self-guided learning, this research has addressed these underutilized technologies to demonstrate the efficiency of immersive learning. This thesis has explored different hand-tracking APIs and devices to integrate real-time hand interaction techniques. These hand interaction techniques and integrated machine learning agents using reinforcement learning are evaluated with different display devices to test compatibility. The proposed approach aims to provide self-guided, more productive, and interactive learning experiences. Further, this research has investigated ethics, privacy, and security issues in XR and covered the future of immersive learning in the Metaverse.<br/

Investigating Real-time Touchless Hand Interaction and Machine Learning Agents in Immersive Learning Environments

The recent surge in the adoption of new technologies and innovations in connectivity, interaction technology, and artificial realities can fundamentally change the digital world. eXtended Reality (XR), with its potential to bridge the virtual and real environments, creates new possibilities to develop more engaging and productive learning experiences. Evidence is emerging that thissophisticated technology offers new ways to improve the learning process for better student interaction and engagement. Recently, immersive technology has garnered much attention as an interactive technology that facilitates direct interaction with virtual objects in the real world. Furthermore, these virtual objects can be surrogates for real-world teaching resources, allowing for virtual labs. Thus XR could enable learning experiences that would not bepossible in impoverished educational systems worldwide. Interestingly, concepts such as virtual hand interaction and techniques such as machine learning are still not widely investigated in immersive learning. Hand interaction technologies in virtual environments can support the kinesthetic learning pedagogical approach, and the need for its touchless interaction nature hasincreased exceptionally in the post-COVID world. By implementing and evaluating real-time hand interaction technology for kinesthetic learning and machine learning agents for self-guided learning, this research has addressed these underutilized technologies to demonstrate the efficiency of immersive learning. This thesis has explored different hand-tracking APIs and devices to integrate real-time hand interaction techniques. These hand interaction techniques and integrated machine learning agents using reinforcement learning are evaluated with different display devices to test compatibility. The proposed approach aims to provide self-guided, more productive, and interactive learning experiences. Further, this research has investigated ethics, privacy, and security issues in XR and covered the future of immersive learning in the Metaverse.<br/

Multimedia Tool For Children Learning: Web Based Pop Up Book

Web-based multimedia pop-up book for children learning is intend to convert traditional learning process into a web using multimedia formatwhich are integrate with multiple media elements such as audio, video, graphics, text and animations. Currently the children have to pull tabs and open pages for the book to 'come to life'. So the purpose of this project is to produce web-based multimedia pop-up book for children learning. The objectives of this project are to create and design interactive multimedia learning book, to come out with user-friendly interface and interactive application and to develop multimedia book that be able to view, use and interact with the online application that providing true multimedia application. The scope of this project is to create and design multimedia web-based education book for children between 5 to 6 years old in order to help children in learning activities, reading skills and improves children computer literacy. The project will adopt methodology of 'code-and-fix' methodology. Three essential stages of development have been performed, which involved Design stage (design the page layout and design the action and animation), Implementation stage (implement by insert and modify required images and implement actions and animations), and finally Test stage (run the actions and compare with intended design). The result in designing successful multimedia web-based for children learning the relevant area are focused on content presentation and user-friendly interface design. In order to achieve the objective of this web-based multimedia tool for children learning, this project should be able facilitate the process of integrating activity into education through online application

Multimedia Presentation Tool Using SMIL

In this project, the author has to develop a tool for multimedia presentation using Synchronized Multimedia Integration Language, which is XML-based declarative language. It can synchronize all the multimedia elements in the presentation as the synchronization is one of the main issues in delivery of multimedia presentation. Recently, the content of lecture materials does not integrate all the multimedia elements (such as video, audio, text and images) in it. Hence, the integration of various multimedia elements will be accomplished in this project to make the content of lecture materials more interesting and understandable. The tool requires multimedia elements from the user before it can synchronize all those elements. In this project, the author practice several modules to develop the application such as selection module, recording module, and integration module. In this paper, the instructional design is discussed as a framework to build the application to be more effective. The author believes that with the adoption of instructional design in students' learning can boost their understanding of their course content. The tool for multimedia presentation can automatically synchronize the video of a lecture with the corresponding PowerPoint slides to generate easy to use and effective presentation

Decoding learning: the proof, promise and potential of digital education

With hundreds of millions of pounds spent on digital technology for education every year – from interactive whiteboards to the rise of one–to–one tablet computers – every new technology seems to offer unlimited promise to learning. many sectors have benefitted immensely from harnessing innovative uses of technology. cloud computing, mobile communications and internet applications have changed the way manufacturing, finance, business services, the media and retailers operate. But key questions remain in education: has the range of technologies helped improve learners’ experiences and the standards they achieve? or is this investment just languishing as kit in the cupboard? and what more can decision makers, schools, teachers, parents and the technology industry do to ensure the full potential of innovative technology is exploited? There is no doubt that digital technologies have had a profound impact upon the management of learning. institutions can now recruit, register, monitor, and report on students with a new economy, efficiency, and (sometimes) creativity. yet, evidence of digital technologies producing real transformation in learning and teaching remains elusive. The education sector has invested heavily in digital technology; but this investment has not yet resulted in the radical improvements to learning experiences and educational attainment. in 2011, the Review of Education Capital found that maintained schools spent £487 million on icT equipment and services in 2009-2010. 1 since then, the education system has entered a state of flux with changes to the curriculum, shifts in funding, and increasing school autonomy. While ring-fenced funding for icT equipment and services has since ceased, a survey of 1,317 schools in July 2012 by the british educational suppliers association found they were assigning an increasing amount of their budget to technology. With greater freedom and enthusiasm towards technology in education, schools and teachers have become more discerning and are beginning to demand more evidence to justify their spending and strategies. This is both a challenge and an opportunity as it puts schools in greater charge of their spending and use of technolog

VRBridge: a Constructivist Approach to Supporting Interaction Design and End-User Authoring in Virtual Reality

For any technology to become widely-used and accepted, it must support end-user authoring and customisation. This means making the technology accessible by enabling understanding of its design issues and reducing its technical barriers. Our interest is in enabling end-users to author dynamic virtual environments (VEs), specifically their interactions: player interactions with objects and the environment; and object interactions with each other and the environment. This thesis describes a method to create tools and design aids which enable end-users to design and implement interactions in a VE and assist them in building the requisite domain knowledge, while reducing the costs of learning a new set of skills. Our design method is based in constructivism, which is a theory that examines the acquisition and use of knowledge. It provides principles for managing complexity in knowledge acquisition: multiplicity of representations and perspectives; simplicity of basic components; encouragement of exploration; support for deep reflection; and providing users with control of their process as much as possible. We derived two main design aids from these principles: multiple, interactive and synchronised domain-specific representations of the design; and multiple forms of non-invasive and user-adaptable scaffolding. The method began with extensive research into representations and scaffolding, followed by investigation of the design strategies of experts, the needs of novices and how best to support them with software, and the requirements of the VR domain. We also conducted a classroom observation of the practices of non-programmers in VR design, to discover their specific problems with effectively conceptualising and communicating interactions in VR. Based on our findings in this research and our constructivist guidelines, we developed VRBridge, an interaction authoring tool. This contained a simple event-action interface for creating interactions using trigger-condition-action triads or Triggersets. We conducted two experimental evaluations during the design of VRBridge, to test the effectiveness of our design aids and the basic tool. The first tested the effectiveness of the Triggersets and additional representations: a Floorplan, a Sequence Diagram and Timelines. We used observation, interviews and task success to evaluate how effectively end-users could analyse and debug interactions created with VRBridge. We found that the Triggersets were effective and usable by novices to analyse an interaction design, and that the representations significantly improved end-user work and experience. The second experiment was large-scale (124 participants) and conducted over two weeks. Participants worked on authoring tasks which embodied typical interactions and complexities in the domain. We used a task exploration metric, questionnaires and computer logging to evaluate aspects of task performance: how effectively end-users could create interactions with VRBridge; how effectively they worked in the domain of VR authoring; how much enjoyment or satisfaction they experienced during the process; and how well they learned over time. This experiment tested the entire system and the effects of the scaffolding and representations. We found that all users were able to complete authoring tasks using VRBridge after very little experience with the system and domain; all users improved and felt more satisfaction over time; users with representations or scaffolding as a design aid completed the task more expertly, explored more effectively, felt more satisfaction and learned better than those without design aids; users with representations explored more effectively and felt more satisfaction than those with scaffolding; and users with both design aids learned better but did not improve in any other way over users with a single design aid. We also gained evidence about how the scaffolding, representations and basic tool were used during the evaluation. The contributions of this thesis are: an effective and efficient theory-based design method; a case study in the use of constructivism to structure a design process and deliver effective tools; a proof-of-concept prototype with which novices can create interactions in VR without traditional programming; evidence about the problems that novices face when designing interactions and dealing with unfamiliar programming concepts; empirical evidence about the relative effectiveness of additional representations and scaffolding as support for designing interactions; guidelines for supporting end-user authoring in general; and guidelines for the design of effective interaction authoring systems in general