Investigating Real-time Touchless Hand Interaction and Machine Learning Agents in Immersive Learning Environments

The recent surge in the adoption of new technologies and innovations in connectivity, interaction technology, and artificial realities can fundamentally change the digital world. eXtended Reality (XR), with its potential to bridge the virtual and real environments, creates new possibilities to develop more engaging and productive learning experiences. Evidence is emerging that thissophisticated technology offers new ways to improve the learning process for better student interaction and engagement. Recently, immersive technology has garnered much attention as an interactive technology that facilitates direct interaction with virtual objects in the real world. Furthermore, these virtual objects can be surrogates for real-world teaching resources, allowing for virtual labs. Thus XR could enable learning experiences that would not bepossible in impoverished educational systems worldwide. Interestingly, concepts such as virtual hand interaction and techniques such as machine learning are still not widely investigated in immersive learning. Hand interaction technologies in virtual environments can support the kinesthetic learning pedagogical approach, and the need for its touchless interaction nature hasincreased exceptionally in the post-COVID world. By implementing and evaluating real-time hand interaction technology for kinesthetic learning and machine learning agents for self-guided learning, this research has addressed these underutilized technologies to demonstrate the efficiency of immersive learning. This thesis has explored different hand-tracking APIs and devices to integrate real-time hand interaction techniques. These hand interaction techniques and integrated machine learning agents using reinforcement learning are evaluated with different display devices to test compatibility. The proposed approach aims to provide self-guided, more productive, and interactive learning experiences. Further, this research has investigated ethics, privacy, and security issues in XR and covered the future of immersive learning in the Metaverse.<br/

Investigating Real-time Touchless Hand Interaction and Machine Learning Agents in Immersive Learning Environments

The recent surge in the adoption of new technologies and innovations in connectivity, interaction technology, and artificial realities can fundamentally change the digital world. eXtended Reality (XR), with its potential to bridge the virtual and real environments, creates new possibilities to develop more engaging and productive learning experiences. Evidence is emerging that thissophisticated technology offers new ways to improve the learning process for better student interaction and engagement. Recently, immersive technology has garnered much attention as an interactive technology that facilitates direct interaction with virtual objects in the real world. Furthermore, these virtual objects can be surrogates for real-world teaching resources, allowing for virtual labs. Thus XR could enable learning experiences that would not bepossible in impoverished educational systems worldwide. Interestingly, concepts such as virtual hand interaction and techniques such as machine learning are still not widely investigated in immersive learning. Hand interaction technologies in virtual environments can support the kinesthetic learning pedagogical approach, and the need for its touchless interaction nature hasincreased exceptionally in the post-COVID world. By implementing and evaluating real-time hand interaction technology for kinesthetic learning and machine learning agents for self-guided learning, this research has addressed these underutilized technologies to demonstrate the efficiency of immersive learning. This thesis has explored different hand-tracking APIs and devices to integrate real-time hand interaction techniques. These hand interaction techniques and integrated machine learning agents using reinforcement learning are evaluated with different display devices to test compatibility. The proposed approach aims to provide self-guided, more productive, and interactive learning experiences. Further, this research has investigated ethics, privacy, and security issues in XR and covered the future of immersive learning in the Metaverse.<br/

GROUND<C>: A METAVERSE LEARNING STRATEGY FOR THE CREATIVE FIELDS

An alternative download link for the appendix of the thesis is here: http://www.citrinitas.com/PhD/2012Ayiter390522phd_appendix.zipIn this thesis I cover the theoretical framework and the practice based implications of bringing the fundamental principles of a cybernetic art educational strategy, the Groundcourse, which was developed and taught during the 1960’s in England by Roy Ascott, into the virtual, three dimensional builder’s world of the metaverse; to be implemented there as a non-institutional, voluntary, self-directed, adult oriented learning system for avatars – one which is expected to be taught by avatar instructors who will formulate the specifics of their curriculum and their methods based upon the cardinal tenets of the Groundcourse, which have been summarized by Roy Ascott as a flexible structure, “within which everything can find its place, and every individual his way,” which would give dimension and substance to the will to create and to change. In order to be able to set the groundwork for the adaptation of the Groundcourse’s principles to my model I have conducted literature reviews in experiential learning theories, with an emphasis on self-directed learning; as well as cybernetic learning. These I have combined with a survey of play theory and virtual world studies, particularly those focusing upon the avatar and metaverse creativity. From all of these I have woven together a foundation which I have combined with a visual documentation which may serve as case studies for my proposal. The new knowledge embodied through this thesis is a learning system for the creative fields that is designed specifically for the residents of online virtual worlds, and yet has its foundations in an earlier, well established and well regarded model

Latent Disentanglement for the Analysis and Generation of Digital Human Shapes

Analysing and generating digital human shapes is crucial for a wide variety of applications ranging from movie production to healthcare. The most common approaches for the analysis and generation of digital human shapes involve the creation of statistical shape models. At the heart of these techniques is the definition of a mapping between shapes and a low-dimensional representation. However, making these representations interpretable is still an open challenge. This thesis explores latent disentanglement as a powerful technique to make the latent space of geometric deep learning based statistical shape models more structured and interpretable. In particular, it introduces two novel techniques to disentangle the latent representation of variational autoencoders and generative adversarial networks with respect to the local shape attributes characterising the identity of the generated body and head meshes. This work was inspired by a shape completion framework that was proposed as a viable alternative to intraoperative registration in minimally invasive surgery of the liver. In addition, one of these methods for latent disentanglement was also applied to plastic surgery, where it was shown to improve the diagnosis of craniofacial syndromes and aid surgical planning

Markerless 3D human pose tracking through multiple cameras and AI: Enabling high accuracy, robustness, and real-time performance

Tracking 3D human motion in real-time is crucial for numerous applications across many fields. Traditional approaches involve attaching artificial fiducial objects or sensors to the body, limiting their usability and comfort-of-use and consequently narrowing their application fields. Recent advances in Artificial Intelligence (AI) have allowed for markerless solutions. However, most of these methods operate in 2D, while those providing 3D solutions compromise accuracy and real-time performance. To address this challenge and unlock the potential of visual pose estimation methods in real-world scenarios, we propose a markerless framework that combines multi-camera views and 2D AI-based pose estimation methods to track 3D human motion. Our approach integrates a Weighted Least Square (WLS) algorithm that computes 3D human motion from multiple 2D pose estimations provided by an AI-driven method. The method is integrated within the Open-VICO framework allowing simulation and real-world execution. Several experiments have been conducted, which have shown high accuracy and real-time performance, demonstrating the high level of readiness for real-world applications and the potential to revolutionize human motion capture.Comment: 19 pages, 7 figure

Augmented Reality in Learning Settings: A Systematic Analysis of its Benefits and Avenues for Future Studies

Despite its increasing use in various settings, Augmented Reality (AR) technology is still often considered experimental, partly due to a lack of clear understanding of the benefits of using AR. This study systematically reviews research on the use of AR in learning settings. Our analysis of 93 relevant articles offers 21 benefits related to the learning gains and outcomes of using AR. Our study shows that the positive effects of using AR on learners’ motivation and joy have been well-studied, whereas the effects on independent learning, concentration, spontaneous learning, critical thinking, and practical skills have not yet been examined in detail. Beyond classifying and discussing the benefits of using AR in learning settings, we elaborate avenues for future studies. We specifically point to the importance of conducting long-term studies to determine the value of using AR in learning beyond the initial novelty and exploring the integration of AR with other technologies

Discourses Across Periods of Time

This literature review explores the revolutionary effect of generative artificial intelligence (AI) and virtual reality (VR) on digital art history, specifically concentrating on their capacity to enable dialogical exchanges with historical figures and deepen the understanding of artworks. This study considers the current state of research, detecting key methodologies, areas of improvement, and possible challenges and ethical concerns. The example historical figure used in this analysis is the iconic Mexican artist Frida Kahlo. Kahlo’s refusal to correspond to a specific artistic style makes her an ideal subject for generative AI and VR-based investigation, offering fresh insights into her work. The incorporation of generative artificial intelligence and virtual reality technologies in humanities education, particularly in digital art history, has grown meaningful interest such as virtual museum exhibits and interactive art history course assignments offered in some universities. These tools allow immersive learning encounters, permitting students to become involved with art in advanced methods by using devices like Oculus VR. Text-based and image-based generative AI adds significantly to digital art history by producing new perceptions, depictions, and realizations from immense datasets. Additionally, the combination of generative AI and VR opens doors to vivid interactions with historical figures aided by natural language processing algorithms. While this tactic enhances historical and art history education, the following paper acknowledges the obstacles of artificial intelligence reproductions in presenting truthful responses. The paper addresses the ethical concerns linked to generative AI, stressing the importance of responsible usage in art history research. Ultimately, generative AI and VR integration promises to unlock new aspects of knowledge and understanding, further improving language learning, literature study, and cultural examination within the digital humanities

Exploring the Darkverse: A Multi-Perspective Analysis of the Negative Societal Impacts of the Metaverse

The Metaverse has the potential to form the next pervasive computing archetype that can transform many aspects of work and life at a societal level. Despite the many forecasted benefits from the metaverse, its negative outcomes have remained relatively unexplored with the majority of views grounded on logical thoughts derived from prior data points linked with similar technologies, somewhat lacking academic and expert perspective. This study responds to the dark side perspectives through informed and multifaceted narratives provided by invited leading academics and experts from diverse disciplinary backgrounds. The metaverse dark side perspectives covered include: technological and consumer vulnerability, privacy, and diminished reality, human–computer interface, identity theft, invasive advertising, misinformation, propaganda, phishing, financial crimes, terrorist activities, abuse, pornography, social inclusion, mental health, sexual harassment and metaverse-triggered unintended consequences. The paper concludes with a synthesis of common themes, formulating propositions, and presenting implications for practice and policy