Investigating Real-time Touchless Hand Interaction and Machine Learning Agents in Immersive Learning Environments

The recent surge in the adoption of new technologies and innovations in connectivity, interaction technology, and artificial realities can fundamentally change the digital world. eXtended Reality (XR), with its potential to bridge the virtual and real environments, creates new possibilities to develop more engaging and productive learning experiences. Evidence is emerging that thissophisticated technology offers new ways to improve the learning process for better student interaction and engagement. Recently, immersive technology has garnered much attention as an interactive technology that facilitates direct interaction with virtual objects in the real world. Furthermore, these virtual objects can be surrogates for real-world teaching resources, allowing for virtual labs. Thus XR could enable learning experiences that would not bepossible in impoverished educational systems worldwide. Interestingly, concepts such as virtual hand interaction and techniques such as machine learning are still not widely investigated in immersive learning. Hand interaction technologies in virtual environments can support the kinesthetic learning pedagogical approach, and the need for its touchless interaction nature hasincreased exceptionally in the post-COVID world. By implementing and evaluating real-time hand interaction technology for kinesthetic learning and machine learning agents for self-guided learning, this research has addressed these underutilized technologies to demonstrate the efficiency of immersive learning. This thesis has explored different hand-tracking APIs and devices to integrate real-time hand interaction techniques. These hand interaction techniques and integrated machine learning agents using reinforcement learning are evaluated with different display devices to test compatibility. The proposed approach aims to provide self-guided, more productive, and interactive learning experiences. Further, this research has investigated ethics, privacy, and security issues in XR and covered the future of immersive learning in the Metaverse.<br/

A Utility Framework for Selecting Immersive Interactive Capability and Technology for Virtual Laboratories

There has been an increase in the use of virtual reality (VR) technology in the education community since VR is emerging as a potent educational tool that offers students with a rich source of educational material and makes learning exciting and interactive. With a rise of popularity and market expansion in VR technology in the past few years, a variety of consumer VR electronics have boosted educators and researchers’ interest in using these devices for practicing engineering and science laboratory experiments. However, little is known about how such devices may be well-suited for active learning in a laboratory environment. This research aims to address this gap by formulating a utility framework to help educators and decision-makers efficiently select a type of VR device that matches with their design and capability requirements for their virtual laboratory blueprint. Furthermore, a framework use case is demonstrated by not only surveying five types of VR devices ranging from low-immersive to full-immersive along with their capabilities (i.e., hardware specifications, cost, and availability) but also considering the interaction techniques in each VR device based on the desired laboratory task. To validate the framework, a research study is carried out to compare these five VR devices and investigate which device can provide an overall best-fit for a 3D virtual laboratory content that we implemented based on the interaction level, usability and performance effectiveness

Investigating Real-time Touchless Hand Interaction and Machine Learning Agents in Immersive Learning Environments

The recent surge in the adoption of new technologies and innovations in connectivity, interaction technology, and artificial realities can fundamentally change the digital world. eXtended Reality (XR), with its potential to bridge the virtual and real environments, creates new possibilities to develop more engaging and productive learning experiences. Evidence is emerging that thissophisticated technology offers new ways to improve the learning process for better student interaction and engagement. Recently, immersive technology has garnered much attention as an interactive technology that facilitates direct interaction with virtual objects in the real world. Furthermore, these virtual objects can be surrogates for real-world teaching resources, allowing for virtual labs. Thus XR could enable learning experiences that would not bepossible in impoverished educational systems worldwide. Interestingly, concepts such as virtual hand interaction and techniques such as machine learning are still not widely investigated in immersive learning. Hand interaction technologies in virtual environments can support the kinesthetic learning pedagogical approach, and the need for its touchless interaction nature hasincreased exceptionally in the post-COVID world. By implementing and evaluating real-time hand interaction technology for kinesthetic learning and machine learning agents for self-guided learning, this research has addressed these underutilized technologies to demonstrate the efficiency of immersive learning. This thesis has explored different hand-tracking APIs and devices to integrate real-time hand interaction techniques. These hand interaction techniques and integrated machine learning agents using reinforcement learning are evaluated with different display devices to test compatibility. The proposed approach aims to provide self-guided, more productive, and interactive learning experiences. Further, this research has investigated ethics, privacy, and security issues in XR and covered the future of immersive learning in the Metaverse.<br/

Interacting with virtual reconstructions in museums: The etruscanning project

Starting from our experience in this domain, we discuss some fundamental concepts about the potentialities of the virtual reconstructions of cultural sites inside museums, with a specific focus on the communication needs, the design, the combination of media, the interaction interfaces, and the embodiment. We conceive a virtual reconstruction as a digital ecosystem, whose main peculiarities are (1) 3D reconstruction, (2) inclusivity, and (3) interactivity. A virtual reconstruction, in a wide sense, should integrate different levels of visualization, both realistic and symbolic; 3D models; metadata; storytelling; behaviors; and tools of visualization and interaction, in order to "reconstruct" and communicate a cultural context, an ecosystem where all the information is integrated. Despite the great advancements of the last years in the digitization process, computer graphics techniques, and archiving strategies, a basic limit of most of virtual museums is that they do not fire up the attention and the involvement of the public: they lack stimulating activities for visitors, narratives metaphors, and emotional impact. The interaction interfaces are not always simple to understand and to control in a few minutes, and they can generate a sense of frustration that causes users to abandon the application after a short and superficial approach. No gap should exist between knowledge and communication. But how can we translate the complexity of the knowledge in appealing to users and into simple applications that fit with the public's need? This article focuses on some communication rules and criteria that are often considered of minor importance by the researchers working in the field of digital cultural heritage but that are really essential to cultural transmission, especially inside museums. We believe that a stronger collaboration between research institutions and museums and among different disciplines would be recommended. Given this premise, we present the Etruscanning EU project, developed in 2011- 2013, focused on the virtual reconstruction of two important Etruscan tombs of the Orientalizing period: the Regolini-Galassi tomb in Cerveteri and the tomb n.5 of Monte Michele in Veii. © 2014 ACM

Developtment Virtual Laboratory of Digital Electronics Using Mobile Virtual Reality

The virtual laboratory of digital electronics using mobile virtual reality technology to save to cost of making real laboratory. Mobile virtual reality can use by student to practice digital electronics skill in their home. This study to finds the skill improvement of student in competency digital electronics using virtual laboratory with mobile virtual reality based. Virtual laboratory can help student to feel the real practice of laboratory in everywhere and every time. The feature of virtual laboratory is: (1) enable to do practice digital electronics in every time and everywhere, (2) student can do experiment like in laboratory using android phone, (3) support with many of android phone series. This study uses a research and development method using waterfall model that have five stage to develop a software application. The results show that the virtual laboratory can help student to improve their skill of digital electronics because easy feature and real time of virtual laborator

PENGEMBANGAN PERMAINAN “RUN BOBBY! FOOD!” DENGAN MENGGUNAKAN MICROSOFT KINECT

Tujuan dari penelitian ini adalah untuk menghasilkan sebuah permainan pada platform PC dengan menggunakan teknologi Microsoft Kinect. Teknologi ini digunakan untuk menyediakan pengalaman bermain yang lebih baik sehingga bisa lebih menarik perhatian pengunjung pameran. Microsoft Kinect digunakan untuk mendeteksi gerakan tubuh manusia sebagai pemain. Selain itu, Kinect juga digunakan sebagai media interaksi antara aplikasi dengan pengguna. Metode penelitian yang digunakan dibagi menjadi identifikasi masalah, pengumpulan data, perancangan aplikasi, implementasi, dan evaluasi. Pengumpulan data dilakukan dengan kuesioner, observasi, studi pustaka, dan wawancara kepada ahli. Perancangan aplikasi “Run Bobby! Food!” dilakukan dengan metode perancangan aplikasi scrum. Evaluasi dilakukan dengan dua pendekatan, yaitu kuesioner dan wawancara kepada ahli. Hasil dari penelitian ini adalah 96,1% responden puas dengan aplikasi.Kata Kunci: computer vision, exhibition, kinect, object detection, scru

Augmenting Reality with Intelligent Interfaces

It is clear that our daily reality will increasingly interface with virtual inputs. We already integrate the virtual into real life through constantly evolving sensor technologies embedded into our smartphones, digital assistants, and connected devices. Simultaneously, we seek more virtual input into our reality through intelligent interfaces for the applications that these devices can run in a context rich, socially connected, and personalized way. As we progress toward a future of ubiquitous Augmented Reality (AR) interfaces, it will be important to consider how this technology can best serve the various populations that can benefit most from the addition of these intelligent interfaces. This paper proposes a new terminological framework to discuss the way AR interacts with users. An intelligent interface that combines digital objects in a real-world context can be referred to as a Pose-Interfaced Presentation (PIP): Pose refers to user location and orientation in space; Interfaced means that the program responds to a user’s intention and actions in an intelligent way; and Presentation refers to the virtual object or data being layered onto the perceptive field of the user. Finally, various benefits of AR are described and examples are provided in the areas of education, worker training, and ESL learning

Computer-Assisted Interactive Documentary and Performance Arts in Illimitable Space

This major component of the research described in this thesis is 3D computer graphics, specifically the realistic physics-based softbody simulation and haptic responsive environments. Minor components include advanced human-computer interaction environments, non-linear documentary storytelling, and theatre performance. The journey of this research has been unusual because it requires a researcher with solid knowledge and background in multiple disciplines; who also has to be creative and sensitive in order to combine the possible areas into a new research direction. [...] It focuses on the advanced computer graphics and emerges from experimental cinematic works and theatrical artistic practices. Some development content and installations are completed to prove and evaluate the described concepts and to be convincing. [...] To summarize, the resulting work involves not only artistic creativity, but solving or combining technological hurdles in motion tracking, pattern recognition, force feedback control, etc., with the available documentary footage on film, video, or images, and text via a variety of devices [....] and programming, and installing all the needed interfaces such that it all works in real-time. Thus, the contribution to the knowledge advancement is in solving these interfacing problems and the real-time aspects of the interaction that have uses in film industry, fashion industry, new age interactive theatre, computer games, and web-based technologies and services for entertainment and education. It also includes building up on this experience to integrate Kinect- and haptic-based interaction, artistic scenery rendering, and other forms of control. This research work connects all the research disciplines, seemingly disjoint fields of research, such as computer graphics, documentary film, interactive media, and theatre performance together.Comment: PhD thesis copy; 272 pages, 83 figures, 6 algorithm

NMC horizon report: 2014 K-12 edition

The internationally recognized&nbsp;NMC Horizon Report&nbsp;series and regional&nbsp;NMC Technology Outlooks&nbsp;are part of the NMC Horizon Project, a comprehensive research venture established in 2002 that identifies and describes key trends, significant challenges, and emerging technologies likely to have a large impact over the coming five years in education around the globe. This volume, the&nbsp;NMC Horizon Report &gt; 2014 K-12 Edition&nbsp;will examines emerging technologies for their potential impact on and use in teaching, learning, and creative inquiry within the environment of pre-college education. The&nbsp;NMC Horizon Report &gt; 2014 K-12 Edition&nbsp;is the sixth in the K-12 series of reports and is produced by the NMC in collaboration with the Consortium for School Networking (CoSN), and made possible via the support of HP. View the work that produced the report at&nbsp;k12.wiki.nmc.org