Development of augmented reality serious games with a vibrotactile feedback jacket

Background: In the past few years, augmented reality (AR) has rapidly advanced and has been applied in different fields. One of the successful AR applications is the immersive and interactive serious games, which can be used for education and learning purposes. Methods: In this project, a prototype of an AR serious game is developed and demonstrated. Gamers utilize a head-mounted device and a vibrotactile feedback jacket to explore and interact with the AR serious game. Fourteen vibration actuators are embedded in the vibrotactile feedback jacket to generate immersive AR experience. These vibration actuators are triggered in accordance with the designed game scripts. Various vibration patterns and intensity levels are synthesized in different game scenes. This article presents the details of the entire software development of the AR serious game, including game scripts, game scenes with AR effects design, signal processing flow, behavior design, and communication configuration. Graphics computations are processed using the graphics processing unit in the system. Results /Conclusions: The performance of the AR serious game prototype is evaluated and analyzed. The computation loads and resource utilization of normal game scenes and heavy computation scenes are compared. With 14 vibration actuators placed at different body positions, various vibration patterns and intensity levels can be generated by the vibrotactile feedback jacket, providing different real-world feedback. The prototype of this AR serious game can be valuable in building large-scale AR or virtual reality educational and entertainment games. Possible future improvements of the proposed prototype are also discussed in this article

MoFaNeRF: Morphable Facial Neural Radiance Field

We propose a parametric model that maps free-view images into a vector space of coded facial shape, expression and appearance with a neural radiance field, namely Morphable Facial NeRF. Specifically, MoFaNeRF takes the coded facial shape, expression and appearance along with space coordinate and view direction as input to an MLP, and outputs the radiance of the space point for photo-realistic image synthesis. Compared with conventional 3D morphable models (3DMM), MoFaNeRF shows superiority in directly synthesizing photo-realistic facial details even for eyes, mouths, and beards. Also, continuous face morphing can be easily achieved by interpolating the input shape, expression and appearance codes. By introducing identity-specific modulation and texture encoder, our model synthesizes accurate photometric details and shows strong representation ability. Our model shows strong ability on multiple applications including image-based fitting, random generation, face rigging, face editing, and novel view synthesis. Experiments show that our method achieves higher representation ability than previous parametric models, and achieves competitive performance in several applications. To the best of our knowledge, our work is the first facial parametric model built upon a neural radiance field that can be used in fitting, generation and manipulation. The code and data is available at https://github.com/zhuhao-nju/mofanerf.Comment: accepted to ECCV2022; code available at http://github.com/zhuhao-nju/mofaner

Comparison of 1064-nm Nd:YAG picosecond laser using fractional micro-lens array vs. ablative fractional 2940-nm Er:YAG laser for the treatment of atrophic acne scar in Asians: a 20-week prospective, randomized, split-face, controlled pilot study

BackgroundThe 1064-nm Nd:YAG picosecond lasers using fractional micro-lens array (P-MLA) was a promising therapy for skin resurfacing. However, no studies have compared P-MLA with ablative fractional 2940-nm Er:YAG lasers (AF-Er) in the treatment of atrophic acne scars.ObjectivesTo evaluate the efficacy and safety of P-MLA and AF-Er for the treatment of atrophic acne scars.MethodsWe performed a prospective, randomized, split-face, controlled pilot study. Thirty-one Asian patients with mild to moderate atrophic acne scars underwent four consecutive sessions of randomized split-face treatment with P-MLA and AF-Fr at 4-week intervals. The efficacy of the two devices were evaluated by Echelle d’Evaluation Clinique des Cicatrices d’acne (ECCA) grading scale, Investigator’s Global Assessment (IGA) score and patient’s satisfaction. VISIA analysis was also performed to evaluate the pore and skin texture. Adverse events were recorded at each follow-up.ResultsThe P-MLA afforded comparable clinical responses in scar appearance as AF-Er based on the investigator’s assessments (ECCA percent reduction: 39.11% vs. 43.73%; IGA score: 2.97 ± 0.65 vs. 3.16 ± 0.68; P > 0.05 for both). However, the result of patient satisfaction indicated the AF-Er-treated side achieved a slightly greater improvement in scar appearance (3.97 ± 0.78 vs. 3.55 ± 0.71; P < 0.05). Overall, the two devices did not differ largely in terms of efficacy. VISIA analysis revealed similar changing patterns of the pore and skin texture between two devices. For safety profiles, no serious side effects were reported on both sides. The P-MLA showed lower pain level, shortened duration of crust shed and edema, and less occurrence of PIH (P < 0.05 for all).ConclusionCompared with AF-Er, P-MLA afforded comparable effect and more safety profiles in treating atrophic acne scars in Asian patients.Clinical trial registrationClinicalTrials.gov, identifier NCT 05686603

NeAI: A Pre-convoluted Representation for Plug-and-Play Neural Ambient Illumination

Recent advances in implicit neural representation have demonstrated the ability to recover detailed geometry and material from multi-view images. However, the use of simplified lighting models such as environment maps to represent non-distant illumination, or using a network to fit indirect light modeling without a solid basis, can lead to an undesirable decomposition between lighting and material. To address this, we propose a fully differentiable framework named neural ambient illumination (NeAI) that uses Neural Radiance Fields (NeRF) as a lighting model to handle complex lighting in a physically based way. Together with integral lobe encoding for roughness-adaptive specular lobe and leveraging the pre-convoluted background for accurate decomposition, the proposed method represents a significant step towards integrating physically based rendering into the NeRF representation. The experiments demonstrate the superior performance of novel-view rendering compared to previous works, and the capability to re-render objects under arbitrary NeRF-style environments opens up exciting possibilities for bridging the gap between virtual and real-world scenes. The project and supplementary materials are available at https://yiyuzhuang.github.io/NeAI/.Comment: Project page: <a class="link-external link-https" href="https://yiyuzhuang.github.io/NeAI/" rel="external noopener nofollow">https://yiyuzhuang.github.io/NeAI/</a

Visualize and Learn Sorting Algorithms in Data Structure Subject in a Game-based Learning

The Data Structure subject is an essential Computer Science subject. Sorting algorithms are important topics in Data Structure where students are expected to learn how various sorting algorithms work and their time complexities. Some sorting algorithms may easily cause confusions to novice students, as they usually find it challenging to understand and memorize these algorithms. There is a need to find a means of technology enhanced learning to improve the learning process of students. Game based learning is a pedagogy where students learn through game playing. This mode of learning could effectively engage students to focus on the learning topics more efficiently. The study uses a sorting algorithm serious game to allow students to learn four types of sorting algorithms: Bubble sort, Selection sort, Insertion sort and Quick sort. The students would carry out self-directed learning lecture materials in the serious game, followed by refreshing their learning using a visualizer, and lastly reinforce their learning through playing a sorting serious game. Two groups of students participate in the experiment, a control group and an experiment group. The experiment group that sues the sorting algorithm games achieves better results, compared to the control group who learns without the serious game. Game-based learning provides a positive learning experience to the students that could improve the learning effectiveness. Coupled with technology such as VR headsets as a future upgrade, it would be a niche factor that would create an immersive learning experience to engage the students and enhance their learning in a virtual environment

Interactive Virtual Reality Game for Online Learning of Science Subject in Primary Schools

Education plays an important role in nurturing children. COVID-19 pandemic brings challenges or disruptions to school education, due to school closures in some countries. Science subject in primary schools is unique as hands-on experiments are important learning components. Its learning process may be affected, as a new norm of online learning or home-based learning. This research project creates a serious game on science subject for primary school students aging within 10 to 11 years old using virtual reality (VR) technology. It consists of three virtual learning phases. Phase 1 explains theories of science topics on electricity and electric circuits. Phase 2 provides interactive hands-on experiment exercises where students can practice theory knowledge learned in the previous phase. An interactive quiz session is offered to reinforce the learning in Phase 3. Interactive VR features enable primary school students learning abstract science concepts in an interesting way compared to conventional classroom settings. Meticulous design attentions have been placed in the details such as visual instructions, voice instructions, speech tempo, animations, and colorful graphics to create a sense of realism and keep students actively engaged. Preliminary case study has been conducted with 10 students at primary schools in Singapore to evaluate learning effectiveness in this research

Evaluations of virtual and augmented reality technology-enhanced learning for higher education

Virtual reality (VR) has good potential to promote technology-enhanced learning. Students can benefit from immersive visualization and intuitive interaction in their learning of abstract concepts, complex structures, and dynamic processes. This paper is interested in evaluating the effects of VR learning games in a Virtual and Augmented Reality Technology-Enhanced Learning (VARTeL) environment within an engineering education setting. A VARTeL flipped classroom is established in the HIVE learning hub at Nanyang Technological University (NTU) Singapore for the immersive and interactive learning. Experiments are designed for the university students conducting the learning, with three interactive and immersive VR games related to science, technology, engineering and mathematics (STEM), i.e., virtual cells, a virtual F1 racing car, and vector geometry. These VR games are a part of the VARTeL apps designed in-house at NTU for STEM education. Quantitative and qualitative analyses are performed. A total of 156 students from Mechanical Engineering participated in the experiment. There are 15 participants selected for an interview after the experiment. Pre-tests and post-tests are performed using two different models, the developed VARTeL and the modified Technology-Rich Outcome-Focused Learning Environment Inventory (TROFLEI), in order to measure the efficiency of the VARTeL environment in Higher Education. Significant improvements of about 24.8% are observed for the post-tests over the pre-tests, which illustrate the effectiveness of the VARTeL for Engineering education. Details of the VR simulation games, methods of data collection, data analyses, as well as the experiment results are discussed. It is observed from the results that all the underlying scales of the modified TROFLEI are above the threshold for the ‘Good’ category, indicating that a very reliable questionnaire is designed in this research. The mean ‘Ideal’ values are about 0.7–2.6% higher than the mean ‘Actual’ values. The limitations of the experiment and future works with recommendations are also presented in this paper

Waterways to Greenways: A Case Study in Shangjie, Zhengzhou, Henan, China

This case study introduces how we used a water sensitive approach to plan a storm water and sponge city project, which expanded into a holistic green infrastructure project. The project is in the Shangjie district in city of Zhengzhou, Henan China. The whole site is 61.16 km² including several waterways. The city is expanding into areas that were previously agricultural. Developers and city both desire to improve the ecological value of the city to boost the economic growth of the Shangjie district. The main goal for the client is to transform the existing industrial city into a more resilient and livable ecological region. Our approach is to holistically solve the region\u27s increasing demand for flood control and storm water management, and to improve ecological and recreational values along these riparian corridors. We propose additional waterways and water bodies to act as green infrastructure, then link the greenways to existing or proposed parks to form a comprehensive greenway network. Our multidisciplinary team conducted detailed investigations and collaborated extensively. The team consists of hydraulic engineers, civil engineers, environmental planners, landscape planners, urban planners, economic planners, and others. We used a variety of technologies, including GIS, Infoworks, remote sensing technology, MIKE model, and lab tests

A convolutional neural network-based auto-segmentation pipeline for breast cancer imaging

Medical imaging is crucial for the detection and diagnosis of breast cancer. Artificial intelligence and computer vision have rapidly become popular in medical image analyses thanks to technological advancements. To improve the effectiveness and efficiency of medical diagnosis and treatment, significant efforts have been made in the literature on medical image processing, segmentation, volumetric analysis, and prediction. This paper is interested in the development of a prediction pipeline for breast cancer studies based on 3D computed tomography (CT) scans. Several algorithms were designed and integrated to classify the suitability of the CT slices. The selected slices from patients were then further processed in the pipeline. This was followed by data generalization and volume segmentation to reduce the computation complexity. The selected input data were fed into a 3D U-Net architecture in the pipeline for analysis and volumetric predictions of cancer tumors. Three types of U-Net models were designed and compared. The experimental results show that Model 1 of U-Net obtained the highest accuracy at 91.44% with the highest memory usage; Model 2 had the lowest memory usage with the lowest accuracy at 85.18%; and Model 3 achieved a balanced performance in accuracy and memory usage, which is a more suitable configuration for the developed pipeline