Smart glasses for 3D multimodal composition

Extended reality technologies – mixed, augmented, and virtual reality, and future-related technologies – are rapidly expanding in many fields, with underexplored potentials for multimodal composition in digital media environments. This research generates new knowledge about the novel wearable technology – smart glasses – to support elementary students’ multimodal story authoring with 3D virtual objects or holograms. The researchers and teachers implemented learning experiences with upper elementary students from three classrooms to compose and illustrate written narratives before retelling the story with Microsoft HoloLens 2 smart glasses, selecting 3D holograms to illustrate the settings, characters, and events from the 3D Viewer software. The findings analyse how smart glasses supported students’ multimodal composition, and relatedly, the new modal resources available to students wearing smart glasses to compose 3D stories. The findings have significance for educators and researchers to understand and utilise the multimodal affordances of augmented and mixed reality environments for composing and storytelling

Interactive virtual method applied in urban design education. Mixed Approach

La memòria d'aquesta tesi tracta sobre l'ús de la transformació digital en l'ensenyament i els processos de disseny urbà, a través de conceptes innovadors i metodologies pràctiques. L'objectiu és promoure l'ús de tecnologies digitals, en particular per avaluar la inclusió de la realitat virtual en diversos entorns d'ensenyament formals i informals del disseny urbà col·laboratiu, per tal de millorar, accelerar i augmentar el seu impacte social de forma positiva i millorar l'ensenyament universitària ajudant al fet que els estudiants consolidin amb major efectivitat les seves habilitats. Amb les característiques d’un sistema de realitat virtual, podem provar la nostra hipòtesi en funció de demostrar: (1) La implementació d'estratègies virtuals gamificades en el camp del disseny urbà generarà una millora i motivació en la participació ciutadana i dels estudiants, ja que l’entorn és més dinàmic, real i permet un disseny col·laboratiu àgil gràcies a les tecnologies visuals immersives, mentre avaluen críticament el resultat d'un disseny urbà i prenen decisions. (2) L'ús de sistemes virtuals interactius per a la comprensió de l'espai tridimensional millora en els estudiants i professionals la percepció de la comprensió de l'espai, generant major enteniment de les condicions d'ubicació, dimensions i relacions dels espais urbans, i defensar els arguments de projectes urbans i propostes urbanes definides interactivament que proven diverses estratègies d'acció. Per provar les nostres hipòtesis, fem servir un mètode quantitatiu i qualitatiu aplicat en diferents escenaris i als diferents perfils d'usuaris, estudiants, professionals i usuaris finals. Els resultats mostraran que és possible potenciar la transformació digital, millorar la motivació pública, la implicació i la satisfacció en els processos de presa de decisions urbanes, així com complementar l'adquisició de competències urbanes específiques necessàries per a la professió en estudiants d'Arquitectura.La memoria de esta tesis trata sobre el uso de la transformación digital en la enseñanza y los procesos de diseño urbano, a través de conceptos innovadores y metodologías prácticas. El objetivo es promover el uso de tecnologías digitales, en particular para evaluar la inclusión de la realidad virtual en diversos entornos de enseñanza formales e informales del diseño urbano colaborativo, con el fin de mejorarlo, acelerar y aumentar su impacto social positivo y mejorar la enseñanza universitaria ayudando a que los estudiantes consoliden con mayor efectividad sus habilidades. Con las características de un sistema de realidad virtual, podemos probar nuestra hipótesis en función de demostrar: (1) El uso de sistemas virtuales interactivos para la comprensión del espacio tridimensional mejora en los estudiantes y profesionales la percepción de la comprensión del espacio, generando mayor entendimiento de las condiciones de ubicación, dimensiones y relaciones de los espacios urbanos, y defender los argumentos de proyectos urbanos y propuestas urbanas definidas interactivamente que ensayan diversas estrategias de acción. (2) La implementación de estrategias virtuales gamificadas en el campo del diseño urbano generará una mejora y motivación en la participación ciudadana y en los estudiantes, ya que es un entorno de colaboración más dinámico, real y ágil gracias a las tecnologías visuales inmersivas, mientras evalúan críticamente el resultado de un diseño urbano y toman decisiones. Para probar nuestras hipótesis, utilizamos un método cuantitativo y cualitativo aplicado en diferentes escenarios y a diferentes perfiles de usuarios, estudiantes, profesionales y usuarios finales. Los resultados mostrarán que es posible potenciar la transformación digital, mejorar la motivación pública, la implicación y la satisfacción en los procesos de toma de decisiones urbanas, así como complementar la adquisición de competencias urbanas específicas necesarias para la profesión en estudiantes de Arquitectura.The memory of this thesis deals with the use of digital transformation in the teaching and processes of urban design, through innovative concepts and practical methodologies. The objective is to promote the use of digital technologies, in particular, to evaluate the inclusion of virtual reality in various formal and informal teaching environments of collaborative urban design, in order to improve it, speed up, and increase its positive social impact and improve university teaching by helping students consolidate their skills more effectively. With the characteristics of the VR system, we can test our hypothesis based on demonstrating: (1) The use of virtual-interactive systems for the understanding of three- dimensional space improve in students and professionals the perception of the comprehension of the space, generating a greater understanding of the location conditions, dimensions and relationships of urban spaces, and defend the arguments of urban projects and interactively defined urban proposals rehearsing various strategies of action. (2) The implementation of virtual gamified strategies in the field of urban design will generate improvement and motivation in citizen participation and students as it is a more dynamic, real and agile collaborative environment thanks to the immersive visual technologies as they critically evaluate the result of the urban design and make decisions. To validate our hypothesis, we use a quantitative and qualitative method applied in different scenarios and to a different profile of users, students, professionals and end users. The results will show that it is possible to empower digital transformation, to improve public motivation, implication, and satisfaction in urban decision-making processes as well as complementing the improvement of the perception of the comprehension of the space needed for the profession in Architecture students

Application of augmented reality as a means of interdisciplinary learning

En el presente artículo se despliega una revisión de la cuestión relacionada con la aplicación de la realidad aumentada como medio de aprendizaje interdisciplinario. La investigación se desarrolló utilizando dos métodos: exploratorio y descriptivo, con un enfoque sistémico, utilizando las siguientes categorías como origen: realidad aumentada, realidad virtual, análisis de sistemas, web, realidad mixta y enseñanza. La búsqueda se realizó en las bases de datos Science Direct, Web Science, Google Académico y Scopus. Se catalogaron ciento dos artículos científicos para desarrollar la revisión. A partir de los artículos seleccionados se abordaron los siguientes interrogantes: ¿Cuáles son los vacíos conceptuales presentes en la realidad aumentada como medio de aprendizaje?; ¿Cómo se desarrolla la realidad aumentada como medio de aprendizaje?; ¿Cuál es la discusión actual sobre la realidad aumentada como medio de aprendizaje? y ¿Cuáles son los temas relevantes de la realidad aumentada como medio de aprendizaje?. Se determinaron las implementaciones de la realidad aumentada en los temas académicos y los factores influyentes directamente en su desarrollo, para tener un impacto positivo en el rendimiento académico. Como conclusión general se encontró: la realidad aumentada mejora el interés académico y facilita el desarrollo de los procesos de enseñanza y aprendizaj

Application of augmented reality as a means of interdisciplinary learning

En el presente artículo se despliega una revisión de la cuestión relacionada con la aplicación de la realidad aumentada como medio de aprendizaje interdisciplinario. La investigación se desarrolló utilizando dos métodos: exploratorio y descriptivo, con un enfoque sistémico, utilizando las siguientes categorías como origen: realidad aumentada, realidad virtual, análisis de sistemas, web, realidad mixta y enseñanza. La búsqueda se realizó en las bases de datos Science Direct, Web Science, Google Académico y Scopus. Se catalogaron ciento dos artículos científicos para desarrollar la revisión. A partir de los artículos seleccionados se abordaron los siguientes interrogantes: ¿Cuáles son los vacíos conceptuales presentes en la realidad aumentada como medio de aprendizaje?; ¿Cómo se desarrolla la realidad aumentada como medio de aprendizaje?; ¿Cuál es la discusión actual sobre la realidad aumentada como medio de aprendizaje? y ¿Cuáles son los temas relevantes de la realidad aumentada como medio de aprendizaje?. Se determinaron las implementaciones de la realidad aumentada en los temas académicos y los factores influyentes directamente en su desarrollo, para tener un impacto positivo en el rendimiento académico. Como conclusión general se encontró: la realidad aumentada mejora el interés académico y facilita el desarrollo de los procesos de enseñanza y aprendizaj

Urban Digital Twins for Smart Cities and Citizens:The Case Study of Herrenberg, Germany

Cities are complex systems connected to economic, ecological, and demographic conditions and change. They are also characterized by diverging perceptions and interests of citizens and stakeholders. Thus, in the arena of urban planning, we are in need of approaches that are able to cope not only with urban complexity but also allow for participatory and collaborative processes to empower citizens. This to create democratic cities. Connected to the field of smart cities and citizens, we present in this paper, the prototype of an urban digital twin for the 30,000-people town of Herrenberg in Germany. Urban digital twins are sophisticated data models allowing for collaborative processes. The herein presented prototype comprises (1) a 3D model of the built environment, (2) a street network model using the theory and method of space syntax, (3) an urban mobility simulation, (4) a wind flow simulation, and (5) a number of empirical quantitative and qualitative data using volunteered geographic information (VGI). In addition, the urban digital twin was implemented in a visualization platform for virtual reality and was presented to the general public during diverse public participatory processes, as well as in the framework of the "Morgenstadt Werkstatt" (Tomorrow's Cities Workshop). The results of a survey indicated that this method and technology could significantly aid in participatory and collaborative processes. Further understanding of how urban digital twins support urban planners, urban designers, and the general public as a collaboration and communication tool and for decision support allows us to be more intentional when creating smart cities and sustainable cities with the help of digital twins. We conclude the paper with a discussion of the presented results and further research directions

An Interactive Augmented Reality Alphabet 3-Dimensional Pop-up Book For learning and Recognizing the English Alphabet

This document describes the process developing an Augmented Reality (AR) alphabet book mobile application. Using only an android phone camera, the child could view the superimposed virtual alphabet 3 dimensional objects in a fun and interactive manner using the marker-less physical alphabet book as the interaction tool. The reason behind choosing alphabet teaching as the topic of the book is that the Alphabet knowledge is the core knowledge of any language. It is a jump-start for children to start reading and recognizing words and sentences, thus learning the alphabet is extremely important, for many researchers, emphasizing on how early, child’s education shapes the child’s successful future. Though there are, a great deal of technology based alphabet books; parents still prefer buying the old style physical books or some might use a virtual technology based book application. The problem is that though the physical book possesses many benefits, that our generation and the generations long before us, have experienced, yet from the current generation children’s point of view, they may in fact find it dull and boring. For, it is commonly recognized, that the current generation children are surrounded all around by technology and gadgets, that can make them board, easily distracted, and may refuse to willingly use a plain non-technology book to learn, and if using a virtual application, they will lose the benefits offered by a physical book. Knowing this, the use of Augmented Reality should solve such a problem. For Augmented Reality (AR) is considered the best of both worlds, where, real and virtual objects are combined in the real environment, that will allow the use of both technology based application and a traditional physical book, combining the benefits of both and meeting the child and the parent midway. Although AR technology is not new, its possible potential in education is just beginning to be investigated. The main aim of this research is to develop an interactive 3-Dimentional alphabet pop-up book, and using digital storytelling, to help teach children to learn and recognize the alphabets. The objectives of the study are to enhance the interactions of the alphabet book, by creating an android application that contains animated interactive 3-Dimentional models, interactive sounds, songs and music. Furthermore, to investigate the use of digital storytelling (music, sounds), interactions and animation effect in learning engagement, through using the augmented reality technology. The scope of this project and research is very wide, it includes the 3D modeling, texturing, rigging & animation, book design and content decision research, furthermore, Augmented Reality and Android applicatio

Updating the art history curriculum: incorporating virtual and augmented reality technologies to improve interactivity and engagement

Master's Project (M.Ed.) University of Alaska Fairbanks, 2017This project investigates how the art history curricula in higher education can borrow from and incorporate emerging technologies currently being used in art museums. Many art museums are using augmented reality and virtual reality technologies to transform their visitors' experiences into experiences that are interactive and engaging. Art museums have historically offered static visitor experiences, which have been mirrored in the study of art. This project explores the current state of the art history classroom in higher education, which is historically a teacher-centered learning environment and the learning effects of that environment. The project then looks at how art museums are creating visitor-centered learning environments; specifically looking at how they are using reality technologies (virtual and augmented) to transition into digitally interactive learning environments that support various learning theories. Lastly, the project examines the learning benefits of such tools to see what could (and should) be implemented into the art history curricula at the higher education level and provides a sample section of a curriculum demonstrating what that implementation could look like. Art and art history are a crucial part of our culture and being able to successfully engage with it and learn from it enables the spread of our culture through digital means and of digital culture

Enhancement of Human Sensory and Motor Skills using Collaboration of Eye - Ear Focus and Body - Arm - Hand Gesture in No-Touch 3D Interactive Aircraft Game

Nowadays, human sensory and motor skills enhancement is very important to be alert from the effect of the Society 5.0 era. We propose the novel approach to enhance human sensory and motor skills using collaboration of senses focus and body parts gesture in no-touch3D interactive aircraft game. A player interacts with a virtual display using gestures of his body, arm, hand, or collaboration of them as a controller to engage in a 3D game. This game is based on an aircraft education theme. By sweeping the hand to the left or right, a player can select one of many different aircraft models according to its specification and performance are shown on the screen based on the estimation of his current skill. The mission is to control aircraft avoiding the random balls attack to prevent a collision. Hand and arm are positioned above a leap motion sensor and in front of a camera. The body, arm, and hand gesture directions are to forward or backward from the screen to the right or left, and also to up or down freely. This system is developed by using Unity 3D, OpenCV, and Python. Many skills are possible to be improved, e.g., motivation, discipline, safety awareness, endurance management, strategy, precision, intelligence, creativity, reliability, toughness, and vision. This project has been tested in the Air Force Museum of Dirgantara Mandala involving more than 1000 visitors as testing participants, and they all feel satisfied. The score is calculated by time in second. The results show endurance time is 32% (<60 s), 31% (<120 s), 25% (<160 s), and 12 % (180 s) of the total300 seconds of the gam

Diverse approaches to learning with immersive Virtual Reality identified from a systematic review

To investigate how learning in immersive Virtual Reality was designed in contemporary educational studies, this systematic literature review identified nine design features and analysed 219 empirical studies on the designs of learning activities with immersive Virtual Reality. Overall, the technological features for physical presence were more readily implemented and investigated than pedagogical features for learning engagement. Further analysis with k-means clustering revealed five approaches with varying levels of interactivity and openness in learning tasks, from watching virtual worlds passively to responding to personalised prompts. Such differences in the design appeared to stem from different practical and educational priorities, such as accessibility, interactivity, and engagement. This review highlights the diversity in the learning task designs in immersive Virtual Reality and illustrates how researchers are navigating practical and educational concerns. We recommend future empirical studies recognise the different approaches and priorities when designing and evaluating learning with immersive Virtual Reality. We also recommend that future systematic reviews investigate immersive Virtual Reality-based learning not only by learning topics or learner demographics, but also by task designs and learning experiences