Overcoming the limitations of commodity augmented reality head mounted displays for use in product assembly

Numerous studies have shown the effectiveness of utilizing Augmented Reality (AR) to deliver work instructions for complex assemblies. Traditionally, this research has been performed using hand-held displays, such as smartphones and tablets, or custom-built Head Mounted Displays (HMDs). AR HMDs have been shown to be especially effective for assembly tasks as they allow the user to remain hands-free while receiving work instructions. Furthermore, in recent years a wave of commodity AR HMDs have come to market including the Microsoft HoloLens, Magic Leap One, Meta 2, and DAQRI Smart Glasses. These devices present a unique opportunity for delivering assembly instructions due to their relatively low cost and accessibility compared to custom built AR HMD solutions of the past. Despite these benefits, the technology behind these HMDs still contains many limitations including input, user interface, spatial registration, navigation and occlusion. To accurately deliver work instructions for complex assemblies, the hardware limitations of these commodity AR HMDs must be overcome. For this research, an AR assembly application was developed for the Microsoft HoloLens using methods specifically designed to address the aforementioned issues. Input and user interface methods were implemented and analyzed to maximize the usability of the application. An intuitive navigation system was developed to guide users through a large training environment, leading them to the current point of interest. The native tracking system of the HoloLens was augmented with image target tracking capabilities to stabilize virtual content, enhance accuracy, and account for spatial drift. This fusion of marker-based and marker-less tracking techniques provides a novel approach to display robust AR assembly instructions on a commodity AR HMD. Furthermore, utilizing this novel spatial registration approach, the position of real-world objects was accurately registered to properly occlude virtual work instructions. To render the desired effect, specialized computer graphics methods and custom shaders were developed and implemented for an AR assembly application. After developing novel methods to display work instructions on a commodity AR HMD, it was necessary to validate that these work instructions were being accurately delivered. Utilizing the sensors on the HoloLens, data was collected during the assembly process regarding head position, orientation, assembly step times, and an estimation of spatial drift. With the addition of wearable physiological sensor data, this data was fused together in a visualization application to validate instructions were properly delivered and provide an opportunity for an analysist to examine trends within an assembly session. Additionally, the spatial drift data was then analyzed to gain a better understanding of how spatial drift accumulates over time and ensure that the spatial registration mitigation techniques was effective. Academic research has shown that AR may substantial reduce cost for assembly operations through a reduction in errors, time, and cognitive workload. This research provides novel solutions to overcome the limitations of commodity AR HMDs and validate their use for product assembly. Furthermore, the research provided in this thesis demonstrates the potential of commodity AR HMDs and how their limitations can be mitigated for use in product assembly tasks

Concept for the comparison of intralogistics designs with real factory layout using augmented reality, SLAM and marker-based tracking

In the automotive industry, the intralogistics planning faces the problem of matching the planning data with the current conditions in the assembly hall. The large variety of parts leads to a constantly changing production. Based on this, we establish an approach for the comparison using augmented reality (AR) and simultaneous localization and mapping (SLAM). The use of SLAM enables the consistent application of AR in an assembly hall. Based on this, the objective of this article is to visualize 3D objects from the corresponding CAD planning tool in the real factory and thus the comparison of the intralogistics design with the real factory is possible due to AR. Nevertheless, there was a lack of practical implementations in intralogistics and therefore the concept is evaluated by two prototypical solutions. The first one is implemented on an iPhone 7 using SLAM. The second prototype is developed on a HoloLens 2 and is based on a hybrid tracking solution, SLAM and marker tracking

Remote maintenance assistance using real-time augmented reality authoring

Maintenance operations and lifecycle engineering have largely been considered one of the most expensive and time-consuming components for industrial equipment. Numerous organizations continually devote large quantities of resources towards maintaining equipment. As such, any optimizations that would reduce maintenance errors and expenses could lead to substantial time and cost savings. Unfortunately, there are often not enough specialists to meet the demand, forcing localized technicians to perform on-site maintenance on equipment outside their area of expertise. Augmented reality (AR) is one technology that has already been shown to improve the maintenance process. While powerful, AR has its own set of challenges, from content authoring to spatial perception. This work details a system that puts both the power of AR and the knowledge of a specialist directly into the hands of an on-site technician. An application was developed that enables a specialist to deliver AR instructions in real-time to assist a technician performing on-site maintenance. Using a novel and simplified authoring interface, specialists can create AR content in real-time, with little to no prior knowledge of augmented reality or the system itself. There has been ample research on different AR-supported processes, such as real-time authoring, video monitoring, and off-site assistance. However, much less work has been done to integrate them and leverage existing personnel knowledge to both author and deliver real-time AR instructions. This work details the development and implementation of such a system. A technical evaluation was also performed to ensure real-time connectivity in geographically distributed environments. Three network configurations were evaluated. A high-latency high-bandwidth network was used to represent a typical modern maintenance facility. A low-bandwidth network was evaluated to mimic older or more isolated maintenance environments. Lastly, a 4G LTE network was tested, showing the potential for the system to be used across global locations. Under all network configurations, the system effectively facilitated the complete disassembly of a hydraulic pump assembly

Videojuego multijugador basado en Realidad Aumentada

[EN] I will conduct a real-time multiplayer game, where the battle environment will be a real scene, using the camera of a mobile device and the Augmented Reality.[ES] Se realizará un videojuego multijugador de combate en tiempo real, en el que el entorno de la batalla será una escena real, haciendo uso de la cámara de un dispositivo móvil y la Realidad AumentadaLópez Punzano, J. (2014). Videojuego multijugador basado en Realidad Aumentada. http://hdl.handle.net/10251/53405Archivo delegad

Decoupled Mapping and Localization for Augmented Reality on a Mobile Phone

International audienceUsing Simultaneous Localization And Mapping (SLAM) methods become more and more common in Augmented Reality (AR). To achieve real-time requirement and to cope with scale factor and the lack of absolute positioning issue, we propose to decouple the localization and the mapping step. We explain the benefits of this approach and how a SLAM strategy can still be used in a way that is meaningful for the end user