Adaptive User Perspective Rendering for Handheld Augmented Reality

Handheld Augmented Reality commonly implements some variant of magic lens rendering, which turns only a fraction of the user's real environment into AR while the rest of the environment remains unaffected. Since handheld AR devices are commonly equipped with video see-through capabilities, AR magic lens applications often suffer from spatial distortions, because the AR environment is presented from the perspective of the camera of the mobile device. Recent approaches counteract this distortion based on estimations of the user's head position, rendering the scene from the user's perspective. To this end, approaches usually apply face-tracking algorithms on the front camera of the mobile device. However, this demands high computational resources and therefore commonly affects the performance of the application beyond the already high computational load of AR applications. In this paper, we present a method to reduce the computational demands for user perspective rendering by applying lightweight optical flow tracking and an estimation of the user's motion before head tracking is started. We demonstrate the suitability of our approach for computationally limited mobile devices and we compare it to device perspective rendering, to head tracked user perspective rendering, as well as to fixed point of view user perspective rendering

Model-based training of manual procedures in automated production systems

Maintenance engineers deal with increasingly complex automated production systems (aPSs). Such systems are characterized by an increasing computerization or the addition of robots that collaborate with human workers. The effects of changing or replacing components of such systems are difficult to assess since there are complex interdependencies between process parameters and the state of the components. This paper proposes a model-based training system that visualizes these interdependencies using domain-independent SysML models. The training system consists of a virtual training system for initial training and an online support system for assistance during maintenance or changeover procedures. Both systems use structural SysML models to visualize the state of the machine at a certain step of a procedure. An evaluation of the system in a changeover procedure against a paper-based manual showed promising results regarding effectiveness, usability and attractiveness.Comment: 25 pages, https://www.sciencedirect.com/science/article/pii/S095741581830080

A systematic review of augmented reality applications in maintenance

Augmented Reality (AR) technologies for supporting maintenance operations have been an academic research topic for around 50 years now. In the last decade, major progresses have been made and the AR technology is getting closer to being implemented in industry. In this paper, the advantages and disadvantages of AR have been explored and quantified in terms of Key Performance Indicators (KPI) for industrial maintenance. Unfortunately, some technical issues still prevent AR from being suitable for industrial applications. This paper aims to show, through the results of a systematic literature review, the current state of the art of AR in maintenance and the most relevant technical limitations. The analysis included filtering from a large number of publications to 30 primary studies published between 1997 and 2017. The results indicate a high fragmentation among hardware, software and AR solutions which lead to a high complexity for selecting and developing AR systems. The results of the study show the areas where AR technology still lacks maturity. Future research directions are also proposed encompassing hardware, tracking and user-AR interaction in industrial maintenance is proposed

AUGMENTED REALITY AND MOBILE SYSTEMS FOR HEAVY EQUIPMENT OPERATORS IN SURFACE MINING

U.S. federal laws mandate that mining companies ensure a safe workplace, implement approved training programs, and promptly report work-related injuries. The mining industry\u27s commitment to innovation reflects a history of adopting advancements to enhance environmental sustainability, workplace safety, and overall productivity, while simultaneously reducing operational costs. This thesis proposes the integration of Augmented Reality (AR) technology and digital applications to enhance the surface mining industry, presenting two innovative solutions: an AR Training System and an Operational Digital System. These business solutions have been developed and applied at a surface mine in the southwest of the US, having the potential to improve the mining industry by enhancing safety, training, operational efficiency, and data-driven decision-making, which comprehends a significant step toward a more sustainable, effective, and technologically driven mining sector, contributing to the industry\u27s evolution and growth. The AR Training System leverages Microsoft´s Power Platform and HoloLens 2 capacities to provide operators with immersive and step-by-step training guides in real working conditions for Dozers, Motor Graders, and End Dump trucks. These AR guides combine 3D models, videos, photos, and interactive elements overlapping mining equipment to enhance learning and safety. The system also offers an efficient approach to data collection during operator training, which has the potential to modify the training guides based on user performance. On the other hand, the Operational Digital System addresses the industry\u27s operational challenges. It streamlines the pre-operation inspection process, tracks equipment status, and accelerates defect identification, shift timing, delays, and loaded tonnage. The system offers a holistic approach to mining operation optimization, facilitating data sharing and management among different departments, enhancing collaboration, and expediting maintenance processes

An evaluation of the Microsoft HoloLens for a manufacturing-guided assembly task

Many studies have confirmed the benefits of using Augmented Reality (AR) work instructions over traditional digital or paper instructions, but few have compared the effects of different AR hardware for complex assembly tasks. For this research, previously published data using Desktop Model Based Instructions (MBI), Tablet MBI, and Tablet AR instructions were compared to new assembly data collected using AR instructions on the Microsoft HoloLens Head Mounted Display (HMD). Participants completed a mock wing assembly task, and measures like completion time, error count, Net Promoter Score, and qualitative feedback were recorded. The HoloLens condition yielded faster completion times than all other conditions. HoloLens users also had lower error rates than those who used the non-AR conditions. Despite the performance benefits of the HoloLens AR instructions, users of this condition reported lower net promoter scores than users of the Tablet AR instructions. The qualitative data showed that some users thought the HoloLens device was uncomfortable and that the tracking was not always exact. Although the user feedback favored the Tablet AR condition, the HoloLens condition resulted in significantly faster assembly times. As a result, it is recommended to use the HoloLens for complex guided assembly instructions with minor changes, such as allowing the user to toggle the AR instructions on and off at will. The results of this paper can help manufacturing stakeholders better understand the benefits of different AR technology for manual assembly tasks