AUGMENTED REALITY AND MOBILE SYSTEMS FOR HEAVY EQUIPMENT OPERATORS IN SURFACE MINING

U.S. federal laws mandate that mining companies ensure a safe workplace, implement approved training programs, and promptly report work-related injuries. The mining industry\u27s commitment to innovation reflects a history of adopting advancements to enhance environmental sustainability, workplace safety, and overall productivity, while simultaneously reducing operational costs. This thesis proposes the integration of Augmented Reality (AR) technology and digital applications to enhance the surface mining industry, presenting two innovative solutions: an AR Training System and an Operational Digital System. These business solutions have been developed and applied at a surface mine in the southwest of the US, having the potential to improve the mining industry by enhancing safety, training, operational efficiency, and data-driven decision-making, which comprehends a significant step toward a more sustainable, effective, and technologically driven mining sector, contributing to the industry\u27s evolution and growth. The AR Training System leverages Microsoft´s Power Platform and HoloLens 2 capacities to provide operators with immersive and step-by-step training guides in real working conditions for Dozers, Motor Graders, and End Dump trucks. These AR guides combine 3D models, videos, photos, and interactive elements overlapping mining equipment to enhance learning and safety. The system also offers an efficient approach to data collection during operator training, which has the potential to modify the training guides based on user performance. On the other hand, the Operational Digital System addresses the industry\u27s operational challenges. It streamlines the pre-operation inspection process, tracks equipment status, and accelerates defect identification, shift timing, delays, and loaded tonnage. The system offers a holistic approach to mining operation optimization, facilitating data sharing and management among different departments, enhancing collaboration, and expediting maintenance processes

Communication in Immersive Social Virtual Reality: A Systematic Review of 10 Years' Studies

As virtual reality (VR) technologies have improved in the past decade, more research has investigated how they could support more effective communication in various contexts to improve collaboration and social connectedness. However, there was no literature to summarize the uniqueness VR provided and put forward guidance for designing social VR applications for better communication. To understand how VR has been designed and used to facilitate communication in different contexts, we conducted a systematic review of the studies investigating communication in social VR in the past ten years by following the PRISMA guidelines. We highlight current practices and challenges and identify research opportunities to improve the design of social VR to better support communication and make social VR more accessible.Comment: Chinese CHI '22: The Tenth International Symposium of Chinese CHI (Chinese CHI 2022

Virtual Meeting Rooms: From Observation to Simulation

Virtual meeting rooms are used for simulation of real meeting behavior and can show how people behave, how they gesture, move their heads, bodies, their gaze behavior during conversations. They are used for visualising models of meeting behavior, and they can be used for the evaluation of these models. They are also used to show the effects of controlling certain parameters on the behavior and in experiments to see what the effect is on communication when various channels of information - speech, gaze, gesture, posture - are switched off or manipulated in other ways. The paper presents the various stages in the development of a virtual meeting room as well and illustrates its uses by presenting some results of experiments to see whether human judges can induce conversational roles in a virtual meeting situation when they only see the head movements of participants in the meeting

Gaze Based Human Intention Analysis

The ability to determine an upcoming action or what decision a human is about to take, can be useful in multiple areas, for example during human-robot collaboration in manufacturing, where knowing the intent of the operator could provide the robot with important information to help it navigate more safely. Another field that could benefit from a system that provides information regarding human intentions is the field of psychological testing where such a system could be used as a platform for new research or be one way to provide information in the diagnostic process. The work presented in this thesis investigates the potential use of virtual reality as a safe, measurable, and customizable environment to collect gaze and movement data, eye tracking as the non-invasive system input that gives insight into the human mind, and deep machine learning as one tool to analyze the data. The thesis defines an experimental procedure that can be used to construct a virtual reality based testing system that gathers gaze and movement data, carry out a test study to gather data from human participants, and implement artificial neural networks in order to analyze human behaviour. This is followed by two studies that gives evidence to the decisions that were made in the experimental procedure and shows the potential uses of such a system