Mixed Reality Applications for Safety Trainings in Wind Energy Sector: A Case Study

The international renewable energy agency (IRENA) forecast that the wind industry will grow at an exponential rate in the coming decades. This enormous growth has created the need and demand for qualified workforce which includes engineers, technicians, and managers in the wind energy sector. Thus, the wind energy training sector needs to implement some innovative technologies in both safety and technical trainings to meet the growing industry demands and to create a qualified workforce. However, before finalizing on any innovative solution for safety training, the challenges that the wind energy training sector faces need to be analyzed. The biggest challenges for the wind energy training sector is to train the workers to work safely with large scale wind turbine structure and its components, working safely with high voltage and working in harsh marine environments. Wind power workers are often exposed to hazards that can result in fatalities or serious injuries due to these challenges. Therefore, the implemented innovative technology must ensure safety and improve efficiency of operations by being aware of the risks associated. However, the wind energy training sector is searching for cost effective solution especially related to remote training, when the technician is not able to attend the training physically. Moreover, the wind energy training sector is also looking for technology that can reduce human error and also reduce cognitive workload. Therefore, the use of innovative technology like mixed reality (MR) might provide potential benefits.MR includes the use of both the virtual reality (VR) which is a simulated immersive experience and the use of augmented reality (AR) which allows the person to see the real world, additionally overlaid with digital graphics and information in real time. However, there is a lack of clarity on how to effectively design mixed reality technologies in safety training of wind sector. There are technical challenges and gaps to identify the suitable hardware platform, suitable software platform and the associated tracking techniques. The purpose of this thesis is to develop: (1) the workflow, (2) the framework which will help to design mixed reality technologies in safety training of wind sector. Also to develop, (3) flowchart and (4) worksheet which will help to identify the critical training modules/scenarios and to identify the suitable type of technology (AR/VR/MR) needed for a particular scenario along with the suitable hardware platform, suitable software platform and associated tracking technique. Finally, to develop (5) demo MR model to demonstrate and validate the developed workflow and to understand the associated practical challenges like complexity of such mixed reality technologies and user familiarity. In order to achieve the purpose of this thesis, a six-step methodology was applied which includes: (1) system analysis, (2) use case analysis, (3) conceptualize, (4) computerize, (5) construct and (6) verify, validate and visualize. The case study started with system analysis which mainly deals with extracting the industrial needs and requirements. The system analysis includes two sub steps. First, is to perform a detailed systematic literature review (SLR) to understand the state of art in VR/AR/MR in the wind industry and other relevant industry. Second, an empirical exploration were the author attended a 5day GWO wind safety training at the Eigersund energy hub to personally experience and understand the training scenarios which will be crucial and beneficial to have a mixed reality application from both technician and company perspective. The use case analysis deals with the creation of the case context which includes selecting the critical training module based on accident data from literature review and to identify one similar framework in other engineering industry. The conceptualize step involves in classifying the selected critical module into training tasks and to identity the risk associated with each training tasks. It also involves in performing the concept study before building the MR model and to develop the scenario modelling chart. The computerize step involves in developing the actual 3D model and the demo MR model. The construct step involves the creation of the flowchart, worksheet, workflow and the framework and the sixth step is to validate and verify the research outputs. Consequently, as the result of the six-step methodology this thesis has provided new knowledge regarding four concepts: (1) sequence or workflow (2) the need for continuous and iterative process to design mixed reality (3) the logic and the rules for the selection of technical specifications (4) worksheet to classify the scenarios and to define training complexity. The thesis concludes that the safety training provider needs to rigorously follow the developed (1) mixed reality analysis (MRA) workflow, (2) mixed reality analysis (MRA) framework, (3) mixed reality technical specification (MRTS) flowchart and the (4) mixed reality technical specification (MRTS) worksheet. The MRA workflow provides the sequence that can enable the industrial practitioner to design MR application in a cost-effective and fit for purpose manner which includes screening out low risk scenarios. The MRA framework clearly indicates that the design to implement MR is an iterative process based on user needs and user level of familiarity. The MRTS flowchart provides the logic to identify the suitable type of technology for a particular scenario, along with the suitable hardware platform, suitable software platform and associated tracking technique. The MRTS flowchart must be used along with the MRTS worksheet which can help to identify the critical training modules/scenarios and further classify them to check if implementation of mixed realities is needed or not. In future, the complexity associated with such technologies must be studied after implementing in real-time. Based on the case study AR/VR should be designed to make work simpler and error free. However, in some cases if using the technology is creating unintended consequences and complexity, then the technology might not be used in such cases

Feasibility Study: Development and Demonstration of Virtual Reality Simulation Training for the BHPB Olympic Dam Site Inductions

This report presents the findings of the project ―Feasibility Study: Development and Demonstration of Virtual Reality Simulation Training for the BHPB Olympic Dam Site Inductions.‖ The project was a collaborative exercise between the University of New South Wales (UNSW) - School of Mining Engineering, the University of Adelaide - Australian Centre for Visual Technologies, BHPB Olympic Dam Expansion, RESA, TAFESA and Skills DMC. The project Chief Investigators were Dr Phillip Stothard (UNSW) and Prof Anton van den Hengel (University of Adelaide).The project was a pilot study research project that looked into the feasibility of developing interactive virtual reality simulations for mine site inductions in the hard rock industry. Many simulations have been successfully implemented into the coal industry and the aim was to build a pilot module that looked at a high risk environment on a surface mine that would also have application to the wider construction industry and other heavy industries. The project collaborators came together as a group of parties interested in virtual reality simulation. The research and development was led by UNSW and University of Adelaide. Invaluable input was provided by the collaborators. The project had a value of $431, 306. Of which$208,563 was in cash and $222,743 was in kind. The budget was fully expended during the course of the project. The subject area of the project was ̳Working at Heights‘ and this was chosen because it is a high risk area. Substantial documentation, mining industry input and effort was placed on building the five sub-modules that form the Working at Heights module. The outcome is a high quality visualisation of an area of the Olympic Dam Mine Site. This high quality visualisation is enhanced by the inclusion of interaction within the module that requires the user to interrogate data within the site and to assess and understand issues that arise when working at heights in relation ladders, scaffolding, open excavations and elevated work platforms. Much project emphasis and time was placed on producing the 3D model. Also, as much information as possible was placed into the module itself as this was to be a pilot example to show to the Olympic Dam Expansion Project Team. The module allows users to interact with Safety Documentation and equipment and procedures that they would encounter on sit

VR-Based Safety Training Program for High-Rise Building Construction

The rates of fatal and non-fatal accidents within the construction industry across the globe are surging despite the massive efforts that are being exerted toward maintaining a safe working environment. Past research has proved that the provision of effective safety training programs is a primary course of action that should be taken to minimize construction accidents, fatalities, and both fatal and nonfatal injuries. However, in acknowledging the limitations of traditional safety training programs within the construction industry, several researchers have addressed the urge to incorporate novel training practices that are based on the modern virtual reality (VR) technology to promote “learning by doing” and “experiential learning” in their educational approaches. Nevertheless, there is a lack of incorporating major learning theories as a solid foundation for the design and development of VR-based training programs. Also, there is a lack of comprehensive VR-based safety training programs that specifically address the safety of high-rise building construction. This research aims to develop a comprehensive, fully immersive, and interactive VR-based safety training program that addresses the hazards and risks pertaining to the construction of high-rise buildings based on major learning theories in an attempt to enhance the learning outcomes of construction workers and safety officers. To conclude, the framework developed proved its efficiency and effectiveness in achieving the desired learning outcomes using VR-based training programs

Construction safety and digital design: a review

As digital technologies become widely used in designing buildings and infrastructure, questions arise about their impacts on construction safety. This review explores relationships between construction safety and digital design practices with the aim of fostering and directing further research. It surveys state-of-the-art research on databases, virtual reality, geographic information systems, 4D CAD, building information modeling and sensing technologies, finding various digital tools for addressing safety issues in the construction phase, but few tools to support design for construction safety. It also considers a literature on safety critical, digital and design practices that raises a general concern about ‘mindlessness’ in the use of technologies, and has implications for the emerging research agenda around construction safety and digital design. Bringing these strands of literature together suggests new kinds of interventions, such as the development of tools and processes for using digital models to promote mindfulness through multi-party collaboration on safet

The role of virtual reality in built environment education

This study builds upon previous research on the integration of Virtual Reality (VR) within the built environment curriculum and aims to investigate the role of VR and three-dimensional (3D) computer modelling on learning and teaching in a school of the built environment. In order to achieve this aim, a number of academic experiences were analysed to explore the applicability and viability of 3D computer modelling and VR into built environment subject areas. Although two-dimensional (2D) representations have been greatly accepted by built environment professions and education, 3D computer representations and VR applications, offering interactivity and immersiveness, are not yet widely accepted. The study attempts to understand the values and challenges of integrating visualisation technologies into built environment teaching and investigates tutors’ perceptions, opinions and concerns with respect to these technologies. The study reports on the integration process and considers how 3D computer modelling and VR technologies can combine with, and extend, the existing range of learning and teaching methods appropriate to different disciplines and programme areas

Proof-of-Concept: Safety Hazard Identification and Impact Minimization Using 3D BIM and VR Devices Through the Case-Studies

Construction hazard is a global issue. Despite numerous research studies, safety guidelines and procedures, fatalities and severe injuries still occur on construction job sites. This research has been performed to identify the research gaps and potentially improve worker behavior along the most hazardous tasks during construction execution using 3D Building Information Modeling (BIM) and Virtual Reality (VR) devices. A safety hazards-related questionnaire for civil engineering and construction students, superintendents, safety, and project managers across six different states including the state of Georgia was deployed. The questionnaire was distributed via an online platform to identify and approach the hazards which occur during the pre-construction design and are latent until the execution of a project. Through a case-study, qualitative, and quantitative-based analysis, the study aims to investigate many hazards that remain unidentified using 3D BIM models and integrating them through VR devices. The research focuses mainly on electrical, mechanical equipment, roofing, and concrete works during the project execution. The chi-square test was used to examine the variability of the independent factors’ hazard recognition performance when they were crossed with the dependent variables (i.e., safety training, technology usage/advanced device training) to test the hypotheses. The study\u27s findings and recommendations can be utilized by construction organizations to evaluate BIM and VR adoption and decide whether and how they should be used for hazard detection and impact mitigation. In order to emphasize on accident causation and the significance of thorough hazard recognition and appropriate risk perception, researchers created a virtual walk-through replicating acceptable actions in close proximity to specific activity risks into a VR environment. Suggestions are also made to improve course design for any construction safety training by looking at the impact of BIM in conjunction with VR on construction safety and hazard mitigation

Design for safety: theoretical framework of the safety aspect of BIM system to determine the safety index

Despite the safety improvement drive that has been implemented in the construction industry in Singapore for many years, the industry continues to report the highest number of workplace fatalities, compared to other industries. The purpose of this paper is to discuss the theoretical framework of the safety aspect of a proposed BIM System to determine a Safety Index. An online questionnaire survey was conducted to ascertain the current workplace safety and health situation in the construction industry and explore how BIM can be used to improve safety performance in the industry. A safety hazard library was developed based on the main contributors to fatal accidents in the construction industry, determined from the formal records and existing literature, and a series of discussions with representatives from the Workplace Safety and Health Institute (WSH Institute) in Singapore. The results from the survey suggested that the majority of the firms have implemented the necessary policies, programmes and procedures on Workplace Safety and Health (WSH) practices. However, BIM is still not widely applied or explored beyond the mandatory requirement that building plans should be submitted to the authorities for approval in BIM format. This paper presents a discussion of the safety aspect of the Intelligent Productivity and Safety System (IPASS) developed in the study. IPASS is an intelligent system incorporating the buildable design concept, theory on the detection, prevention and control of hazards, and the Construction Safety Audit Scoring System (ConSASS). The system is based on the premise that safety should be considered at the design stage, and BIM can be an effective tool to facilitate the efforts to enhance safety performance. IPASS allows users to analyse and monitor key aspects of the safety performance of the project before the project starts and as the project progresses

A Framework for Creating Virtual Reality Models for More Effective Coastal Flood Risk Communication

Coastal cities are exposed to increasing risks of flooding from sea-level rise. Climate change is expected to double the frequency of coastal flooding within the next decade, and some areas could experience floods of a magnitude 100 times higher than currently (Vitousek et al., 2017). People living in at-risk areas often ignore the impact of climate change on flood intensity and frequency. Immersive visual storytelling techniques proved promising and powerful tools to engage with and raise awareness of flood hazards. Here, we are introducing a framework to use Virtual Reality (VR) to reach better people living in coastal cities and help them understand the impact of climate change on their community. We developed a virtual experience in which people can be immersed in a coastal flood and experience its intensity. We used a combination of UAV imagery and digital photogrammetric techniques to create a virtual environment in which people can recognize real locations in their neighborhood and used GIS flood data to apply a water texture in Unity3D to create the flood levels

Real-time simulation of construction workers using combined human body and hand tracking for robotic construction worker system

Construction is an inherently less safe sector than other sectors because it exposes workers to harsh and dangerous working environments. The nature of the construction industry results in a comparatively high incidence of serious injuries and death caused by falls from a height, musculoskeletal disorders and being struck by objects. This paper presents a new concept that can tackle this problem in the future. The central hypothesis of this study is that it is possible to eliminate injuries if we move the human construction worker off-site and remotely link his/her motions to a Robotic Construction Worker (RCW) on-site. As a first steppingstone towards this ultimate goal, two systems essential for the RCW were developed in this study. First, a novel system that combines 3D body and hand position tracking was developed to capture the movements of human construction worker. This combination of tracking enables the capture of changes in the orientations and articulations of the entire human body. Second, a real-time simulation system that connects a human construction worker off-site to a virtual RCW was developed to demonstrate the proposed concept in a variety of construction scenarios. The simulation results demonstrate the future viability of the RCW concept and indicate the promise of this system for eliminating the health and safety risks faced by human construction workers