An Interactive Mobile Equipment Task-Training with Virtual Reality

Improving the quality of equipment training for the Heavy Equipment Operators (HEO) is a critical task in the ever-lasting effort to improve safety and eliminate equipment-related injuries in mining environments. Traditional miner training includes the use of hardcopy documents and video instructions. However, modern mobile and computer technology offers tremendous potential to improve the training process. One major responsibility for the heavy-machine operators is proper machine inspection. Establishing new training methods that utilize modern technologies—such as 360-degree images, videos, and Virtual Reality—and implementing computers for training purposes have a potential to help new operators learn how to conduct proper machine inspections in a more efficient and technically correct way. This technology could potentially provide a higher knowledge retention rate for heavy machine operators. This study utilizes a 360-degree camera, open-source platform WordPress™, and the software Unity3D in order to create materials and tools for the HEOs training, which in turn will help trainees to better understand the pre-shift machine inspection. The outcomes of this research are organized into three major phases: Gathering materials, Computer-Based Task-Training (CBTT) software, and Virtual Reality (VR) application. The 360-degree images/videos, 2D images, and sounds were first gathered, edited, and incorporated into the CBTT and VR applications. The major feature of the CBTT, developed with WordPress™, is its training template with instructions on creating a new training course for the HEOs. The training courses developed for this study cover seven different machines that are widely used in surface mines. Also, the CBTT software is optimized for use on mobile devices (such as smart-phones and tablets). The VR applications for the same machines are developed with one of the most popular game engines - Unity3D. To interact with the virtual world, a trainee can use the Head-Mounted Display (HMD) iii Oculus Rift CV 1, which establishes full immersion while performing the virtual tour of a selected machine. If used without HMD, applications become non-immersive desktop versions and can be controlled by using a mouse and keyboard. The user-friendly interface and multimedia environment are comfortable to handle, flexible, and potentially more convenient to use than paper-based documents. The CBTT and VR applications were developed, tested, and implemented at a surface coal mine in the southern United States

Videos in Context for Telecommunication and Spatial Browsing

The research presented in this thesis explores the use of videos embedded in panoramic imagery to transmit spatial and temporal information describing remote environments and their dynamics. Virtual environments (VEs) through which users can explore remote locations are rapidly emerging as a popular medium of presence and remote collaboration. However, capturing visual representation of locations to be used in VEs is usually a tedious process that requires either manual modelling of environments or the employment of specific hardware. Capturing environment dynamics is not straightforward either, and it is usually performed through specific tracking hardware. Similarly, browsing large unstructured video-collections with available tools is difficult, as the abundance of spatial and temporal information makes them hard to comprehend. At the same time, on a spectrum between 3D VEs and 2D images, panoramas lie in between, as they offer the same 2D images accessibility while preserving 3D virtual environments surrounding representation. For this reason, panoramas are an attractive basis for videoconferencing and browsing tools as they can relate several videos temporally and spatially. This research explores methods to acquire, fuse, render and stream data coming from heterogeneous cameras, with the help of panoramic imagery. Three distinct but interrelated questions are addressed. First, the thesis considers how spatially localised video can be used to increase the spatial information transmitted during video mediated communication, and if this improves quality of communication. Second, the research asks whether videos in panoramic context can be used to convey spatial and temporal information of a remote place and the dynamics within, and if this improves users' performance in tasks that require spatio-temporal thinking. Finally, the thesis considers whether there is an impact of display type on reasoning about events within videos in panoramic context. These research questions were investigated over three experiments, covering scenarios common to computer-supported cooperative work and video browsing. To support the investigation, two distinct video+context systems were developed. The first telecommunication experiment compared our videos in context interface with fully-panoramic video and conventional webcam video conferencing in an object placement scenario. The second experiment investigated the impact of videos in panoramic context on quality of spatio-temporal thinking during localization tasks. To support the experiment, a novel interface to video-collection in panoramic context was developed and compared with common video-browsing tools. The final experimental study investigated the impact of display type on reasoning about events. The study explored three adaptations of our video-collection interface to three display types. The overall conclusion is that videos in panoramic context offer a valid solution to spatio-temporal exploration of remote locations. Our approach presents a richer visual representation in terms of space and time than standard tools, showing that providing panoramic contexts to video collections makes spatio-temporal tasks easier. To this end, videos in context are suitable alternative to more difficult, and often expensive solutions. These findings are beneficial to many applications, including teleconferencing, virtual tourism and remote assistance

Photo-based automatic 3D reconstruction of train accident scenes

Railway accidents place significant demands on the resources of, and support from, railway emergency management departments. Once an accident occurs, an efficient incident rescue plan needs to be delivered as early as possible to minimise the loss of life and property. However, in the railway sector, most relevant departments currently face a challenge in drawing up a rescue scheme effectively and accurately with the insufficient information collected from the scene of a train accident. To assist with the rescue planning, we propose a framework which can rapidly and automatically construct a 3D virtual scene of a train accident by utilising photos of the accident spot. The framework uses a hybrid 3D reconstruction method to extract the position and pose information of the carriages involved in an accident. It adopts a geographic information system and a 3D visualisation engine to model and display the landscapes and buildings at the site of a train accident. In order to assess and validate our prototype, we quantitatively evaluate our main algorithm and demonstrate the usage of our technology with two case studies including a simulated scene with an in-lab setting and a real train derailment scene from on-site pictures. The results of both are accoun table with high accuracy and represent the ability of timely modelling and visualisation of a train accident scene

The use of low cost virtual reality and digital technology to aid forensic scene interpretation and recording

© Cranfield University 2005. All rights reserved. No part of this publication may be reproduced without the written permission of the copyright owner.Crime scenes are often short lived and the opportunities must not be lost in acquiring sufficient information before the scene is disturbed. With the growth in information technology (IT) in many other scientific fields, there are also substantial opportunities for IT in the area of forensic science. The thesis sought to explore means by which IT can assist and benefit the ways that forensic information can be illustrated and elucidated in a logical manner. The central research hypothesis considers that through the utilisation of low cost IT, the visual presentation of information will be of significant benefit to forensic science in particular for the recoding of crime scenes and its presentation in court. The research hypothesis was addressed by first exploring the current crime scene documentation techniques; their strengths and weaknesses, giving indication to the possible niche that technology could occupy within forensic science. The underlying principles of panoramic technology were examined, highlighting its ability to express spatial information efficiently. Through literature review and case studies, the current status of the technology within the forensic community and courtrooms was also explored to gauge its possible acceptance as a forensic tool. This led to the construction of a low cost semi-automated imaging system capable of capturing the necessary images for the formation of a panorama. This provides the ability to pan around; effectively placing the viewer at the crime scene. Evaluation and analysis involving forensic personnel was performed to assess the capabilities and effectiveness of the imaging system as a forensic tool. The imaging system was found to enhance the repertoire of techniques available for crime scene documentation; possessing sufficient capabilities and benefits to warrant its use within the area of forensics, thereby supporting the central hypothesis

Vision Sensors and Edge Detection

Vision Sensors and Edge Detection book reflects a selection of recent developments within the area of vision sensors and edge detection. There are two sections in this book. The first section presents vision sensors with applications to panoramic vision sensors, wireless vision sensors, and automated vision sensor inspection, and the second one shows image processing techniques, such as, image measurements, image transformations, filtering, and parallel computing

Subsurface robotic exploration for geomorphology, astrobiology and mining during MINAR6 campaign, Boulby Mine, UK : part I (Rover development)

Acknowledgement. The authors of this paper would like to thank Kempe Foundation for its generous funding support to develop KORE, the workshop at the Teknikens Hus, Luleå, for their invaluable and unconditional support in helping with the fabrication of the Rover components and the organizers of the MINAR campaign comprising the UK Centre of Astrobiology, Dark Matter Research Facility and the Israel Chemicals Limited (ICL), UK.Peer reviewedPublisher PD

Digital Facilities: A Bim Capturing Reality Framework And Integration With Building Management System

Qatar University (QU) had been sharing the vision of the country in developing various technological aspects and approaches in order to achieve global standard qualities and accomplishments. In this research, from the construction management perspective, referring to QU digitalization initiative objectives and considering global market processes, computerized capturing framework of campus existing building will be executed, discussed and analyzed in-depth, through Building Information Modeling (BIM). Current worldwide capturing techniques, such as as-built drawings, laser scanning and UAV will be implemented and illustrated along with their cost, time and quality analysis. The study will also discuss the novel approach of integrating BIM Model with present Building Management System (BMS) of QU; and its contribution towards Facility Management (FM). This integrative activity between BMS real-time values readings and BIM data representation will add-value to Facility Management (FM) workflow efficiency and performance of the university or any other applicable party with the same environment. Concluding the best practice of reality digital capturing towards a successful integration prior to achieving study objectives, with diverse recommendations, discussions and future work

Northern Tornadoes Project. Annual Report 2022

NTP’s fourth year of detecting, surveying and documenting tornadoes and other damaging wind events across Canada saw a return to a more familiar pattern. In 2021, only two tornadoes were recorded across the Prairies over the 60 days with the highest climatological frequency (mid-June to mid-August).In 2022, 39 tornadoes were confirmed there, with 33 of them during that peak period. However, a catastrophic spring derecho set the stage for a very different and very active season in Ontario and Québec. The May 21st derecho is now one of the most deadly and costly thunderstorm events on record in Canada. While over a billion dollars in insured losses was recorded, 12 people lost their lives and at least another 12 were injured. The storm’s damage path extended over 1,000 km across the most densely populated region in Canada. Though NTP field teams were deployed shortly after the event occurred, it took the rest of the summer to fully investigate this devastating event. Through the rest of the season, Ontario and Québec recorded 77 tornadoes, nearly double what was recorded across the entire Prairies. And the total number of tornadoes across Canada during the 2022 season is tied for the highest we’ve recorded at 117 – and closer to the 150 or so that we thought might be occurring based on statistical analysis. So, it appears that the number of \u27missing tornadoes\u27 is indeed dropping due to the efforts of NTP

Virtual Heritage: new technologies for edutainment

Cultural heritage represents an enormous amount of information and knowledge. Accessing this treasure chest allows not only to discover the legacy of physical and intangible attributes of the past but also to provide a better understanding of the present. Museums and cultural institutions have to face the problem of providing access to and communicating these cultural contents to a wide and assorted audience, meeting the expectations and interests of the reference end-users and relying on the most appropriate tools available. Given the large amount of existing tangible and intangible heritage, artistic, historical and cultural contents, what can be done to preserve and properly disseminate their heritage significance? How can these items be disseminated in the proper way to the public, taking into account their enormous heterogeneity? Answering this question requires to deal as well with another aspect of the problem: the evolution of culture, literacy and society during the last decades of 20th century. To reflect such transformations, this period witnessed a shift in the museum’s focus from the aesthetic value of museum artifacts to the historical and artistic information they encompass, and a change into the museums’ role from a mere "container" of cultural objects to a "narrative space" able to explain, describe, and revive the historical material in order to attract and entertain visitors. These developments require creating novel exhibits, able to tell stories about the objects and enabling visitors to construct semantic meanings around them. The objective that museums presently pursue is reflected by the concept of Edutainment, Education + Entertainment. Nowadays, visitors are not satisfied with ‘learning something’, but would rather engage in an ‘experience of learning’, or ‘learning for fun’, being active actors and players in their own cultural experience. As a result, institutions are faced with several new problems, like the need to communicate with people from different age groups and different cultural backgrounds, the change in people attitude due to the massive and unexpected diffusion of technology into everyday life, the need to design the visit by a personal point of view, leading to a high level of customization that allows visitors to shape their path according to their characteristics and interests. In order to cope with these issues, I investigated several approaches. In particular, I focused on Virtual Learning Environments (VLE): real-time interactive virtual environments where visitors can experience a journey through time and space, being immersed into the original historical, cultural and artistic context of the work of arts on display. VLE can strongly help archivists and exhibit designers, allowing to create new interesting and captivating ways to present cultural materials. In this dissertation I will tackle many of the different dimensions related to the creation of a cultural virtual experience. During my research project, the entire pipeline involved into the development and deployment of VLE has been investigated. The approach followed was to analyze in details the main sub-problems to face, in order to better focus on specific issues. Therefore, I first analyzed different approaches to an effective recreation of the historical and cultural context of heritage contents, which is ultimately aimed at an effective transfer of knowledge to the end-users. In particular, I identified the enhancement of the users’ sense of presence in VLE as one of the main tools to reach this objective. Presence is generally expressed as the perception of 'being there', i.e. the subjective belief of users that they are in a certain place, even if they know that the experience is mediated by the computer. Presence is related to the number of senses involved by the VLE and to the quality of the sensorial stimuli. But in a cultural scenario, this is not sufficient as the cultural presence plays a relevant role. Cultural presence is not just a feeling of 'being there' but of being - not only physically, but also socially, culturally - 'there and then'. In other words, the VLE must be able to transfer not only the appearance, but also all the significance and characteristics of the context that makes it a place and both the environment and the context become tools capable of transferring the cultural significance of a historic place. The attention that users pay to the mediated environment is another aspect that contributes to presence. Attention is related to users’ focalization and concentration and to their interests. Thus, in order to improve the involvement and capture the attention of users, I investigated in my work the adoption of narratives and storytelling experiences, which can help people making sense of history and culture, and of gamification approaches, which explore the use of game thinking and game mechanics in cultural contexts, thus engaging users while disseminating cultural contents and, why not?, letting them have fun during this process. Another dimension related to the effectiveness of any VLE is also the quality of the user experience (UX). User interaction, with both the virtual environment and its digital contents, is one of the main elements affecting UX. With respect to this I focused on one of the most recent and promising approaches: the natural interaction, which is based on the idea that persons need to interact with technology in the same way they are used to interact with the real world in everyday life. Then, I focused on the problem of presenting, displaying and communicating contents. VLE represent an ideal presentation layer, being multiplatform hypermedia applications where users are free to interact with the virtual reconstructions by choosing their own visiting path. Cultural items, embedded into the environment, can be accessed by users according to their own curiosity and interests, with the support of narrative structures, which can guide them through the exploration of the virtual spaces, and conceptual maps, which help building meaningful connections between cultural items. Thus, VLE environments can even be seen as visual interfaces to DBs of cultural contents. Users can navigate the VE as if they were browsing the DB contents, exploiting both text-based queries and visual-based queries, provided by the re-contextualization of the objects into their original spaces, whose virtual exploration can provide new insights on specific elements and improve the awareness of relationships between objects in the database. Finally, I have explored the mobile dimension, which became absolutely relevant in the last period. Nowadays, off-the-shelf consumer devices as smartphones and tablets guarantees amazing computing capabilities, support for rich multimedia contents, geo-localization and high network bandwidth. Thus, mobile devices can support users in mobility and detect the user context, thus allowing to develop a plethora of location-based services, from way-finding to the contextualized communication of cultural contents, aimed at providing a meaningful exploration of exhibits and cultural or tourist sites according to visitors’ personal interest and curiosity