Augmented reality (AR) and virtual reality (VR) applications during Covid-19 pandemic among preclinical medical and dentistry students: a mini-review

The efficacy of online learning in delivering theoretical knowledge with appropriate content to students is imperative, especially in the Covid-19 pandemic era. Substantial interactive teaching materials were developed for higher education. However, some were designed immensely general, especially in fulfilling the syllabus of preclinical medical and dentistry students. Augmented reality (AR) is an interactive three-dimension (3D) experience that uses computers to overlay virtual information in the real world whereas virtual reality (VR) is a computer-generated artificial recreation of a real-life experience or situation. Interestingly, both can be complemented and integrated into online and traditional teaching methods. Implementation of these technologies will increase the learning efficacy in understanding the human body’s anatomical and physiological changes in the normal or pathological state. As AR and VR technologies are continuously evolving, this review provides the preview and current updates on AR and VR applications in medical and dentistry education which may benefit the educators within these specialities

The integration of CFD and VR methods to assist auxiliary ventilation practice

The current trend towards the adoption of retreat longwall mining methods and the associated rapid development of the access drivages has exacerbated the environmental conditions experienced within these workings. The combined use of roof bolt and continuous miner systems has improved the face advance rate within rapid development drivages. In order to maintain adequate dust and gas control it is essential that the auxiliary ventilation and monitoring systems are correctly installed and maintained. The causes of many potential environmental hazards experienced within auxiliary ventilated rapid development drivages, are often attributed to a failure by the workforce and supervisory officials to maintain the correct installation, maintenance and operational standards of the ventilation and mining systems. The potential ventilation hazards encountered may include: the failure to deliver the required fresh air quantity and velocity to rapidly dilute and disperse methane gas liberated in the vicinity of the cutting face, or the failure to maintain sufficient exhaust air quantity in the vicinity of the cut to adequately capture dust produced on cutting and loading of the extracted mineral. Results of recent research studies have demonstrated that validated Computational Fluid Dynamics (CFD) simulation models can adequately replicate examples of good and bad ventilation. CFD models may be constructed and solved to examine the relative ventilation benefits produced by alternative mining and auxiliary ventilation configurations. These models enable the practitioner to predict and visualise the velocity, pressure and contaminant fields within an auxiliary ventilated drivage. This research project has developed a prototype educational aid, which animates and visualises these airflow and pollutant dispersion patterns within a Virtual Reality (VR) model. By introducing a pollutant such as methane into the CFD models, the VR simulation highlights regions of potential methane concentration build-up to the trainee. The application also allows the user to select/investigate the environmental consequences of enacting a number of remedial actions

Technological framework for ubiquitous interactions using context–aware mobile devices

This report presents research and development of dedicated system architecture, designed to enable its users to interact with each other as well as to access information on Points of Interest that exist in their immediate environment. This is accomplished through managing personal preferences and contextual information in a distributed manner and in real-time. The advantage of this system architecture is that it uses mobile devices, heterogeneous sensors and a selection of user interface paradigms to produce a sociotechnical framework to enhance the perception of the environment and promote intuitive interactions. The thrust of the work has been on software development and component integration. Iterative prototyping was adopted as a development method in order to effectively implement the users’ feedback and establish a platform for collaboration that closely meets the requirements and aids their decision-making process. The requirement acquisition was followed by the system-modelling phase in order to produce a robust software prototype. The implementation includes component-based development and extensive use of design patterns over native programming. Conclusively, the software product has become the means to evaluate differences in the use of mixed reality technologies in a ubiquitous scenario. The prototype can query a number of context sources such as sensors, or details of the personal profile, to acquire relevant data. The data (and metadata) is stored in opensource structures, so that they are accessible at every layer of the system architecture and at any time. By proactively processing the acquired context, the system can assist the users in their tasks (e.g. navigation) without explicit input – e.g. by simply creating a gesture with the device. However, advanced interaction with the application via the user interface is available for requests that are more complex. Representations of the real world objects, their spatial relations and other captured features of interest are visualised on scalable interfaces, ranging from 2D to 3D models and from photorealism to stylised clues and symbols. Two principal modes of operation have been implemented; one, using geo-referenced virtual reality models of the environment, updated in real time, and second, using the overlay of descriptive annotations and graphics on the video images of the surroundings, captured by a video camera. The latter is referred to as augmented reality. The continuous feed of the device position and orientation data, from the GPS receiver and the digital compass, into the application, makes the framework fit for use in unknown environments and therefore suitable for ubiquitous operation. This is one of the novelties of the proposed framework, because it enables a whole range of social, peer-to-peer interactions to take place. The scenarios of how the system could be employed to pursue these remote interactions and collaborative efforts on mobile devices are addressed in the context of urban navigation. The conceptual design and implementation of the novel location and orientation based algorithm for mobile AR are presented in detail. The system is, however, multifaceted and capable of supporting peer-to-peer exchange of information in a pervasive fashion, usable in various contexts. The modalities of these interactions are explored and laid out in several scenarios, but particularly in the context of user adoption. Two evaluation tasks took place. The preliminary evaluation examined certain aspects that influence user interaction while being immersed in a virtual environment, whereas the second summative evaluation compared the utility and certain usability aspects of the AR and VR interfaces

Linking Spatial Video and GIS

Spatial Video is any form of geographically referenced videographic data. The forms in which it is acquired, stored and used vary enormously; as does the standard of accuracy in the spatial data and the quality of the video footage. This research deals with a specific form of Spatial Video where these data have been captured from a moving road-network survey vehicle. The spatial data are GPS sentences while the video orientation is approximately orthogonal and coincident with the direction of travel. GIS that use these data are usually bespoke standalone systems or third party extensions to existing platforms. They specialise in using the video as a visual enhancement with limited spatial functionality and interoperability. While enormous amounts of these data exist, they do not have a generalised, cross-platform spatial data structure that is suitable for use within a GIS. The objectives of this research have been to define, develop and implement a novel Spatial Video data structure and demonstrate how this can achieve a spatial approach to the study of video. This data structure is called a Viewpoint and represents the capture location and geographical extent of each video frame. It is generalised to represent any form or format of Spatial Video. It is shown how a Viewpoint improves on existing data structure methodologies and how it can be theoretically defined in 3D space. A 2D implementation is then developed where Viewpoints are constructed from the spatial and camera parameters of each survey in the study area. A number of problems are defined and solutions provided towards the implementation of a post-processing system to calculate, index and store each video frame Viewpoint in a centralised spatial database. From this spatial database a number of geospatial analysis approaches are demonstrated that represent novel ways of using and studying Spatial Video based on the Viewpoint data structure. Also, a unique application is developed where the Viewpoints are used as a spatial control to dynamically access and play video in a location aware system. While video has been to date largely ignored as a GIS spatial data source; it is shown through this novel Viewpoint implementation and the geospatial analysis demonstrations that this need not be the case anymore

Application of mixed and virtual reality in geoscience and engineering geology

Visual learning and efficient communication in mining and geotechnical practices is crucial, yet often challenging. With the advancement of Virtual Reality (VR) and Mixed Reality (MR) a new era of geovisualization has emerged. This thesis demonstrates the capabilities of a virtual continuum approach using varying scales of geoscience applications. An application that aids analyses of small-scale geological investigation was constructed using a 3D holographic drill core model. A virtual core logger was also developed to assist logging in the field and subsequent communication by visualizing the core in a complementary holographic environment. Enriched logging practices enhance interpretation with potential economic and safety benefits to mining and geotechnical infrastructure projects. A mine-scale model of the LKAB mine in Sweden was developed to improve communication on mining induced subsidence between geologists, engineers and the public. GPS, InSAR and micro-seismicity data were hosted in a single database, which was geovisualized through Virtual and Mixed Reality. The wide array of applications presented in this thesis illustrate the potential of Mixed and Virtual Reality and improvements gained on current conventional geological and geotechnical data collection, interpretation and communication at all scales from the micro- (e.g. thin section) to the macro- scale (e.g. mine)

Human-Centric Machine Vision

Recently, the algorithms for the processing of the visual information have greatly evolved, providing efficient and effective solutions to cope with the variability and the complexity of real-world environments. These achievements yield to the development of Machine Vision systems that overcome the typical industrial applications, where the environments are controlled and the tasks are very specific, towards the use of innovative solutions to face with everyday needs of people. The Human-Centric Machine Vision can help to solve the problems raised by the needs of our society, e.g. security and safety, health care, medical imaging, and human machine interface. In such applications it is necessary to handle changing, unpredictable and complex situations, and to take care of the presence of humans

Entanglement : an investigation into the effective union of contemporary art and science communication : a thesis presented in partial fulfilment of the requirements for the degree of Doctor of Philosophy in Fine Arts at Massey University, Wellington, New Zealand

All Figures are copyrighted and re-used with permission.Virtual reality (VR) technology is increasingly providing opportunities for new contemporary art experiences. This creative practice research has been developed to provide one such contribution. It offers innovative employment of the immersive capabilities of VR to engage with and convey complex scientific theories, and to stimulate changes in mental processes to unlock these concepts. The research highlights empirical similarities between art and science to propose that creative aspects of art can be considered proximate to the creative qualities required to understand quantum theories. In order to reveal this, the body of research engaged specifically with quantum entanglement, because of its well documented existence¹ combined with the more challenging considerations of how ‘communication’ can occur at a quantum level. By providing metaphoric immersive experiences of quantum entanglement, a contribution of ‘scientific communication’ is made as defined by the evocation of awareness, enjoyment, and interest, questioning of opinions and providing new perspectives of understanding.² This research posits that there is a fertile, effective terrain to explore in the union of the fields of contemporary art and science communication. Considerations of constructivist theories of knowledge and the concept of paradigm shifts³ are used whereby new insights into knowledge processes can be experienced through VR art. Here, simulacra, cognitive dissonance and the technological sublime afford a framework to create experiences of conflicting realities. It is due to the immersive strengths of VR which are exploited and subverted through my designs that these experiences can be facilitated for the viewer. The culmination of this research is Entangled, a VR art installation which provides interplays between virtual and physical spaces while also offering entry-points to contemplate and understand quantum theories. Critical analysis of this project is supported by focus group and questionnaire responses. These findings prove how viewers perceived the project as an aesthetic art work and that by recognising scientific underpinnings, an effective engagement and participation in elements of scientific communication occurred at varying levels. The work provided new perspectives on the properties of quantum entanglement. This facilitated cognitive and experiential awareness providing opportunities for viewers to encounter conflicting knowledge systems. The challenge in this creative practice research was to create aesthetic experiences that contravene common sense reasoning and provide insights into the type of thought processes and experiential perception that is required to deepen and expand our understanding of our physical reality. In the present era of an evolution of super- technologies, now past its nascent stage, Entangled offers exposure to the types of interfaces that this thesis asserts will increasingly be encountered when comprehending our reality in the 21st century and beyond.⁴ ¹ References to the proven existence of quantum entanglement are provided in section 1.6. ² This definition of scientific communication is expanded in section 1.1. ³ Paradigm shifts are times when the familiar framework has to be profoundly changed. This is discussed in detail in section 1.3. ⁴ Quantum entanglement is only one possible area that will cause our experience of reality to change radically. For example biotechnologies, nanotechnologies, artificial intelligence (AI) and human/AI interfaces to name some

High fidelity olfaction simulation for virtual environments

Olfaction is a key human sense due to its close connection to the brain, specifically memory and decision making, along with its effects on behaviour and emotions. Clearly, in order for virtual environments to reach perceptual equivalence with the real world, olfaction needs to be incorporated. Perceptual equivalence is defined as the same cognitive response occurring with the users whether they are exposed to real or synthesized stimuli regardless of the levels of duration, intensity or nature. While there have been previous attempts to use olfactory stimuli in virtual environments, these have been limited and not addressed some key aspects of olfaction and its corresponding stimuli, such the effects of high odour concentrations and smell habituation. This thesis presents a framework for a physically accurate olfactory pipeline in order to help provide high fidelity perceptually equivalent virtual environments. The pipeline consists of stages for capturing, data storage, reproduction, and finally delivering stimuli to end users. In particular this thesis focuses on the final delivery stage. It presents a novel physical calibration for the olfaction delivery system and a new human perceptual calibration procedure to ensure a robust and repeatable experience. For the first half of the delivery stage, this thesis provides the blueprint for a physically accurate smell display. This is subsequently shown be capable of reproducing smell stimuli with both accuracy, to intended real world concentrations, and precision, such that the outputted olfactory stimuli are consistently presented at the specified level. In addition, similar to other existing olfactory displays, this smell display is a low cost solution as well as a straightforward design with the intrinsic ability to provide virtually, immediate temporal displacement. The validation of the physical calibration of the display is based on fundamental laws of chemistry to provide the same result under the same specified conditions. For the second half of the delivery stage, a perceptual calibration procedure is presented to calculate Just Noticeable Differences for olfactory stimuli. The purpose of this is to be able to create normalised stimuli levels which are perceivable by the general population but are also perceptually similar across different odours. The work provides two Just Noticeable Difference stimuli levels for three target single molecule odours which have been calibrated on the olfactory display based on a sample population (N=10). These determined stimuli levels can subsequently be utilised as generalisable points for further experimental use with participants when investigating both olfactory phenomena and relationships between olfactory and other sensual stimuli. A final experimental study is proposed in this thesis to clearly demonstrate the capabilities of the research and to explore the possibility of olfactory attention-masking phenomena, similar to those seen in the visual domain. Participants were asked to preferentially rank the odours presented to them based on images of the smell source. Odours were the same as those used in the perception calibration, and were presented as randomised dual conditions either in combination with the other odours or with a blank. Participants were consistently able to correctly identify the correct odours with the correct images, ranking the correct answers either primary or secondary. In addition, a number of preferences for some odours over others were identified. In conclusion, this thesis outlines a novel framework for addressing the physical and perceptual aspects of olfaction in order to provide an accurate representation of a real world equivalent olfactory experience. Experiments show humans are readily able to distinguish between odours when presented at the same time and the evidence obtained suggests there may be attention-masking phenomena in the olfactory domai

Presenting ancient history through serious games: a case-study in Assyriology

This thesis investigates how ancient history can be effectively presented through serious games for educational purposes, by presenting a case study in the field of Assyriology. It is anticipated that the results are applicable to other fields of history and cultural heritage. A model is presented to describe how heritage and historical content can be manifested in video games, for the design of serious games for heritage and the analysis of commercial games that present historical material. The theories of reduced fidelity constrained virtual environments are applied to serious games for heritage, to reduce required development resources. A constrained implementation of a serious game for Assyriology is tested against an equivalent 3D environment, and results indicate the constrained environment can achieve comparable levels of presence, enjoyment, quality, and interest in the subject. Based on an interview with an Assyriology field expert, a methodology for the analysis and design of serious games for heritage is presented, based on activity theory. The methodology is applied to the analysis and redesign of a serious game for Assyriology, and the development of a playable prototype. An online user-test showed the redesigned game was enjoyed by participants and was effective at achieving its learning objectives

Maritime Augmented Reality mit a prioriWissen aus Seekarten

The main objective of this thesis is to provide a concept to augment mar- itime sea chart information into the camera view of the user. The benefit is the simpler navigation due to the offered 3D information and the overlay onto the real 3D environment. In the maritime context special conditions hold. The sensor technologies have to be reliable in the environment of a ship’s ferrous construction. The aug- mentation of the objects has to be very precise due to the far distances of observable objects on the sea surface. Furthermore, the approach has to be reliable due to the wide range of light conditions. For a practical solution, the system has to be mobile, light-weight and with a real-time performance. To achieve this goal, the requirements are set, the possible measurement units and the data base structure are presented. First, the requirements are analyzed and a suitable system is designed. By the combination of proper sensor techniques, the local position and orienta- tion of the user can be estimated. To verify the concept, several prototypes with exchangeable units have been evaluated. This first concept is based on a marker-based approach which leads to some drawbacks. To overcome the drawbacks, the second aspect is the improvement of the sys- tem and the analysis of markerless approaches. One possible strategy will be presented. The approach uses the statistical technique of Bayesian networks to vote for single objects in the environment. By this procedure it will be shown, that due to the a priori information the underlying sea chart system has the most benefit. The analysis of the markerless approach shows, that the sea charts structure has to be adapted to the new requirements of interactive 3D augmentation scenes. After the analysis of the chart data concept, an approach for the optimization of the charts by building up an object-to-object topology within the charts data and the Bayesian object detection approach is presented. Finally, several evaluations show the performance of the imple- mented evaluation application.Diese Arbeit stellt ein Konzept zur Verfügung, um Seekarteninformationen in eine Kamera so einzublenden, dass die Informationen lagerichtig im Sichtfeld des Benutzers erscheinen. Der Mehrwert ist eine einfachere Navigation durch die Nutzung von 3D-Symbolen in der realen Umgebung. Im maritimen Umfeld gelten besondere Anforderungen an die Aufgabenstellung. Die genutzten Sensoren müssen in der Lage sein, robuste Daten in Anwesenheit der eisenhaltigen Materialien auf dem Schiff zu liefern. Die Augmentierung muss hoch genau berechnet werden, da die beobachtbaren Objekte zum Teil sehr weit entfernt auf der Meeresoberfläche verteilt sind. Weiterhin gelten die Bedingungen einer Außenumgebung, wie variierende Wetter- und Lichtbedingungen. Um eine praktikable Anwendung gewährleisten zu können, ist ein mobiles, leicht-gewichtiges und echtzeitfähiges System zu entwickeln. In dieser Arbeit werden die Anforderungen gesetzt und Konzepte für die Hardware- und Softwarelösungen beschrieben. Im ersten Teil werden die Anforderungen analysiert und ein geeignetes Hardwaresystem entwickelt. Durch die passende Kombination von Sensortechnologien kann damit die lokale Position und Orientierung des Benutzers berechnet werden. Um das Konzept zu evaluieren sind verschiedene modulare Hardware- und Softwarekonzepte als Prototypen umgesetzt worden. Das erste Softwarekonzept befasst sich mit einem markerbasierten Erkennungsalgorithmus, der in der Evaluation einige Nachteile zeigt. Dementsprechende Verbesserungen wurden in einem zweiten Softwarekonzept durch einen markerlosen Ansatz umgesetzt. Dieser Lösungsansatz nutzt Bayes'sche Netzwerke zur Erkennung einzelner Objekte in der Umgebung. Damit kann gezeigt werden, dass mit der Hilfe von a priori Informationen die dem System zugrunde liegenden Seekarten sehr gut zu diesem Zweck genutzt werden können. Die Analyse des Systemkonzeptes zeigt des weiteren, dass die Datenstruktur der Seekarten für die Anforderungen einer interaktiven, benutzergeführten 3D- Augmentierungsszene angepasst werden müssen. Nach der ausführlichen Analyse des Seekarten-Datenkonzeptes wird ein Lösungsansatz zur Optimierung der internen Seekartenstruktur aufgezeigt. Dies wird mit der Erstellung einer Objekt-zu-Objekt-Topologie in der Datenstruktur und der Verbindung zum Bayes'schen Objekterkennungsalgorithmus umgesetzt. Anschließend zeigen Evaluationen die Fähigkeiten des endgültigen Systems