Medical Telementoring Using an Augmented Reality Transparent Display

Background The goal of this study was to design and implement a novel surgical telementoring system called the System for Telementoring with Augmented Reality (STAR) that uses a virtual transparent display to convey precise locations in the operating field to a trainee surgeon. This system was compared with a conventional system based on a telestrator for surgical instruction. Methods A telementoring system was developed and evaluated in a study which used a 1 × 2 between-subjects design with telementoring system, that is, STAR or conventional, as the independent variable. The participants in the study were 20 premedical or medical students who had no prior experience with telementoring. Each participant completed a task of port placement and a task of abdominal incision under telementoring using either the STAR or the conventional system. The metrics used to test performance when using the system were placement error, number of focus shifts, and time to task completion. Results When compared with the conventional system, participants using STAR completed the 2 tasks with less placement error (45% and 68%) and with fewer focus shifts (86% and 44%), but more slowly (19% for each task). Conclusions Using STAR resulted in decreased annotation placement error, fewer focus shifts, but greater times to task completion. STAR placed virtual annotations directly onto the trainee surgeon's field of view of the operating field by conveying location with great accuracy; this technology helped to avoid shifts in focus, decreased depth perception, and enabled fine-tuning execution of the task to match telementored instruction, but led to greater times to task completion

Augmented Reality Future Step Visualization for Robust Surgical Telementoring

Introduction Surgical telementoring connects expert mentors with trainees performing urgent care in austere environments. However, such environments impose unreliable network quality, with significant latency and low bandwidth. We have developed an augmented reality telementoring system that includes future step visualization of the medical procedure. Pregenerated video instructions of the procedure are dynamically overlaid onto the trainee's view of the operating field when the network connection with a mentor is unreliable. Methods Our future step visualization uses a tablet suspended above the patient's body, through which the trainee views the operating field. Before trainee use, an expert records a “future library” of step-by-step video footage of the operation. Videos are displayed to the trainee as semitransparent graphical overlays. We conducted a study where participants completed a cricothyroidotomy under telementored guidance. Participants used one of two telementoring conditions: conventional telestrator or our system with future step visualization. During the operation, the connection between trainee and mentor was bandwidth throttled. Recorded metrics were idle time ratio, recall error, and task performance. Results Participants in the future step visualization condition had 48% smaller idle time ratio (14.5% vs. 27.9%, P < 0.001), 26% less recall error (119 vs. 161, P = 0.042), and 10% higher task performance scores (rater 1 = 90.83 vs. 81.88, P = 0.008; rater 2 = 88.54 vs. 79.17, P = 0.042) than participants in the telestrator condition. Conclusions Future step visualization in surgical telementoring is an important fallback mechanism when trainee/mentor network connection is poor, and it is a key step towards semiautonomous and then completely mentor-free medical assistance systems

Models of Aviation Technical Communication with Augmented Reality

A fundamental characteristic of augmented reality (AR) is the overlay of computer graphics (e.g., installation instructions) on views of world objects (e.g., a section of an aircraft wing) and registration of those graphics to features in the world scene. In AR, the mechanisms of information retrieval rely on detection of features of real-world objects, and communication is by way of virtual content in a real world scene; causing information to become a function of the composite scene, where real and virtual elements are interdependent. Various models of technical media communication are made possible by these characteristics and are discussed in this paper, including authoring, where a world object reveals what a planner intends it to reveal; image-based dialog, where substrate video is merged with annotations to communicate about world objects; and seamless collaboration, where attributes of in-person communication are replicated in AR. Familiar communication and information processing models are therefore expanded by AR. Communication and cognition aspects of human information processing are discussed in this paper in light of AR capabilities

Augmented Reality

Augmented Reality (AR) is a natural development from virtual reality (VR), which was developed several decades earlier. AR complements VR in many ways. Due to the advantages of the user being able to see both the real and virtual objects simultaneously, AR is far more intuitive, but it's not completely detached from human factors and other restrictions. AR doesn't consume as much time and effort in the applications because it's not required to construct the entire virtual scene and the environment. In this book, several new and emerging application areas of AR are presented and divided into three sections. The first section contains applications in outdoor and mobile AR, such as construction, restoration, security and surveillance. The second section deals with AR in medical, biological, and human bodies. The third and final section contains a number of new and useful applications in daily living and learning

Augmented reality meeting table: a novel multi-user interface for architectural design

Immersive virtual environments have received widespread attention as providing possible replacements for the media and systems that designers traditionally use, as well as, more generally, in providing support for collaborative work. Relatively little attention has been given to date however to the problem of how to merge immersive virtual environments into real world work settings, and so to add to the media at the disposal of the designer and the design team, rather than to replace it. In this paper we report on a research project in which optical see-through augmented reality displays have been developed together with prototype decision support software for architectural and urban design. We suggest that a critical characteristic of multi user augmented reality is its ability to generate visualisations from a first person perspective in which the scale of rendition of the design model follows many of the conventions that designers are used to. Different scales of model appear to allow designers to focus on different aspects of the design under consideration. Augmenting the scene with simulations of pedestrian movement appears to assist both in scale recognition, and in moving from a first person to a third person understanding of the design. This research project is funded by the European Commission IST program (IST-2000-28559)

The Effectiveness of Augmented Reality for Astronauts on Lunar Missions: An Analog Study

The uses of augmented reality and head-up displays are becoming more prominent in industries such as aviation, automotive, and medicine. An augmented reality device such as the Microsoft HoloLens can project holograms onto the user’s natural field of view to assist with completion of a variety of tasks. Unfortunately, only a little research and development has begun in the space sector for astronauts using these head-up displays. Future lunar missions could incorporate augmented reality for astronauts to ease task load and improve accuracy. This study evaluated the usability, subjective workload, and task performance of 22 participants using the Microsoft HoloLens to complete tasks that are analogous to those completed by astronauts on a lunar mission, including navigation, rock sample collection, and maintenance tasks. Results from the usability survey, NASA-TLX, and usability interview suggested that augmented reality could support astronaut missions by means of reduced workload and task errors. Usability data information collected from the participants sought to improve on the user interface and confirmed the aforementioned results. The researcher concluded that further research must be conducted to test the development of augmented reality interfaces along with the usability aspect by the National Aeronautics and Space Administration astronauts