Investigating Precise Control in Spatial Interactions: Proxemics, Kinesthetics, and Analytics

Augmented and Virtual Reality (AR/VR) technologies have reshaped the way in which we perceive the virtual world. In fact, recent technological advancements provide experiences that make the physical and virtual worlds almost indistinguishable. However, the physical world affords subtle sensorimotor cues which we subconsciously utilize to perform simple and complex tasks in our daily lives. The lack of this affordance in existing AR/VR systems makes it difficult for their mainstream adoption over conventional $2D$ user interfaces. As a case in point, existing spatial user interfaces (SUI) lack the intuition to perform tasks in a manner that is perceptually familiar to the physical world. The broader goal of this dissertation lies in facilitating an intuitive spatial manipulation experience, specifically for motor control. We begin by investigating the role of proximity to an action on precise motor control in spatial tasks. We do so by introducing a new SUI called the Clock-Maker's Work-Space (CMWS), with the goal of enabling precise actions close to the body, akin to the physical world. On evaluating our setup in comparison to conventional mixed-reality interfaces, we find CMWS to afford precise actions for bi-manual spatial tasks. We further compare our SUI with a physical manipulation task and observe similarities in user behavior across both tasks. We subsequently narrow our focus on studying precise spatial rotation. We utilize haptics, specifically force-feedback (kinesthetics) for augmenting fine motor control in spatial rotational task. By designing three kinesthetic rotation metaphors, we evaluate precise rotational control with and without haptic feedback for 3D shape manipulation. Our results show that haptics-based rotation algorithms allow for precise motor control in 3D space, also, help reduce hand fatigue. In order to understand precise control in its truest form, we investigate orthopedic surgery training from the point of analyzing bone-drilling tasks. We designed a hybrid physical-virtual simulator for bone-drilling training and collected physical data for analyzing precise drilling action. We also developed a Laplacian based performance metric to help expert surgeons evaluate the resident training progress across successive years of orthopedic residency

Tendencias y características de la realidad virtual: Una revisión de la literatura

Virtual Reality has greatly evolved between 1940s and 1990s. Since then, it has been implemented in multiple research and knowledge areas, the most recognized being the entertainment industry. In order to establish the state of Virtual Reality and get an idea of its prospects, 537 scientific documents have been reviewed, applying search criteria, more specifically, Virtual Reality applied to education. A bibliometric analysis was realized based on summarizing each document, as well as its keywords and trends, then proceeding to categorize each one, according to the field of application. It was found that Virtual Reality is having relevance in medicine and training area, due to the ability to simulate difficult situations and above all, specific conditions raised by instructors. The future of Virtual Reality as a tool to train professionals in multiples areas is promising.La realidad virtual ha tenido una gran avance entre la década 1940 y 1990. Desde entonces, se ha implementado en múltiples áreas, tanto de la investigación como del conocimiento, siendo más reconocida en la industria del entretenimiento. Con el propósito de establecer el estado de la realidad virtual y obtener una idea de su futuro, 537 documentos científicos han sido revisados aplicando criterios de búsqueda específicos, como realidad virtual aplicada a la educación. Un análisis bibliométrico fue realizado, teniendo como base un resumen y descripción de cada artículo, así como sus palabras claves y tendencias, procediendo a categorizar cada documento de acuerdo con su campo de aplicación. Se encontró que la realidad virtual tiene una gran relevancia en la medicina y en el entrenamiento de personas, debido a su capacidad de simular situaciones difíciles y, sobre todo, condiciones específicas requeridas por instructores. El futuro de la realidad virtual como herramienta de entrenamiento para los profesionales de múltiples áreas es promisori

Virtual Reality Simulation of Glenoid Reaming Procedure

Glenoid reaming is a bone machining operation in Total Shoulder Arthroplasty (TSA) in which the glenoid bone is resurfaced to make intimate contact with implant undersurface. While this step is crucial for the longevity of TSA, many surgeons find it technically challenging. With the recent advances in Virtual Reality (VR) simulations, it has become possible to realistically replicate complicated operations without any need for patients or cadavers, and at the same time, provide quantitative feedback to improve surgeons\u27 psycho-motor skills. In light of these advantages, the current thesis intends to develop tools and methods required for construction of a VR simulator for glenoid reaming, in an attempt to construct a reliable tool for preoperative training and planning for surgeons involved with TSA. Towards the end, this thesis presents computational algorithms to appropriately represent surgery tool and bone in the VR environment, determine their intersection and compute realistic haptic feedback based on the intersections. The core of the computations is constituted by sampled geometrical representations of both objects. In particular, point cloud model of the tool and voxelized model of bone - that is derived from Computed Tomography (CT) images - are employed. The thesis shows how to efficiently construct these models and adequately represent them in memory. It also elucidates how to effectively use these models to rapidly determine tool-bone collisions and account for bone removal momentarily. Furthermore, the thesis applies cadaveric experimental data to study the mechanics of glenoid reaming and proposes a realistic model for haptic computations. The proposed model integrates well with the developed computational tools, enabling real-time haptic and graphic simulation of glenoid reaming. Throughout the thesis, a particular emphasis is placed upon computational efficiency, especially on the use of parallel computing using Graphics Processing Units (GPUs). Extensive implementation results are also presented to verify the effectiveness of the developments. Not only do the results of this thesis advance the knowledge in the simulation of glenoid reaming, but they also rigorously contribute to the broader area of surgery simulation, and can serve as a step forward to the wider implementation of VR technology in surgeon training programs

Evaluating Human Performance for Image-Guided Surgical Tasks

The following work focuses on the objective evaluation of human performance for two different interventional tasks; targeted prostate biopsy tasks using a tracked biopsy device, and external ventricular drain placement tasks using a mobile-based augmented reality device for visualization and guidance. In both tasks, a human performance methodology was utilized which respects the trade-off between speed and accuracy for users conducting a series of targeting tasks using each device. This work outlines the development and application of performance evaluation methods using these devices, as well as details regarding the implementation of the mobile AR application. It was determined that the Fitts’ Law methodology can be applied for evaluation of tasks performed in each surgical scenario, and was sensitive to differentiate performance across a range which spanned experienced and novice users. This methodology is valuable for future development of training modules for these and other medical devices, and can provide details about the underlying characteristics of the devices, and how they can be optimized with respect to human performance