Development of laparoscopic cholecystectomy simulator based on unity game engine

Fast development of the minimally invasive surgery (MIS) technology demands extra surgical skills training to meet advanced technological challenges. However, massive capital expenditures and ethical issues with safety considerations exist in traditional surgical training methods (e.g. using cadavers or animals). Those limitations turn Virtual Reality (VR) surgery simulation into a plausible alternative to provide a safe and repeatable virtual training environment. In this paper, we design and develop a game engine based laparoscopic cholecystectomy training simulator for surgeons to understand the surgery procedure and practice their surgical skills as well as decision making skills. Our design leverages physical simulation and haptic force feedback to offer trainees a realistic visual and tactile experience, respectively. We explore the possibility of using game engine rather than developing from scratch to build the surgical simulator. Based on the results and user feedbacks from a pilot experiment, we conclude that game engine is a viable option for creating a cost-effective, flexible and highly interactive virtual surgery training platform for pedagogical purpose, which can shorten the development time with some compromise in functionality

A multiple optical tracking based approach for enhancing hand-based interaction in virtual reality simulations

A thesis submitted in partial fulfilment of the requirements of the University of Wolverhampton for the degree of Doctor of Philosophy.Research exploring natural virtual reality interaction has seen significant success in optical tracker-based approaches, enabling users to freely interact using their hands. Optical based trackers can provide users with real-time, high-fidelity virtual hand representations for natural interaction and an immersive experience. However, work in this area has identified four issues: occlusion, field-of-view, stability and accuracy. To overcome the four key issues, researchers have investigated approaches such as using multiple sensors. Research has shown multi-sensor-based approaches to be effective in improving recognition accuracy. However, such approaches typically use statically positioned sensors, which introduce body occlusion issues that make tracking hands challenging. Machine learning approaches have also been explored to improve gesture recognition. However, such approaches typically require a pre-set gesture vocabulary limiting user actions with larger vocabularies hindering real-time performance. This thesis presents an optical hand-based interaction system that comprises two Leap Motion sensors mounted onto a VR headset at different orientations. Novel approaches to the aggregation and validation of sensor data are presented. A machine learning sub-system is developed to validate hand data received by the sensors. Occlusion detection, stability detection, inferred hands and a hand interpolation sub-system are also developed to ensure that valid hand representations are always shown to the user. In addition, a mesh conformation sub-system ensures 3D objects are appropriately held in a user’s virtual hand. The presented system addresses the four key issues of optical sessions to provide a smooth and consistent user experience. The MOT system is evaluated against traditional interaction approaches; gloves, motion controllers and a single front-facing sensor configuration. The comparative sensor evaluation analysed the validity and availability of tracking data, along with each sensors effect on the MOT system. The results show the MOT provides a more stable experience than the front-facing configuration and produces significantly more valid tracking data. The results also demonstrated the effectiveness of a 45-degree sensor configuration in comparison to a front-facing. Furthermore, the results demonstrated the effectiveness of the MOT systems solutions at handling the four key issues with optical trackers