Handling Topological Changes during Elastic Registration: Application to Augmented Reality in Laparoscopic Surgery

International audiencePurpose: Locating the internal structures of an organ is a critical aspect of many surgical procedures. Minimally invasive surgery, associated with augmented reality techniques, offers the potential to visualize inner structures, allowing for improved analysis, depth perception or for supporting planning and decision systems.Methods: Most of the current methods dealing with rigid or non-rigid augmented reality make the assumption that the topology of the organ is not modified. As surgery relies essentially on cutting and dissection of anatomical structures, such methods are limited to the early stages of the surgery.We solve this shortcoming with the introduction of a method for physics-based elastic registration using a single view from a monocular camera.Singularities caused by topological changes are detected and propagated to the pre-operative model. This significantly improves the coherence between the actual laparoscopic view and the model, and provides added value in terms of navigation and decision-making, e.g. by overlaying the internal structures of an organ on the laparoscopic view.Results: Our real time augmentation method is assessed on several scenarios, using synthetic objects and real organs. In all cases, the impact of our approach is demonstrated, both qualitatively and quantitatively.Conclusion: The presented approach tackles the challenge of localizing internal structures throughout a complete surgical procedure, even after surgical cuts. This information is crucial for surgeons to improve the outcome for their surgical procedure and avoid complications

XR in Surgery. Spatial and embodied computing in digital surgery: Technology, Application, Design

Die Arbeit untersucht die Anwendungsfelder, Darstellungsprinzipien und Funktionsweise von extended reality in der chirurgischen Praxis. Anhand zweier Fallstudien zu mixed und virtual reality werden die Einsatzorte und Praktiken von head-mounted Displays sowie das erforderliche Anwendungswissen herausgearbeitet. An den Beispielen der Einblendung anatomischer Darstellungen ins Sichtfeld des chirurgischen Personals sowie der volumetrischen Darstellung chirurgischer Arbeitsabläufe in einer virtuellen Trainingsumgebung werden zwei konkrete Anwendungsszenarien für den Einsatz von extended reality in der chirurgischen Praxis entwickelt. Darauf aufbauend werden die Chancen und Risiken evaluiert, die sich insgesamt für chirurgische Handlungen und Entscheidungen im Bezug auf die Wahrnehmung, Interpretation und Gestaltung von Bildern der extended reality ergeben.The work investigates the technology, application and design of extended reality in surgical practice. Based on two case studies on mixed and virtual reality, it analyses use of head-mounted displays as well as the required applied knowledge. Using the examples of superimposing anatomical images onto the field of vision of surgical staff and the volumetric representation of surgical workflows in a virtual training environment, two concrete application scenarios for the use of extended reality in surgical practice are developed. Based on these scenarios, the chances and risks are evaluated, which result overall for surgical acting and decision making in terms of perception, interpretation and design of images of extended reality