Interfaces for Modular Surgical Planning and Assistance Systems

Modern surgery of the 21st century relies in many aspects on computers or, in a wider sense, digital data processing. Department administration, OR scheduling, billing, and - with increasing pervasion - patient data management are performed with the aid of so called Surgical Information Systems (SIS) or, more general, Hospital Information Systems (HIS). Computer Assisted Surgery (CAS) summarizes techniques which assist a surgeon in the preparation and conduction of surgical interventions. Today still predominantly based on radiology images, these techniques include the preoperative determination of an optimal surgical strategy and intraoperative systems which aim at increasing the accuracy of surgical manipulations. CAS is a relatively young field of computer science. One of the unsolved "teething troubles" of CAS is the absence of technical standards for the interconnectivity of CAS system. Current CAS systems are usually "islands of information" with no connection to other devices within the operating room or hospital-wide information systems. Several workshop reports and individual publications point out that this situation leads to ergonomic, logistic, and economic limitations in hospital work. Perioperative processes are prolonged by the manual installation and configuration of an increasing amount of technical devices. Intraoperatively, a large amount of the surgeons'' attention is absorbed by the requirement to monitor and operate systems. The need for open infrastructures which enable the integration of CAS devices from different vendors in order to exchange information as well as commands among these devices through a network has been identified by numerous experts with backgrounds in medicine as well as engineering. This thesis contains two approaches to the integration of CAS systems: - For perioperative data exchange, the specification of new data structures as an amendment to the existing DICOM standard for radiology image management is presented. The extension of DICOM towards surgical application allows for the seamless integration of surgical planning and reporting systems into DICOM-based Picture Archiving and Communication Systems (PACS) as they are installed in most hospitals for the exchange and long-term archival of patient images and image-related patient data. - For the integration of intraoperatively used CAS devices, such as, e.g., navigation systems, video image sources, or biosensors, the concept of a surgical middleware is presented. A c++ class library, the TiCoLi, is presented which facilitates the configuration of ad-hoc networks among the modules of a distributed CAS system as well as the exchange of data streams, singular data objects, and commands between these modules. The TiCoLi is the first software library for a surgical field of application to implement all of these services. To demonstrate the suitability of the presented specifications and their implementation, two modular CAS applications are presented which utilize the proposed DICOM extensions for perioperative exchange of surgical planning data as well as the TiCoLi for establishing an intraoperative network of autonomous, yet not independent, CAS modules.Die moderne Hochleistungschirurgie des 21. Jahrhunderts ist auf vielerlei Weise abhängig von Computern oder, im weiteren Sinne, der digitalen Datenverarbeitung. Administrative Abläufe, wie die Erstellung von Nutzungsplänen für die verfügbaren technischen, räumlichen und personellen Ressourcen, die Rechnungsstellung und - in zunehmendem Maße - die Verwaltung und Archivierung von Patientendaten werden mit Hilfe von digitalen Informationssystemen rationell und effizient durchgeführt. Innerhalb der Krankenhausinformationssysteme (KIS, oder englisch HIS) stehen für die speziellen Bedürfnisse der einzelnen Fachabteilungen oft spezifische Informationssysteme zur Verfügung. Chirurgieinformationssysteme (CIS, oder englisch SIS) decken hierbei vor allen Dingen die Bereiche Operationsplanung sowie Materialwirtschaft für spezifisch chirurgische Verbrauchsmaterialien ab. Während die genannten HIS und SIS vornehmlich der Optimierung administrativer Aufgaben dienen, stehen die Systeme der Computerassistierten Chirugie (CAS) wesentlich direkter im Dienste der eigentlichen chirugischen Behandlungsplanung und Therapie. Die CAS verwendet Methoden der Robotik, digitalen Bild- und Signalverarbeitung, künstlichen Intelligenz, numerischen Simulation, um nur einige zu nennen, zur patientenspezifischen Behandlungsplanung und zur intraoperativen Unterstützung des OP-Teams, allen voran des Chirurgen. Vor allen Dingen Fortschritte in der räumlichen Verfolgung von Werkzeugen und Patienten ("Tracking"), die Verfügbarkeit dreidimensionaler radiologischer Aufnahmen (CT, MRT, ...) und der Einsatz verschiedener Robotersysteme haben in den vergangenen Jahrzehnten den Einzug des Computers in den Operationssaal - medienwirksam - ermöglicht. Weniger prominent, jedoch keinesfalls von untergeordnetem praktischen Nutzen, sind Beispiele zur automatisierten Überwachung klinischer Messwerte, wie etwa Blutdruck oder Sauerstoffsättigung. Im Gegensatz zu den meist hochgradig verteilten und gut miteinander verwobenen Informationssystemen für die Krankenhausadministration und Patientendatenverwaltung, sind die Systeme der CAS heutzutage meist wenig oder überhaupt nicht miteinander und mit Hintergrundsdatenspeichern vernetzt. Eine Reihe wissenschaftlicher Publikationen und interdisziplinärer Workshops hat sich in den vergangen ein bis zwei Jahrzehnten mit den Problemen des Alltagseinsatzes von CAS Systemen befasst. Mit steigender Intensität wurde hierbei auf den Mangel an infrastrukturiellen Grundlagen für die Vernetzung intraoperativ eingesetzter CAS Systeme miteinander und mit den perioperativ eingesetzten Planungs-, Dokumentations- und Archivierungssystemen hingewiesen. Die sich daraus ergebenden negativen Einflüsse auf die Effizienz perioperativer Abläufe - jedes Gerät muss manuell in Betrieb genommen und mit den spezifischen Daten des nächsten Patienten gefüttert werden - sowie die zunehmende Aufmerksamkeit, welche der Operateur und sein Team auf die Überwachung und dem Betrieb der einzelnen Geräte verwenden muss, werden als eine der "Kinderkrankheiten" dieser relativ jungen Technologie betrachtet und stehen einer Verbreitung über die Grenzen einer engagierten technophilen Nutzergruppe hinaus im Wege. Die vorliegende Arbeit zeigt zwei parallel von einander (jedoch, im Sinne der Schnittstellenkompatibilität, nicht gänzlich unabhängig voneinander) zu betreibende Ansätze zur Integration von CAS Systemen. - Für den perioperativen Datenaustausch wird die Spezifikation zusätzlicher Datenstrukturen zum Transfer chirurgischer Planungsdaten im Rahmen des in radiologischen Bildverarbeitungssystemen weit verbreiteten DICOM Standards vorgeschlagen und an zwei Beispielen vorgeführt. Die Erweiterung des DICOM Standards für den perioperativen Einsatz ermöglicht hierbei die nahtlose Integration chirurgischer Planungssysteme in existierende "Picture Archiving and Communication Systems" (PACS), welche in den meisten Fällen auf dem DICOM Standard basieren oder zumindest damit kompatibel sind. Dadurch ist einerseits der Tatsache Rechnung getragen, dass die patientenspezifische OP-Planung in hohem Masse auf radiologischen Bildern basiert und andererseits sicher gestellt, dass die Planungsergebnisse entsprechend der geltenden Bestimmungen langfristig archiviert und gegen unbefugten Zugriff geschützt sind - PACS Server liefern hier bereits wohlerprobte Lösungen. - Für die integration intraoperativer CAS Systeme, wie etwa Navigationssysteme, Videobildquellen oder Sensoren zur Überwachung der Vitalparameter, wird das Konzept einer "chirurgischen Middleware" vorgestellt. Unter dem Namen TiCoLi wurde eine c++ Klassenbibliothek entwickelt, auf deren Grundlage die Konfiguration von ad-hoc Netzwerken während der OP-Vorbereitung mittels plug-and-play Mechanismen erleichtert wird. Nach erfolgter Konfiguration ermöglicht die TiCoLi den Austausch kontinuierlicher Datenströme sowie einzelner Datenpakete und Kommandos zwischen den Modulen einer verteilten CAS Anwendung durch ein Ethernet-basiertes Netzwerk. Die TiCoLi ist die erste frei verfügbare Klassenbibliothek welche diese Funktionalitäten dediziert für einen Einsatz im chirurgischen Umfeld vereinigt. Zum Nachweis der Tauglichkeit der gezeigten Spezifikationen und deren Implementierungen, werden zwei modulare CAS Anwendungen präsentiert, welche die vorgeschlagenen DICOM Erweiterungen zum perioperativen Austausch von Planungsergebnissen sowie die TiCoLi zum intraoperativen Datenaustausch von Messdaten unter echzeitnahen Anforderungen verwenden

Kontextsensitivität für den Operationssaal der Zukunft

The operating room of the future is a topic of high interest. In this thesis, which is among the first in the recently defined field of Surgical Data Science, three major topics for automated context awareness in the OR of the future will be examined: improved surgical workflow analysis, the newly developed event impact factors, and as application combining these and other concepts the unified surgical display.Der Operationssaal der Zukunft ist ein Forschungsfeld von großer Bedeutung. In dieser Dissertation, die eine der ersten im kürzlich definierten Bereich „Surgical Data Science“ ist, werden drei Themen für die automatisierte Kontextsensitivität im OP der Zukunft untersucht: verbesserte chirurgische Worflowanalyse, die neuentwickelten „Event Impact Factors“ und als Anwendungsfall, der diese Konzepte mit anderen kombiniert, das vereinheitlichte chirurgische Display

Kontextsensitivität für den Operationssaal der Zukunft

The operating room of the future is a topic of high interest. In this thesis, which is among the first in the recently defined field of Surgical Data Science, three major topics for automated context awareness in the OR of the future will be examined: improved surgical workflow analysis, the newly developed event impact factors, and as application combining these and other concepts the unified surgical display.Der Operationssaal der Zukunft ist ein Forschungsfeld von großer Bedeutung. In dieser Dissertation, die eine der ersten im kürzlich definierten Bereich „Surgical Data Science“ ist, werden drei Themen für die automatisierte Kontextsensitivität im OP der Zukunft untersucht: verbesserte chirurgische Worflowanalyse, die neuentwickelten „Event Impact Factors“ und als Anwendungsfall, der diese Konzepte mit anderen kombiniert, das vereinheitlichte chirurgische Display

Development of A Soft Robotic Approach for An Intra-abdominal Wireless Laparoscopic Camera

In Single-Incision Laparoscopic Surgery (SILS), the Magnetic Anchoring and Guidance System (MAGS) arises as a promising technique to provide larger workspaces and field of vision for the laparoscopes, relief space for other instruments, and require fewer incisions. Inspired by MAGS, many concept designs related to fully insertable magnetically driven laparoscopes are developed and tested on the transabdominal operation. However, ignoring the tissue interaction and insertion procedure, most of the designs adopt rigid structures, which not only damage the patients\u27 tissue with excess stress concentration and sliding motion but also require complicated operation for the insertion. Meanwhile, lacking state tracking of the insertable camera including pose and contact force, the camera systems operate in open-loop control. This provides mediocre locomotion precision and limited robustness to uncertainties in the environment. This dissertation proposes, develops, and validates a soft robotic approach for an intra-abdominal wireless laparoscopic camera. Contributions presented in this work include (1) feasibility of a soft intra-abdominal laparoscopic camera with friendly tissue interaction and convenient insertion, (2) six degrees of freedom (DOF) real-time localization, (3) Closed-loop control for a robotic-assisted laparoscopic system and (4) untethering solution for wireless communication and high-quality video transmission. Embedding magnet pairs into the camera and external actuator, the camera can be steered and anchored along the abdominal wall through transabdominal magnetic coupling. To avoid the tissue rapture by the sliding motion and dry friction, a wheel structure is applied to achieve rolling motion. Borrowing the ideas from soft robotic research, the main body of the camera implements silicone material, which grants it the bendability to passively attach along the curved abdominal wall and the deformability for easier insertion. The six-DOF pose is estimated in real-time with internal multi-sensor fusion and Newton-Raphson iteration. Combining the pose tracking and force-torque sensor measurement, an interaction model between the deformable camera and tissue is established to evaluate the interaction force over the tissue surface. Moreover, the proposed laparoscopic system is integrated with a multi-DOF manipulator into a robotic-assisted surgical system, where a closed-loop control is realized based on a feedback controller and online optimization. Finally, the wireless control and video streaming are accomplished with Bluetooth Low Energy (BLE) and Analog Video (AV) transmission. Experimental assessments have been implemented to evaluate the performance of the laparoscopic system

From Concept to Market: Surgical Robot Development

Surgical robotics and supporting technologies have really become a prime example of modern applied information technology infiltrating our everyday lives. The development of these systems spans across four decades, and only the last few years brought the market value and saw the rising customer base imagined already by the early developers. This chapter guides through the historical development of the most important systems, and provide references and lessons learnt for current engineers facing similar challenges. A special emphasis is put on system validation, assessment and clearance, as the most commonly cited barrier hindering the wider deployment of a system

Internet of robotic things : converging sensing/actuating, hypoconnectivity, artificial intelligence and IoT Platforms

The Internet of Things (IoT) concept is evolving rapidly and influencing newdevelopments in various application domains, such as the Internet of MobileThings (IoMT), Autonomous Internet of Things (A-IoT), Autonomous Systemof Things (ASoT), Internet of Autonomous Things (IoAT), Internetof Things Clouds (IoT-C) and the Internet of Robotic Things (IoRT) etc.that are progressing/advancing by using IoT technology. The IoT influencerepresents new development and deployment challenges in different areassuch as seamless platform integration, context based cognitive network integration,new mobile sensor/actuator network paradigms, things identification(addressing, naming in IoT) and dynamic things discoverability and manyothers. The IoRT represents new convergence challenges and their need to be addressed, in one side the programmability and the communication ofmultiple heterogeneous mobile/autonomous/robotic things for cooperating,their coordination, configuration, exchange of information, security, safetyand protection. Developments in IoT heterogeneous parallel processing/communication and dynamic systems based on parallelism and concurrencyrequire new ideas for integrating the intelligent “devices”, collaborativerobots (COBOTS), into IoT applications. Dynamic maintainability, selfhealing,self-repair of resources, changing resource state, (re-) configurationand context based IoT systems for service implementation and integrationwith IoT network service composition are of paramount importance whennew “cognitive devices” are becoming active participants in IoT applications.This chapter aims to be an overview of the IoRT concept, technologies,architectures and applications and to provide a comprehensive coverage offuture challenges, developments and applications

Surgical Data Science - from Concepts toward Clinical Translation

Recent developments in data science in general and machine learning in particular have transformed the way experts envision the future of surgery. Surgical Data Science (SDS) is a new research field that aims to improve the quality of interventional healthcare through the capture, organization, analysis and modeling of data. While an increasing number of data-driven approaches and clinical applications have been studied in the fields of radiological and clinical data science, translational success stories are still lacking in surgery. In this publication, we shed light on the underlying reasons and provide a roadmap for future advances in the field. Based on an international workshop involving leading researchers in the field of SDS, we review current practice, key achievements and initiatives as well as available standards and tools for a number of topics relevant to the field, namely (1) infrastructure for data acquisition, storage and access in the presence of regulatory constraints, (2) data annotation and sharing and (3) data analytics. We further complement this technical perspective with (4) a review of currently available SDS products and the translational progress from academia and (5) a roadmap for faster clinical translation and exploitation of the full potential of SDS, based on an international multi-round Delphi process

User-centered visual analysis using a hybrid reasoning architecture for intensive care units

One problem pertaining to Intensive Care Unit information systems is that, in some cases, a very dense display of data can result. To ensure the overview and readability of the increasing volumes of data, some special features are required (e.g., data prioritization, clustering, and selection mechanisms) with the application of analytical methods (e.g., temporal data abstraction, principal component analysis, and detection of events). This paper addresses the problem of improving the integration of the visual and analytical methods applied to medical monitoring systems. We present a knowledge- and machine learning-based approach to support the knowledge discovery process with appropriate analytical and visual methods. Its potential benefit to the development of user interfaces for intelligent monitors that can assist with the detection and explanation of new, potentially threatening medical events. The proposed hybrid reasoning architecture provides an interactive graphical user interface to adjust the parameters of the analytical methods based on the users' task at hand. The action sequences performed on the graphical user interface by the user are consolidated in a dynamic knowledge base with specific hybrid reasoning that integrates symbolic and connectionist approaches. These sequences of expert knowledge acquisition can be very efficient for making easier knowledge emergence during a similar experience and positively impact the monitoring of critical situations. The provided graphical user interface incorporating a user-centered visual analysis is exploited to facilitate the natural and effective representation of clinical information for patient care

When Internet of Things meets Metaverse: Convergence of Physical and Cyber Worlds

In recent years, the Internet of Things (IoT) is studied in the context of the Metaverse to provide users immersive cyber-virtual experiences in mixed reality environments. This survey introduces six typical IoT applications in the Metaverse, including collaborative healthcare, education, smart city, entertainment, real estate, and socialization. In the IoT-inspired Metaverse, we also comprehensively survey four pillar technologies that enable augmented reality (AR) and virtual reality (VR), namely, responsible artificial intelligence (AI), high-speed data communications, cost-effective mobile edge computing (MEC), and digital twins. According to the physical-world demands, we outline the current industrial efforts and seven key requirements for building the IoT-inspired Metaverse: immersion, variety, economy, civility, interactivity, authenticity, and independence. In addition, this survey describes the open issues in the IoT-inspired Metaverse, which need to be addressed to eventually achieve the convergence of physical and cyber worlds.info:eu-repo/semantics/publishedVersio