Doctor of Philosophy

dissertationIn this dissertation, we present methods for intuitive telemanipulation of manipulators that use piezoelectric stick-slip actuators (PSSAs). Commercial micro/nano-manipulators, which utilize PSSAs to achieve high precision over a large workspace, are typically controlled by a human operator at the joint level, leading to unintuitive and time-consuming telemanipulation. Prior work has considered the use of computer-vision-feedback to close a control loop for improved performance, but computer-vision-feedback is not a viable option for many end users. We discuss how open-loop models of the micro/nano-manipulator can be used to achieve desired end-effector movements, and we explain the process of obtaining open-loop models. We propose a rate-control telemanipulation method that utilizes the obtained model, and we experimentally quantify the effectiveness of the method using a common commercial manipulator (the Kleindiek MM3A). The utility of open-loop control methods for PSSAs with a human in the loop depends directly on the accuracy of the open-loop models of the manipulator. Prior research has shown that modeling of piezoelectric actuators is not a trivial task as they are known to suffer from nonlinearities that degrade their performance. We study the effect of static (non-inertial) loads on a prismatic and a rotary PSSA, and obtain a model relating the step size of the actuator to the load. The actuator-specific parameters of the model are calibrated by taking measurements in specific configurations of the manipulator. Results comparing the obtained model to experimental data are presented. PSSAs have properties that make them desirable over traditional DC-motor actuators for use in retinal surgery. We present a telemanipulation system for retinal surgery that uses a full range of existing disposable instruments. The system uses a PSSA-based manipulator that is compact and light enough that it could reasonably be made head-mounted to passively compensate for head movements. Two mechanisms are presented that enable the system to use existing disposable actuated instruments, and an instrument adapter enables quick-change of instruments during surgery. A custom stylus for a haptic interface enables intuitive and ergonomic telemanipulation of actuated instruments. Experimental results with a force-sensitive phantom eye show that telemanipulated surgery results in reduced forces on the retina compared to manual surgery, and training with the system results in improved performance. Finally, we evaluate operator efficiency with different haptic-interface kinematics for telemanipulated retinal surgery. Surgical procedures of the retina require precise manipulation of instruments inserted through trocars in the sclera. Telemanipulated robotic systems have been developed to improve retinal surgery, but there is not a unique mapping of the motions of the surgeon's hand to the lower-dimensional motions of the instrument through the trocar. We study operator performance during a precision positioning task on a force-sensing phantom retina, reminiscent of telemanipulated retinal surgery, with three common haptic-interface kinematics implemented in software on a PHANTOM Premium 6DOF haptic interface. Results from a study with 12 human subjects show that overall performance is best with the kinematics that represent a compact and inexpensive option, and that subjects' subjective preference agrees with the objective performance results

Haptics in Robot-Assisted Surgery: Challenges and Benefits

Robotic surgery is transforming the current surgical practice, not only by improving the conventional surgical methods but also by introducing innovative robot-enhanced approaches that broaden the capabilities of clinicians. Being mainly of man-machine collaborative type, surgical robots are seen as media that transfer pre- and intra-operative information to the operator and reproduce his/her motion, with appropriate filtering, scaling, or limitation, to physically interact with the patient. The field, however, is far from maturity and, more critically, is still a subject of controversy in medical communities. Limited or absent haptic feedback is reputed to be among reasons that impede further spread of surgical robots. In this paper objectives and challenges of deploying haptic technologies in surgical robotics is discussed and a systematic review is performed on works that have studied the effects of providing haptic information to the users in major branches of robotic surgery. It has been tried to encompass both classical works and the state of the art approaches, aiming at delivering a comprehensive and balanced survey both for researchers starting their work in this field and for the experts

Force-Sensing-Based Multi-Platform Robotic Assistance for Vitreoretinal Surgery

Vitreoretinal surgery aims to treat disorders of the retina, vitreous body, and macula, such as retinal detachment, diabetic retinopathy, macular hole, epiretinal membrane and retinal vein occlusion. Challenged by several technical and human limitations, vitreoretinal practice currently ranks amongst the most demanding fields in ophthalmic surgery. Of vitreoretinal procedures, membrane peeling is the most common to be performed, over 0.5 million times annually, and among the most prone to complications. It requires an extremely delicate tissue manipulation by various micron scale maneuvers near the retina despite the physiological hand tremor of the operator. In addition, to avoid injuries, the applied forces on the retina need to be kept at a very fine level, which is often well below the tactile sensory threshold of the surgeon. Retinal vein cannulation is another demanding procedure where therapeutic agents are injected into occluded retinal veins. The feasibility of this treatment is limited due to challenges in identifying the moment of venous puncture, achieving cannulation and maintaining it throughout the drug delivery period. Recent advancements in medical robotics have significant potential to address most of the challenges in vitreoretinal practice, and therefore to prevent traumas, lessen complications, minimize intra-operative surgeon effort, maximize surgeon comfort, and promote patient safety. This dissertation presents the development of novel force-sensing tools that can easily be used on various robotic platforms, and robot control methods to produce integrated assistive surgical systems that work in partnership with surgeons against the current limitations in vitreoretinal surgery, specifically focusing on membrane peeling and vein cannulation procedures. Integrating high sensitivity force sensing into the ophthalmic instruments enables precise quantitative monitoring of applied forces. Auditory feedback based upon the measured forces can inform (and warn) the surgeon quickly during the surgery and help prevent injury due to excessive forces. Using these tools on a robotic platform can attenuate hand tremor of the surgeon, which effectively promotes tool manipulation accuracy. In addition, based upon certain force signatures, the robotic system can precisely identify critical instants, such as the venous puncture in retinal vein cannulation, and actively guide the tool towards clinical targets, compensate any involuntary motion of the surgeon, or generate additional motion that will make the surgical task easier. The experimental results using two distinct robotic platforms, the Steady-Hand Eye Robot and Micron, in combination with the force-sensing ophthalmic instruments, show significant performance improvement in artificial dry phantoms and ex vivo biological tissues

Robotically assisted eye surgery : a haptic master console

Vitreo-retinal surgery encompasses the surgical procedures performed on the vitreous humor and the retina. A procedure typically consists of the removal of the vitreous humor, the peeling of a membrane and/or the repair of a retinal detachment. Operations are performed with needle shaped instruments which enter the eye through surgeon made scleral openings. An instrument is moved by hand in four degrees of freedom (three rotations and one translation) through this opening. Two rotations (? and ? ) are for a lateral instrument tip movement. The other two DoFs (z and ?) are the translation and rotation along the instrument axis. Actuation of for example a forceps can be considered as a fifth DoF. Characteristically, the manipulation of delicate, micrometer range thick intraocular tissue is required. Today, eye surgery is performed with a maximum of two instruments simultaneously. The surgeon relies on visual feedback only, since instrument forces are below the human detection limit. A microscope provides the visual feedback. It forces the surgeon to work in a static and non ergonomic body posture. Although the surgeon’s proficiency improves throughout his career, hand tremor may become a problem around his mid-fifties. Robotically assisted surgery with a master-slave system enhances dexterity. The slave with instrument manipulators is placed over the eye. The surgeon controls the instrument manipulators via haptic interfaces at the master. The master and slave are connected by electronic hardware and control software. Implementation of tremor filtering in the control software and downscaling of the hand motion allow prolongation of the surgeon’s career. Furthermore, it becomes possible to do tasks like intraocular cannulation which can not be done by manually performed surgery. This thesis focusses on the master console. Eye surgery procedures are observed in the operating room of different hospitals to gain insight in the requirements for the master. The master console as designed has an adjustable frame, a 3D display and two haptic interfaces with a coarse adjustment arm each. The console is mounted at the head of the operating table and is combined with the slave. It is compact, easy to place and allows the surgeon to have a direct view on and a physical contact with the patient. Furthermore, it fits in today’s manual surgery arrangement. Each haptic interface has the same five degrees of freedom as the instrument inside the eye. Through these interfaces, the surgeon can feel the augmented instrument forces. Downscaling of the hand motion results in a more accurate instrument movement compared to manually performed surgery. Together with the visual feedback, it is like the surgeon grasps the instrument near the tip inside the eye. The similarity between hand motion and motion of the instrument tip as seen on the display results in an intuitive manipulation. Pre-adjustment of the interface is done via the coarse adjustment arm. Mode switching enables to control three or more instruments manipulators with only two interfaces. Two one degree of freedom master-slave systems with force feedback are built to derive the requirements for the haptic interface. Hardware in the loop testing provides valuable insights and shows the possibility of force feedback without the use of force sensors. Two five DoF haptic interfaces are realized for bimanual operation. Each DoF has a position encoder and a force feedback motor. A correct representation of the upscaled instrument forces is only possible if the disturbance forces are low. Actuators are therefore mounted to the fixed world or in the neighborhood of the pivoting point for a low contribution to the inertia. The use of direct drive for ' and and low geared, backdriveable transmissions for the other three DoFs gives a minimum of friction. Disturbance forces are further minimized by a proper cable layout and actuator-amplifier combinations without torque ripple. The similarity in DoFs between vitreo-retinal eye surgery and minimally invasive surgery (MIS) enables the system to be used for MIS as well. Experiments in combination with a slave robot for laparoscopic and thoracoscopic surgery show that an instrument can be manipulated in a comfortable and intuitive way. User experience of surgeons and others is utilized to improve the haptic interface further. A parallel instead of a serial actuation concept for the ' and DoFs reduces the inertia, eliminates the flexible cable connection between frame and motor and allows that the heat of the motor is transferred directly to the frame. A newly designed z-?? module combines the actuation and suspension of the hand held part of the interface and has a three times larger z range than in the first design of the haptic interface

Vitreo-retinal eye surgery robot : sustainable precision

Vitreo-retinal eye surgery encompasses the surgical procedures performed on the vitreous humor and the retina. A procedure typically consists of the removal of the vitreous humor, the peeling of a membrane and/or the repair of a retinal detachment. Vitreo-retinal surgery is performed minimal invasively. Small needle shaped instruments are inserted into the eye. Instruments are manipulated by hand in four degrees of freedom about the insertion point. Two rotations move the instrument tip laterally, in addition to a translation in axial instrument direction and a rotation about its longitudinal axis. The manipulation of the instrument tip, e.g. a gripping motion can be considered as a fifth degree of freedom. While performing vitreo-retinal surgery manually, the surgeon faces various challenges. Typically, delicate micrometer range thick tissue is operated, for which steady hand movements and high accuracy instrument manipulation are required. Lateral instrument movements are inverted by the pivoting insertion point and scaled depending on the instrument insertion depth. A maximum of two instruments can be used simultaneously. There is nearly no perception of surgical forces, since most forces are below the human detection limit. Therefore, the surgeon relies only on visual feedback, obtained via a microscope or endoscope. Both vision systems force the surgeon to work in a static and non ergonomic body posture. Although the surgeon’s proficiency improves throughout his career, hand tremor will become a problem at higher age. Robotically assisted surgery with a master-slave system can assist the surgeon in these challenges. The slave system performs the actual surgery, by means of instrument manipulators which handle the instruments. The surgeon remains in control of the instruments by operating haptic interfaces via a master. Using electronic hardware and control software, the master and slave are connected. Amongst others, advantages as tremor filtering, up-scaled force feedback, down-scaled motions and stabilized instrument positioning will enhance dexterity on surgical tasks. Furthermore, providing the surgeon an ergonomic body posture will prolong the surgeon’s career. This thesis focuses on the design and realization of a high precision slave system for eye surgery. The master-slave system uses a table mounted design, where the system is compact, lightweight, easy to setup and equipped to perform a complete intervention. The slave system consists of two main parts: the instrument manipulators and their passive support system. Requirements are derived from manual eye surgery, conversations with medical specialists and analysis of the human anatomy and vitreo-retinal interventions. The passive support system provides a stiff connection between the instrument manipulator, patient and surgical table. Given the human anatomical diversity, presurgical adjustments can be made to allow the instrument manipulators to be positioned over each eye. Most of the support system is integrated within the patient’s headrest. On either the left or right side, two exchangeable manipulator-support arms can be installed onto the support system, depending on the eye being operated upon. The compact, lightweight and easy to install design, allows for a short setup time and quick removal in case of a complication. The slave system’s surgical reach is optimized to emulate manually performed surgery. For bimanual instrument operation, two instrument manipulators are used. Additional instrument manipulators can be used for non-active tools e.g. an illumination probe or an endoscope. An instrument manipulator allows the same degrees of freedom and a similar reach as manually performed surgery. Instrument forces are measured to supply force feedback to the surgeon via haptic interfaces. The instrument manipulator is designed for high stiffness, is play free and has low friction to allow tissue manipulation with high accuracy. Each instrument manipulator is equipped with an on board instrument change system, by which instruments can be changed in a fast and secure way. A compact design near the instrument allows easy access to the surgical area, leaving room for the microscope and peripheral equipment. The acceptance of a surgical robot for eye surgery mostly relies on equipment safety and reliability. The design of the slave system features various safety measures, e.g. a quick release mechanism for the instrument manipulator and additional locks on the pre-surgical adjustment fixation clamp. Additional safety measures are proposed, like a hard cover over the instrument manipulator and redundant control loops in the controlling FPGA. A method to fixate the patient’s head to the headrest by use of a custom shaped polymer mask is proposed. Two instrument manipulators and their passive support system have been realized so far, and the first experimental results confirm the designed low actuation torque and high precision performance

Augmentation Of Human Skill In Microsurgery

Surgeons performing highly skilled microsurgery tasks can benefit from information and manual assistance to overcome technological and physiological limitations to make surgery safer, efficient, and more successful. Vitreoretinal surgery is particularly difficult due to inherent micro-scale and fragility of human eye anatomy. Additionally, surgeons are challenged by physiological hand tremor, poor visualization, lack of force sensing, and significant cognitive load while executing high-risk procedures inside the eye, such as epiretinal membrane peeling. This dissertation presents the architecture and the design principles for a surgical augmentation environment which is used to develop innovative functionality to address the fundamental limitations in vitreoretinal surgery. It is an inherently information driven modular system incorporating robotics, sensors, and multimedia components. The integrated nature of the system is leveraged to create intuitive and relevant human-machine interfaces and generate a particular system behavior to provide active physical assistance and present relevant sensory information to the surgeon. These include basic manipulation assistance, audio-visual and haptic feedback, intraoperative imaging and force sensing. The resulting functionality, and the proposed architecture and design methods generalize to other microsurgical procedures. The system's performance is demonstrated and evaluated using phantoms and in vivo experiments

The Hand-Held Force Magnifier: Surgical Tools to Augment the Sense of Touch

Modern surgeons routinely perform procedures with noisy, sub-threshold, or obscured visual and haptic feedback,either due to the necessary approach, or because the systems on which they are operating are exceeding delicate. For example, in cataract extraction, ophthalmic surgeons must peel away thin membranes in order to access and replace the lens of the eye. Elsewhere, dissection is now commonly performed with energy-delivering tools – rather than sharp blades – and damage to deep structures is possible if tissue contact is not well controlled. Surgeons compensate for their lack of tactile sensibility by relying solely on visual feedback, observing tissue deformation and other visual cues through surgical microscopes or cameras. Using visual information alone can make a procedure more difficult, because cognitive mediation is required to convert visual feedback into motor action. We call this the “haptic problem” in surgery because the human sensorimotor loop is deprived of critical tactile afferent information, increasing the chance for intraoperative injury and requiring extensive training before clinicians reach independent proficiency. Tools that enhance the surgeon’s direct perception of tool-tissue forces can therefore potentially reduce the risk of iatrogenic complications and improve patient outcomes. Towards this end, we have developed and characterized a new robotic surgical tool, the Hand-Held Force Magnifier (HHFM), which amplifies forces at the tool tip so they may be readily perceived by the user, a paradigm we call “in-situ” force feedback. In this dissertation, we describe the development of successive generations of HHFM prototypes, and the evaluation of a proposed human-in-the-loop control framework using the methods of psychophysics. Using these techniques, we have verified that our tool can reduce sensory perception thresholds, augmenting the user’s abilities beyond what is normally possible. Further, we have created models of human motor control in surgically relevant tasks such as membrane puncture, which have shown to be sensitive to push-pull direction and handedness effects. Force augmentation has also demonstrated improvements to force control in isometric force generation tasks. Finally, in support of future psychophysics work, we have developed an inexpensive, high-bandwidth, single axis haptic renderer using a commercial audio speaker

Robocatch: Design and Making of a Hand-Held Spillage-Free Specimen Retrieval Robot for Laparoscopic Surgery

Specimen retrieval is an important step in laparoscopy, a minimally invasive surgical procedure performed to diagnose and treat a myriad of medical pathologies in fields ranging from gynecology to oncology. Specimen retrieval bags (SRBs) are used to facilitate this task, while minimizing contamination of neighboring tissues and port-sites in the abdominal cavity. This manual surgical procedure requires usage of multiple ports, creating a traffic of simultaneous operations of multiple instruments in a limited shared workspace. The skill-demanding nature of this procedure makes it time-consuming, leading to surgeons’ fatigue and operational inefficiency. This thesis presents the design and making of RoboCatch, a novel hand-held robot that aids a surgeon in performing spillage-free retrieval of operative specimens in laparoscopic surgery. The proposed design significantly modifies and extends conventional instruments that are currently used by surgeons for the retrieval task: The core instrumentation of RoboCatch comprises a webbed three-fingered grasper and atraumatic forceps that are concentrically situated in a folded configuration inside a trocar. The specimen retrieval task is achieved in six stages: 1) The trocar is introduced into the surgical site through an instrument port, 2) the three webbed fingers slide out of the tube and simultaneously unfold in an umbrella like-fashion, 3) the forceps slide toward, and grasp, the excised specimen, 4) the forceps retract the grasped specimen into the center of the surrounding grasper, 5) the grasper closes to achieve a secured containment of the specimen, and 6) the grasper, along with the contained specimen, is manually removed from the abdominal cavity. The resulting reduction in the number of active ports reduces obstruction of the port-site and increases the procedure’s efficiency. The design process was initiated by acquiring crucial parameters from surgeons and creating a design table, which informed the CAD modeling of the robot structure and selection of actuation units and fabrication material. The robot prototype was first examined in CAD simulation and then fabricated using an Objet30 Prime 3D printer. Physical validation experiments were conducted to verify the functionality of different mechanisms of the robot. Further, specimen retrieval experiments were conducted with porcine meat samples to test the feasibility of the proposed design. Experimental results revealed that the robot was capable of retrieving masses of specimen ranging from 1 gram to 50 grams. The making of RoboCatch represents a significant step toward advancing the frontiers of hand-held robots for performing specimen retrieval tasks in minimally invasive surgery

Die empirische selektive Drahtsondierung und ihre visuelle Darstellung als Grundlage für autonom robotisch gesteuerte Gefäßsondierungen

Einführung: Gegenstand dieser Dissertationsarbeit ist die empirische Gefäßsondierung: die Kombination von 2 Bewegungsachsen– Pendelbewegung und Rotation- macht sie wiederholbar und objektivierbar. Sie kann -per Hand durchgeführt- als Alternative zur herkömmlichen Sondierungstechnik dienen, aber auch automatisiert durch eine technische Vorrichtung durchgeführt werden. Somit kann sie als Grundlage für das erste voll automatische robotische Systems für endovaskuläre Interventionen dienen. Material und Methode: Zunächst wurden 40 Durchleuchtungsserien aus Interventionen von PatientInnen nachbearbeitet, die während einer empirischen Drahtsondierung aufgezeichnet worden waren. Die Interventionen (Uterusmyomembolisationen, Interventionen im Bereich der Niere, der Leber und des Beins) wurden auf die in der Abteilung übliche Art und Weise durchgeführt und in keiner Weise beeinflusst. Zur Verwendung kamen u.a. die Software Syngo iFlow der Firma Siemens, durch die alle Bilder einer Durchleuchtungsserie in einem Bild zusammengefasst werden konnten. Es wurden Gefäßbaum-Modelle aus Acrylglas hergestellt und per Hand in empirischer Technik sondiert. Im nächsten Schritt wurde eine maschinelle Vorrichtung hergestellt, die in der Lage ist, die empirische Drahtsondierung durchzuführen. Während der Sondierungen der Modelle per Hand und durch die maschinelle Vorrichtung wurden Durchleuchtungsserien gewonnen und mit o.g. Software nachbearbeitet. Die Erhebung der Daten und Durchführung der Versuche erfolgte im Zeitraum von Februar 2013 bis Mai 2015. Ergebnisse: Bei der händischen empirischen Sondierung in vitro zeigte sich, dass mit der richtigen Frequenz bei Rotations- und Pendelbewegung alle Abzweigungen mit einem Abzweigungswinkel von > 90 Grad gut erreicht wurden. Bei der Nachbearbeitung der Durchleuchtungsserien von PatientInnen, bei denen die Drahtsondierung empirisch durchgeführt worden war, imponierten im Vergleich zur Sondierung der Gefäßbaummodelle insgesamt mehr Abzweigungen und einer größere zurückgelegte Strecke. Eine weitere Bewegungskomponente, nämlich das Vorschieben des Drahtes, sobald auf dem Weg die gewünschte Abzweigung erreicht war, kam zum Einsatz. Durch Nachbearbeitung mit o.g. Software entstand ein vergleichbares zweidimensionales Bild des Gefäßbaumabschnitts. Bei der Sondierung durch die selbst hergestellte maschinelle Vorrichtung gelangte das Drahtende in alle Abzweigungen mit Abzweigungswinkel >90 Grad, nachdem im Vorhinein die Frequenz der Rotation und der Pendelbewegung angepasst worden waren. Gegenüber der Sondierung per Hand imponierte eine größere Regelmäßigkeit der Bewegung, die Versuche waren wiederholbar. Die Grenzen der Anwendung stellt eine besonders komplexe Anatomie dar. Diskussion: Bei der empirische Gefäßsondierung per Hand spielen im Vergleich zur herkömmlichen zielgerichteten Drahtsondierung Erfahrung, räumliches Vorstellungsvermögen und Geschicklichkeit eine weniger wichtige Rolle. Die maschinelle empirische Drahtsondierung ist objektivierbar, denn die einzelnen Komponenten stehen fest. Sie kann als Grundlage für den Entwurf eines autonom gesteuerten robotischen Systems zur Gefäßsondierung dienen. Dafür müsste der Draht bei Erreichen der richtigen Abzweigung gestoppt und bis zur nächsten relevanten Abzweigung vorgeschoben werden. Die jeweiligen Abzweigungen müssten wie bei einem handelsüblichen Navigationssystem vorher festgelegt werden.Introduction: The subject of this dissertation is empirical vascular probing: By combining 2 axes of motion- pendulum motion and rotation- it is repeatable and objectifiable. Performed per hand it can serve as an alternative to the conventional probing technique. Automated it can be carried out by a technical device. Thus, it can serve as the basis for the first fully automated robotic system for endovascular interventions. Material and Methods: First, 40 fluoroscopy series from interventions of patients were post-processed, which were recorded during empirical wire probing. The interventions (uterine myoma embolizations, interventions in the kidney, liver and leg) were performed in the departments usual way and were not influenced in any way. The software used was e.g. Syngo iFlow from Siemens, through which all images of a fluoroscopic series could be combined into one image. Vascular tree models were made of acrylic glass and probed by hand using the empirical technique. In the next step, a mechanical device was manufactured, which is able to perform the empirical wire probing. During the probing of the models by hand and by the mechanical device, fluoroscopy series were obtained and post-processed with the above mentioned software. The data collection and execution of the experiments took place from February 2013 to May 2015. Results: Manual empirical probing in vitro showed that with the correct frequency in rotational and pendulum motion, all branches with a branch angle of > 90 degrees were reached. In the post-processing of the fluoroscopy series of patients in whom wire probing had been performed empirically, more bifurcations and a greater distance probed overall were apparent in comparison to the probing of the acrylic glass models. Another movement component, the advancing oft the probing wire as soon as the desired branch was reached, was apparent. Post-processing with the aforementioned software produced a comparable two-dimensional image of the vascular tree. During probing by the self-made mechanical device, the wire end reached all branches with branch angles >90 degrees, after the frequency of rotation and pendulum movement had been adjusted in advance. Compared to probing by hand, a greater regularity of the movement was impressive. The experiments were repeatable. Limitations of the application are a particularly complex anatomy. Discussion: In empirical vascular probing by hand experience and manual dexterity are much less imperative than in conventional targeted wire probing. Empirical wire probing by a mechanical device can be objectified as the individual components are definite. It can serve as a basis for designing an autonomously controlled robotic system for vascular probing. For this, the wire would have to be stopped when the correct vascular branch is reached and be moved to the next relevant branch. The respective branches would have to be determined in advance similar to commercial navigation system