Vitreoretinal Surgical Robotic System with Autonomous Orbital Manipulation using Vector-Field Inequalities

Vitreoretinal surgery pertains to the treatment of delicate tissues on the fundus of the eye using thin instruments. Surgeons frequently rotate the eye during surgery, which is called orbital manipulation, to observe regions around the fundus without moving the patient. In this paper, we propose the autonomous orbital manipulation of the eye in robot-assisted vitreoretinal surgery with our tele-operated surgical system. In a simulation study, we preliminarily investigated the increase in the manipulability of our system using orbital manipulation. Furthermore, we demonstrated the feasibility of our method in experiments with a physical robot and a realistic eye model, showing an increase in the view-able area of the fundus when compared to a conventional technique. Source code and minimal example available at https://github.com/mmmarinho/icra2023_orbitalmanipulation.Comment: 7 pages, 7 figures, accepted on ICRA202

Robot Control for Remote Ophthalmology and Pediatric Physical Rehabilitation

The development of a robotic slit-lamp for remote ophthalmology is the primary purpose of this work. In addition to novel mechanical designs and implementation, it was also a goal to develop a control system that was flexible enough to be adapted with minimal user adjustment to various styles and configurations of slit-lamps. The system was developed with intentions of commercialization, so common hardware was used for all components to minimize the costs. In order to improve performance using this low-cost hardware, investigations were made to attempt to achieve better performance by applying control theory algorithms in the system software. Ultimately, the controller was to be flexible enough to be applied to other areas of human-robot interaction including pediatric rehabilitation via the use of humanoid robotic aids. This application especially requires a robust controller to facilitate safe interaction. Though all of the prototypes were successfully developed and made to work sufficiently with the control hardware, the application of advanced control did not yield notable gains as was hoped. Further investigations were made attempting to alter the performance of the control system, but the components selected did not have the physical capabilities for improved response above the original software implemented. Despite this disappointment, numerous novel advances were made in the area of teleoperated ophthalmic technology and pediatric physical rehabilitation tools. This includes a system that is used to remote control a slit-lamp and lens for examinations and some laser procedures. Secondly, a series of of humanoid systems suitable for both medical research and therapeutic modeling were developed. This included a robotic face used as an interactive system for ophthalmic testing and training. It can also be used as one component in an interactive humanoid robotic system that includes hands and arms to allow use of teaching sign language, social skills or modeling occupational therapy tasks. Finally, a humanoid system is presented that can serve as a customized surrogate between a therapist and client to model physical therapy tasks in a realistic manner. These systems are all functional, safe and low-cost to allow for feasible implementation with patients in the near future

Non linear force feedback enhancement for cooperative robotic neurosurgery enforces virtual boundaries on cortex surface

Surgeons can benefit from the cooperation with a robotic assistant during the repetitive execution of precise targeting tasks on soft tissues, such as brain cortex stimulation procedures in open-skull neurosurgery. Position-based force-to-motion control schemes may not be satisfactory solution to provide the manipulator with the high compliance desirable during guidance along wide trajectories. A new torque controller with non-linear force feedback enhancement (FFE) is presented to provide augmented haptic perception to the operator from instrument-tissue interaction. Simulation tests were performed to evaluate the system stability according to different non-linear force modulation functions (power, sigmoidal and arc tangent). The FFE controller with power modulation was experimentally validated with a pool of non-expert users using brain-mimicking gelatin phantoms (8%-16% concentration). Besides providing hand tremor rejection for a stable holding of the tool, the FFE controller was proven to allow for a safer tissue contact with respect to both robotic assistance without force feedback and freehand executions (50% and 75% reduction of the indentation depth, respectively). Future work will address the evaluation of the safety features of the FFE controller with expert surgeons on a realistic brain phantom, also accounting for unpredictable tissue's motions as during seizures due to cortex stimulation

Doctor of Philosophy

dissertationIn this dissertation, we present methods for intuitive telemanipulation of manipulators that use piezoelectric stick-slip actuators (PSSAs). Commercial micro/nano-manipulators, which utilize PSSAs to achieve high precision over a large workspace, are typically controlled by a human operator at the joint level, leading to unintuitive and time-consuming telemanipulation. Prior work has considered the use of computer-vision-feedback to close a control loop for improved performance, but computer-vision-feedback is not a viable option for many end users. We discuss how open-loop models of the micro/nano-manipulator can be used to achieve desired end-effector movements, and we explain the process of obtaining open-loop models. We propose a rate-control telemanipulation method that utilizes the obtained model, and we experimentally quantify the effectiveness of the method using a common commercial manipulator (the Kleindiek MM3A). The utility of open-loop control methods for PSSAs with a human in the loop depends directly on the accuracy of the open-loop models of the manipulator. Prior research has shown that modeling of piezoelectric actuators is not a trivial task as they are known to suffer from nonlinearities that degrade their performance. We study the effect of static (non-inertial) loads on a prismatic and a rotary PSSA, and obtain a model relating the step size of the actuator to the load. The actuator-specific parameters of the model are calibrated by taking measurements in specific configurations of the manipulator. Results comparing the obtained model to experimental data are presented. PSSAs have properties that make them desirable over traditional DC-motor actuators for use in retinal surgery. We present a telemanipulation system for retinal surgery that uses a full range of existing disposable instruments. The system uses a PSSA-based manipulator that is compact and light enough that it could reasonably be made head-mounted to passively compensate for head movements. Two mechanisms are presented that enable the system to use existing disposable actuated instruments, and an instrument adapter enables quick-change of instruments during surgery. A custom stylus for a haptic interface enables intuitive and ergonomic telemanipulation of actuated instruments. Experimental results with a force-sensitive phantom eye show that telemanipulated surgery results in reduced forces on the retina compared to manual surgery, and training with the system results in improved performance. Finally, we evaluate operator efficiency with different haptic-interface kinematics for telemanipulated retinal surgery. Surgical procedures of the retina require precise manipulation of instruments inserted through trocars in the sclera. Telemanipulated robotic systems have been developed to improve retinal surgery, but there is not a unique mapping of the motions of the surgeon's hand to the lower-dimensional motions of the instrument through the trocar. We study operator performance during a precision positioning task on a force-sensing phantom retina, reminiscent of telemanipulated retinal surgery, with three common haptic-interface kinematics implemented in software on a PHANTOM Premium 6DOF haptic interface. Results from a study with 12 human subjects show that overall performance is best with the kinematics that represent a compact and inexpensive option, and that subjects' subjective preference agrees with the objective performance results

Design, Manufacturing and Control of an Advanced High-Precision Robotic System for Microsurgery

Microsurgeries like ophthalmic surgery confront many challenges like limited workspace and hand motion, steady hand movements, manipulating delicate thin tissues, and holding the instrument in place for a long time. New developments in robotically-assisted surgery can highly benefits this field and facilitate those complicated surgeries. Robotic eye surgery can save time, reduce surgical complications and inspire more delicate surgical procedures that cannot be done currently by surgeon’s hands. In this thesis work, the requirements for ophthalmic surgeries were studied and based on that a robotic system with 6 DOF is proposed and designed. This robotic system is capable of handling the position and orientation of the surgical instrument with theoretical accuracy of 10 μm. The design features a remote center of motion that defines the point of entry into the eye or patient’s body. The forward and inverse kinematics equations and workspace analysis of the robot is also discussed and presented. Six miniature DC motors with their PID controllers were installed on robot arms in order to run 6 DOF systems. Therefore, the dynamic behavior of a DC motor was studied and modeled and then the position and velocity transfer functions were derived and used to study the behavior of the system and also to manually tune the PID controller. The function of different elements of the control system including encoder, controller modules, Controller Area Network (CAN) and the controller software were discussed as well. The graphical user interface called EPOS Studio and performs as the motion controller is introduced and the way it organizes communications among the elements of the control system was described

A high-resolution and low-cost mesoscale tactile force sensor based on mode-localization effect and fabricated using rapid prototyping

This paper presents a novel design of a high resolution and low-cost tactile force sensor, based on the concept of mode-localization in two weakly coupled resonators (WCRs). The sensor is fabricated at mesoscale by utilizing rapid prototyping techniques. The two WCRs in the sensor are operated at resonance by using an electrostatic ac-tuation. Change in the oscillation amplitude ratios and resonant frequency shift, corresponding to an input force is utilized as an output metric for the measurement of force. The application of an applied force on the WCRs in-duced electrostatic strain, which acted as a negative stiffness perturbation. The outer body of sensor is manufac-tured using a soft silicone elastomer and shaped using molds based on laser cutting technique. The proposed tac-tile force sensor is analyzed numerically through finite-element-method (FEM) based simulations. For the testing of tactile force sensor, an actuation and sensing electronics scheme is developed. The experimental results re-vealed that the sensor is capable of measuring input force up to 20 mN with a relative amplitude ratio (AR) and resonant frequency shift based sensitivity of 27040 ppm/mN and 3553 ppm/mN respectively. The experimen-tally evaluated resolution for the sensor is 7.3 µN. The sensor shows the stability in response to the thermal varia-tions and low-frequency vibrational environments

Robotic Manipulator for Positioning a Magnetic Actuator Dedicated to Drug Delivery in the Cochlea

International audienceThe actuators dedicated to micrometric applications are known for their precision but also for their restricted workspace. The use of a robotic manipulator as a carrier makes it possible to considerably increase this workspace. In this paper, we present a novel robotic system specially designed for positioning a magnetic actuator based on permanent magnets, used as an end-effector of the robot for steering magnetic microrobot throughout the cochlea. Using the classical mathematical tools of serial robotics, we determined the direct and inverse kinematic models of the manipulator, thus defining a reference trajectory to move the microrobot on a space as small as possible and take account of the geometrical specifications based on medical needs. A prototype has been realized with a 3D printer to experimentally validate the numerical results. In addition, the mechanical considerations for the construction of the prototype are presented

Control and Estimation Methods Towards Safe Robot-assisted Eye Surgery

Vitreoretinal surgery is among the most delicate surgical tasks in which physiological hand tremor may severely diminish surgeon performance and put the eye at high risk of injury. Unerring targeting accuracy is required to perform precise operations on micro-scale tissues. Tool tip to tissue interaction forces are usually below human tactile perception, which may result in exertion of excessive forces to the retinal tissue leading to irreversible damages. Notable challenges during retinal surgery lend themselves to robotic assistance which has proven beneficial in providing a safe steady-hand manipulation. Efficient assistance from the robots heavily relies on accurate sensing and intelligent control algorithms of important surgery states and situations (e.g. instrument tip position measurements and control of interaction forces). This dissertation provides novel control and state estimation methods to improve safety during robot-assisted eye surgery. The integration of robotics into retinal microsurgery leads to a reduction in surgeon perception of tool-to-tissue forces at sclera. This blunting of human tactile sensory input, which is due to the inflexible inertia of the robot, is a potential iatrogenic risk during robotic eye surgery. To address this issue, a sensorized surgical instrument equipped with Fiber Bragg Grating (FBG) sensors, which is capable of measuring the sclera forces and instrument insertion depth into the eye, is integrated to the Steady-Hand Eye Robot (SHER). An adaptive control scheme is then customized and implemented on the robot that is intended to autonomously mitigate the risk of unsafe scleral forces and excessive insertion of the instrument. Various preliminary and multi-user clinician studies are then conducted to evaluate the effectiveness of the control method during mock retinal surgery procedures. In addition, due to inherent flexibility and the resulting deflection of eye surgical instruments as well as the need for targeting accuracy, we have developed a method to enhance deflected instrument tip position estimation. Using an iterative method and microscope data, we develop a calibration- and registration-independent (RI) framework to provide online estimates of the instrument stiffness (least squares and adaptive). The estimations are then combined with a state-space model for tip position evolution obtained based on the forward kinematics (FWK) of the robot and FBG sensor measurements. This is accomplished using a Kalman Filtering (KF) approach to improve the instrument tip position estimation during robotic surgery. The entire framework is independent of camera-to-robot coordinate frame registration and is evaluated during various phantom experiments to demonstrate its effectiveness

Dynamic Active Constraints for Surgical Robots using Vector Field Inequalities

Robotic assistance allows surgeons to perform dexterous and tremor-free procedures, but robotic aid is still underrepresented in procedures with constrained workspaces, such as deep brain neurosurgery and endonasal surgery. In these procedures, surgeons have restricted vision to areas near the surgical tooltips, which increases the risk of unexpected collisions between the shafts of the instruments and their surroundings. In this work, our vector-field-inequalities method is extended to provide dynamic active-constraints to any number of robots and moving objects sharing the same workspace. The method is evaluated with experiments and simulations in which robot tools have to avoid collisions autonomously and in real-time, in a constrained endonasal surgical environment. Simulations show that with our method the combined trajectory error of two robotic systems is optimal. Experiments using a real robotic system show that the method can autonomously prevent collisions between the moving robots themselves and between the robots and the environment. Moreover, the framework is also successfully verified under teleoperation with tool-tissue interactions.Comment: Accepted on T-RO 2019, 19 Page