Modular instrument for a haptically-enabled robotic surgical system (HeroSurg)

To restore the sense of touch in robotic surgical systems, a modular force feedback-enabled laparoscopic instrument is developed and employed in a robotic-assisted minimally invasive surgical system (HeroSurg). Strain gauge technology is incorporated into the instrument to measure tip/tissue lateral interaction forces. The modularity feature of the proposed instrument makes it interchangeable between various tip types of different functionalities, e.g., cutter, grasper, and dissector, without losing force sensing capability. Series of experiments are conducted and results are reported to evaluate force sensing capability of the instrument. The results reveal mean errors of 1.32 g and 1.98° in the measurements of tip/tissue load magnitude and direction across all experiments, respectively

Force Feedback for the Patient Side Manipulator of the daVinci Research Kit

Teleoperated robotic surgical systems such as daVinci are widely used for laparoscopic surgeries. The currently available daVinci system does not provide haptic feedback. Prior research has shown that the addition of haptic feedback improves surgeons\u27 performance during minimally invasive surgeries. Other authors have implemented haptic feedback in the daVinci robot, but some issues made them impractical for clinical use. In this work, two strain gauge force-sensing devices were created for the patient side manipulator of the daVinci surgical robot. These devices were designed to be easily added to the existing system and used for the real-time force feedback in the daVinci robot. The proposed system has high sensitivity and resolution, matches the required force measurement range, and has high signal-to-noise ratio, which implies high signal quality. However, the absolute errors of the currently built devices are high due to the manufacturing techniques used on the prototype that could be improved upon for a deployed device. This work demonstrates fast 3-DOF force measurements on the daVinci robot without any robot or instrument modifications. While the present system has significant systematic errors, these can be mitigated by altering the mechanical design to improve the accuracy of the system

Robotic manipulators for single access surgery

This thesis explores the development of cooperative robotic manipulators for enhancing surgical precision and patient outcomes in single-access surgery and, specifically, Transanal Endoscopic Microsurgery (TEM). During these procedures, surgeons manipulate a heavy set of instruments via a mechanical clamp inserted in the patient’s body through a surgical port, resulting in imprecise movements, increased patient risks, and increased operating time. Therefore, an articulated robotic manipulator with passive joints is initially introduced, featuring built-in position and force sensors in each joint and electronic joint brakes for instant lock/release capability. The articulated manipulator concept is further improved with motorised joints, evolving into an active tool holder. The joints allow the incorporation of advanced robotic capabilities such as ultra-lightweight gravity compensation and hands-on kinematic reconfiguration, which can optimise the placement of the tool holder in the operating theatre. Due to the enhanced sensing capabilities, the application of the active robotic manipulator was further explored in conjunction with advanced image guidance approaches such as endomicroscopy. Recent advances in probe-based optical imaging such as confocal endomicroscopy is making inroads in clinical uses. However, the challenging manipulation of imaging probes hinders their practical adoption. Therefore, a combination of the fully cooperative robotic manipulator with a high-speed scanning endomicroscopy instrument is presented, simplifying the incorporation of optical biopsy techniques in routine surgical workflows. Finally, another embodiment of a cooperative robotic manipulator is presented as an input interface to control a highly-articulated robotic instrument for TEM. This master-slave interface alleviates the drawbacks of traditional master-slave devices, e.g., using clutching mechanics to compensate for the mismatch between slave and master workspaces, and the lack of intuitive manipulation feedback, e.g. joint limits, to the user. To address those drawbacks a joint-space robotic manipulator is proposed emulating the kinematic structure of the flexible robotic instrument under control.Open Acces

Doctor of Philosophy

dissertationIn this dissertation, we present methods for intuitive telemanipulation of manipulators that use piezoelectric stick-slip actuators (PSSAs). Commercial micro/nano-manipulators, which utilize PSSAs to achieve high precision over a large workspace, are typically controlled by a human operator at the joint level, leading to unintuitive and time-consuming telemanipulation. Prior work has considered the use of computer-vision-feedback to close a control loop for improved performance, but computer-vision-feedback is not a viable option for many end users. We discuss how open-loop models of the micro/nano-manipulator can be used to achieve desired end-effector movements, and we explain the process of obtaining open-loop models. We propose a rate-control telemanipulation method that utilizes the obtained model, and we experimentally quantify the effectiveness of the method using a common commercial manipulator (the Kleindiek MM3A). The utility of open-loop control methods for PSSAs with a human in the loop depends directly on the accuracy of the open-loop models of the manipulator. Prior research has shown that modeling of piezoelectric actuators is not a trivial task as they are known to suffer from nonlinearities that degrade their performance. We study the effect of static (non-inertial) loads on a prismatic and a rotary PSSA, and obtain a model relating the step size of the actuator to the load. The actuator-specific parameters of the model are calibrated by taking measurements in specific configurations of the manipulator. Results comparing the obtained model to experimental data are presented. PSSAs have properties that make them desirable over traditional DC-motor actuators for use in retinal surgery. We present a telemanipulation system for retinal surgery that uses a full range of existing disposable instruments. The system uses a PSSA-based manipulator that is compact and light enough that it could reasonably be made head-mounted to passively compensate for head movements. Two mechanisms are presented that enable the system to use existing disposable actuated instruments, and an instrument adapter enables quick-change of instruments during surgery. A custom stylus for a haptic interface enables intuitive and ergonomic telemanipulation of actuated instruments. Experimental results with a force-sensitive phantom eye show that telemanipulated surgery results in reduced forces on the retina compared to manual surgery, and training with the system results in improved performance. Finally, we evaluate operator efficiency with different haptic-interface kinematics for telemanipulated retinal surgery. Surgical procedures of the retina require precise manipulation of instruments inserted through trocars in the sclera. Telemanipulated robotic systems have been developed to improve retinal surgery, but there is not a unique mapping of the motions of the surgeon's hand to the lower-dimensional motions of the instrument through the trocar. We study operator performance during a precision positioning task on a force-sensing phantom retina, reminiscent of telemanipulated retinal surgery, with three common haptic-interface kinematics implemented in software on a PHANTOM Premium 6DOF haptic interface. Results from a study with 12 human subjects show that overall performance is best with the kinematics that represent a compact and inexpensive option, and that subjects' subjective preference agrees with the objective performance results

Kinesthetic Haptics Sensing and Discovery with Bilateral Teleoperation Systems

In the mechanical engineering field of robotics, bilateral teleoperation is a classic but still increasing research topic. In bilateral teleoperation, a human operator moves the master manipulator, and a slave manipulator is controlled to follow the motion of the master in a remote, potentially hostile environment. This dissertation focuses on kinesthetic perception analysis in teleoperation systems. Design of the controllers of the systems is studied as the influential factor of this issue. The controllers that can provide different force tracking capability are compared using the same experimental protocol. A 6 DOF teleoperation system is configured as the system testbed. An innovative master manipulator is developed and a 7 DOF redundant manipulator is used as the slave robot. A singularity avoidance inverse kinematics algorithm is developed to resolve the redundancy of the slave manipulator. An experimental protocol is addressed and three dynamics attributes related to kineshtetic feedback are investigated: weight, center of gravity and inertia. The results support our hypothesis: the controller that can bring a better force feedback can improve the performance in the experiments

Force Sensing Surgical Scissor Blades using Fibre Bragg Grating Sensors

This thesis considers the development and analysis of unique sensorised surgical scissor blades for application in minimally invasive robotic surgery (MIRS). The lack of haptic (force and tactile) feedback to the user is currently an unresolved issue with modern MIRS systems. This thesis presents details on smart sensing scissor blades which enable the measurement of instrument-tissue interaction forces for the purpose of force reflection and tissue property identification. A review of current literature established that there exists a need for small compact, biocompatible, sterilisable and robust sensors which meet the demands of current MIRS instruments. Therefore, the sensorised blades exploit the strain sensing capabilities of a single fibre Bragg grating (FBG) sensor bonded to their surface. The nature and magnitude of the strain likely to be experienced by the blades, and consequently the FBG sensor, while cutting soft tissue samples were characterised through the use of an application specific test-bed. Using the sensorised blades to estimate fracture properties is proposed, hence two methods of extracting fracture toughness information from the test samples are assessed and compared. Investigations were carried out on the factors affecting the transfer of strain from the blade material to the core of the FBG sensor for surface mounted or partially embedded arrangements. Results show that adhesive bond length, thickness and stiffness need to be carefully specified when bonding FBG sensors to ensure effective strain transfer. Calibration and dynamic cutting experiments were carried out using the characterisation test-bed. The complex nature of the blade interaction forces were modelled, primarily for the purpose of decoupling the direct, lateral, friction and fracture strains experienced by the bonded FBG sensor during cutting. The modelled and experimental results show that the approach taken in sensorising the blade enables detailed cutting force data to be obtained and consequently leads to a unique method in estimating the kinetic friction coefficient for the blades. The forces measured using the FBG are validated against a commercial load cell used in the test-bed. This research work demonstrates that this unique approach of placing a single optical fibre onto the scissor blades can, in an unobtrusive manner, measure interblade friction forces and material fracture properties occurring at the blade-tissue interface

Robotic Ultrasound Tomography and Collaborative Control

Ultrasound computed tomography (USCT) offers quantitative anatomical tissue characterization for cancer detection, and has shown similar diagnostic power to MRI on ex vivo prostate tissue. While most USCT research and commercial development has focused on submerging target anatomy in a transducer-lined cylindrical water-tank, this approach is not practical for imaging deep anatomy like the prostate and an alternative acquisition system using aligned abdominal and endolumenal ultrasound probes is required. This work outlines a clinical workflow, calibration scheme, and motion framework for an innovative dual-robotic USCT acquisition system specific to in vivo prostate imaging – one arm wielding a linear abdominal probe, the other wielding a linear transrectal ultrasound (TRUS) probe. After a three-way calibration, the robotic system works to autonomously keep the abdominal probe collinear with the physician-rotated TRUS probe using a hybrid force-position convex contour tracking scheme, while impedance control enforces its gentle contact with the patient’s pubic region for capturing the transmission ultrasound slices needed for limited-angle tomographic reconstruction. TRUS rotation was induced by joystick control for precision during testing, however collaborative control via admittance control of hand forces presents a useful workflow option to the physician. An improved robot admittance control algorithm for transparent collaborative control utilizing Kalman filtering was developed and verified to smooth robot hand guidance. Such an improvement additionally has important implications for generally alleviating ultrasonographer musculoskeletal strain through cooperatively controlled robots. The ultimate dual-robotic USCT system proved repeatable and sufficiently accurate for tomography based on pelvic phantom testing. Future steps in system verification and validation are discussed, as is incorporation into feasibility studies to test the potential and utility of the system for future prostate malignancy diagnosis and staging in vivo

Sensorless Physical Human-robot Interaction Using Deep-Learning

Physical human-robot interaction has been an area of interest for decades. Collaborative tasks, such as joint compliance, demand high-quality joint torque sensing. While external torque sensors are reliable, they come with the drawbacks of being expensive and vulnerable to impacts. To address these issues, studies have been conducted to estimate external torques using only internal signals, such as joint states and current measurements. However, insufficient attention has been given to friction hysteresis approximation, which is crucial for tasks involving extensive dynamic to static state transitions. In this paper, we propose a deep-learning-based method that leverages a novel long-term memory scheme to achieve dynamics identification, accurately approximating the static hysteresis. We also introduce modifications to the well-known Residual Learning architecture, retaining high accuracy while reducing inference time. The robustness of the proposed method is illustrated through a joint compliance and task compliance experiment.Comment: 7 pages, ICRA 2024 Submissio

Evaluation of an Actuated Wrist Orthosis for Use in Assistive Upper Extremity Rehabilitation

Cerebral palsy (CP) is a neurological condition caused by damage to motor control centers of the brain. This leads to physical and cognitive deficiencies that can reduce an individual’s quality of life. Specifically, motor deficiencies of the upper extremity can make it difficult for an individual to complete everyday tasks, including eating, drinking, getting dressed, or combing their hair. Physical therapy, involving repetitive tasks, has been shown to be effective in training normal motion of the limb by invoking the neuroplasticity of the brain and its ability to adapt in order to facilitate motor learning. Creating a device for use with Activities of Daily Living (ADLs) provides an additional tool for task-based therapy with the goal of improving functional outcome. A custom wrist orthotic has been designed and developed that assists flexion/extension of the wrist and rotation of the forearm, while leaving the hand open for the grasp and manipulation of objects. Actuated joints are driven with geared brushless DC motors on a lightweight, exoskeleton frame coupled to a passive arm that tracks positional changes within the task space. Control of actuation is accomplished with a custom mapping strategy, created from nominal movement profiles for 5 ADLs collected from healthy subjects. A simple relationship was created between position within the workspace and orientation necessary for task completion to determine needed assistance. Validation of the design subjected the device to three different conditions, including robot guidance of the limb, co-contraction of the forearm, and the use of alternate approaches to complete the task. Co-contraction and alternate approach conditions were used to simulate characteristics of impaired subjects, including rigidity spasticity, and lack of muscle control. Robot guidance achieved an average orientation error of 5° or less in at least 75% of iterations across all tasks, while co-contraction and alternate approach was able to do this in flexion/extension, but saw much higher errors in forearm rotation. Causes for performance deficiencies were attributed to lack of torque bandwidth at the motor and response delay due to signal filtering, aspects that will be corrected in the next iteration of the design