End-to-End Real-time Catheter Segmentation with Optical Flow-Guided Warping during Endovascular Intervention

Accurate real-time catheter segmentation is an important pre-requisite for robot-assisted endovascular intervention. Most of the existing learning-based methods for catheter segmentation and tracking are only trained on small-scale datasets or synthetic data due to the difficulties of ground-truth annotation. Furthermore, the temporal continuity in intraoperative imaging sequences is not fully utilised. In this paper, we present FW-Net, an end-to-end and real-time deep learning framework for endovascular intervention. The proposed FW-Net has three modules: a segmentation network with encoder-decoder architecture, a flow network to extract optical flow information, and a novel flow-guided warping function to learn the frame-to-frame temporal continuity. We show that by effectively learning temporal continuity, the network can successfully segment and track the catheters in real-time sequences using only raw ground-truth for training. Detailed validation results confirm that our FW-Net outperforms state-of-the-art techniques while achieving real-time performance.Comment: ICRA 202

Modeling and Force Estimation of Cardiac Catheters for Haptics-enabled Tele-intervention

Robot-assisted cardiovascular intervention (RCI) systems have shown success in reducing the x-ray exposure to surgeons and patients during cardiovascular interventional procedures. RCI systems typically are teleoperated systems with leader-follower architecture. With such system architecture, the surgeon is placed out of the x-ray exposure zone and uses a console to control the robot remotely. Despite its success in reducing x-ray exposure, clinicians have identified the lack of force feedback as to its main technological limitation that can lead to vascular perforation of the patient’s vessels and even their death. The objective of this thesis was to develop, verify, and validate mechatronics technology for real-time accurate and robust haptic feedback rendering for RCI systems. To attain the thesis objective, first, a thorough review of the state-of-the-art clinical requirements, modeling approaches and methods, and current knowledge gaps for the provision of force feedback for RCI systems was performed. Afterward, a real-time tip force estimation method based on image-based shape-sensing and learning-from-simulation was developed and validated. The learning-based model was fairly accurate but required a large database for training which was computationally expensive. Next, a new mechanistic model, i.e., finite arc method (FAM) for soft robots was proposed, formulated, solved, and validated that allowed for fast and accurate modeling of catheter deformation. With FAM, the required training database for the proposed learning-from-simulation method would be generated with high speed and accuracy. In the end, to robustly relay the estimated forces from real-time imaging from the follower robot to the leader haptic device, a novel impedance-based force feedback rendering modality was proposed and implemented on a representative teleoperated RCI system for experimental validation. The proposed method was compared with the classical direct force reflection method and showed enhanced stability, robustness, and accuracy in the presence of communication disruption. The results of this thesis showed that the performance of the proposed integrated force feedback rendering system was in fair compliance with the clinical requirements and had superior robustness compared to the classical direct force reflection method

A Survey on the Current Status and Future Challenges Towards Objective Skills Assessment in Endovascular Surgery

Minimally-invasive endovascular interventions have evolved rapidly over the past decade, facilitated by breakthroughs in medical imaging and sensing, instrumentation and most recently robotics. Catheter based operations are potentially safer and applicable to a wider patient population due to the reduced comorbidity. As a result endovascular surgery has become the preferred treatment option for conditions previously treated with open surgery and as such the number of patients undergoing endovascular interventions is increasing every year. This fact coupled with a proclivity for reduced working hours, results in a requirement for efficient training and assessment of new surgeons, that deviates from the “see one, do one, teach one” model introduced by William Halsted, so that trainees obtain operational expertise in a shorter period. Developing more objective assessment tools based on quantitative metrics is now a recognised need in interventional training and this manuscript reports the current literature for endovascular skills assessment and the associated emerging technologies. A systematic search was performed on PubMed (MEDLINE), Google Scholar, IEEXplore and known journals using the keywords, “endovascular surgery”, “surgical skills”, “endovascular skills”, “surgical training endovascular” and “catheter skills”. Focusing explicitly on endovascular surgical skills, we group related works into three categories based on the metrics used; structured scales and checklists, simulation-based and motion-based metrics. This review highlights the key findings in each category and also provides suggestions for new research opportunities towards fully objective and automated surgical assessment solutions