22 research outputs found
Autonomous Tissue Scanning under Free-Form Motion for Intraoperative Tissue Characterisation
In Minimally Invasive Surgery (MIS), tissue scanning with imaging probes is
required for subsurface visualisation to characterise the state of the tissue.
However, scanning of large tissue surfaces in the presence of deformation is a
challenging task for the surgeon. Recently, robot-assisted local tissue
scanning has been investigated for motion stabilisation of imaging probes to
facilitate the capturing of good quality images and reduce the surgeon's
cognitive load. Nonetheless, these approaches require the tissue surface to be
static or deform with periodic motion. To eliminate these assumptions, we
propose a visual servoing framework for autonomous tissue scanning, able to
deal with free-form tissue deformation. The 3D structure of the surgical scene
is recovered and a feature-based method is proposed to estimate the motion of
the tissue in real-time. A desired scanning trajectory is manually defined on a
reference frame and continuously updated using projective geometry to follow
the tissue motion and control the movement of the robotic arm. The advantage of
the proposed method is that it does not require the learning of the tissue
motion prior to scanning and can deal with free-form deformation. We deployed
this framework on the da Vinci surgical robot using the da Vinci Research Kit
(dVRK) for Ultrasound tissue scanning. Since the framework does not rely on
information from the Ultrasound data, it can be easily extended to other
probe-based imaging modalities.Comment: 7 pages, 5 figures, ICRA 202
Regularising disparity estimation via multi task learning with structured light reconstruction
3D reconstruction is a useful tool for surgical planning and guidance.
However, the lack of available medical data stunts research and development in
this field, as supervised deep learning methods for accurate disparity
estimation rely heavily on large datasets containing ground truth information.
Alternative approaches to supervision have been explored, such as
self-supervision, which can reduce or remove entirely the need for ground
truth. However, no proposed alternatives have demonstrated performance
capabilities close to what would be expected from a supervised setup. This work
aims to alleviate this issue. In this paper, we investigate the learning of
structured light projections to enhance the development of direct disparity
estimation networks. We show for the first time that it is possible to
accurately learn the projection of structured light on a scene, implicitly
learning disparity. Secondly, we \textcolor{black}{explore the use of a multi
task learning (MTL) framework for the joint training of structured light and
disparity. We present results which show that MTL with structured light
improves disparity training; without increasing the number of model parameters.
Our MTL setup outperformed the single task learning (STL) network in every
validation test. Notably, in the medical generalisation test, the STL error was
1.4 times worse than that of the best MTL performance. The benefit of using MTL
is emphasised when the training data is limited.} A dataset containing
stereoscopic images, disparity maps and structured light projections on medical
phantoms and ex vivo tissue was created for evaluation together with virtual
scenes. This dataset will be made publicly available in the future
Caveats on the first-generation da Vinci Research Kit: latent technical constraints and essential calibrations
Telesurgical robotic systems provide a well established form of assistance in
the operating theater, with evidence of growing uptake in recent years. Until
now, the da Vinci surgical system (Intuitive Surgical Inc, Sunnyvale,
California) has been the most widely adopted robot of this kind, with more than
6,700 systems in current clinical use worldwide [1]. To accelerate research on
robotic-assisted surgery, the retired first-generation da Vinci robots have
been redeployed for research use as "da Vinci Research Kits" (dVRKs), which
have been distributed to research institutions around the world to support both
training and research in the sector. In the past ten years, a great amount of
research on the dVRK has been carried out across a vast range of research
topics. During this extensive and distributed process, common technical issues
have been identified that are buried deep within the dVRK research and
development architecture, and were found to be common among dVRK user feedback,
regardless of the breadth and disparity of research directions identified. This
paper gathers and analyzes the most significant of these, with a focus on the
technical constraints of the first-generation dVRK, which both existing and
prospective users should be aware of before embarking onto dVRK-related
research. The hope is that this review will aid users in identifying and
addressing common limitations of the systems promptly, thus helping to
accelerate progress in the field.Comment: 15 pages, 7 figure
Detecting the Sensing Area of A Laparoscopic Probe in Minimally Invasive Cancer Surgery
In surgical oncology, it is challenging for surgeons to identify lymph nodes
and completely resect cancer even with pre-operative imaging systems like PET
and CT, because of the lack of reliable intraoperative visualization tools.
Endoscopic radio-guided cancer detection and resection has recently been
evaluated whereby a novel tethered laparoscopic gamma detector is used to
localize a preoperatively injected radiotracer. This can both enhance the
endoscopic imaging and complement preoperative nuclear imaging data. However,
gamma activity visualization is challenging to present to the operator because
the probe is non-imaging and it does not visibly indicate the activity
origination on the tissue surface. Initial failed attempts used segmentation or
geometric methods, but led to the discovery that it could be resolved by
leveraging high-dimensional image features and probe position information. To
demonstrate the effectiveness of this solution, we designed and implemented a
simple regression network that successfully addressed the problem. To further
validate the proposed solution, we acquired and publicly released two datasets
captured using a custom-designed, portable stereo laparoscope system. Through
intensive experimentation, we demonstrated that our method can successfully and
effectively detect the sensing area, establishing a new performance benchmark.
Code and data are available at
https://github.com/br0202/Sensing_area_detection.gitComment: Accepted by MICCAI 202
Identifying Visible Tissue in Intraoperative Ultrasound Images during Brain Surgery: A Method and Application
Intraoperative ultrasound scanning is a demanding visuotactile task. It
requires operators to simultaneously localise the ultrasound perspective and
manually perform slight adjustments to the pose of the probe, making sure not
to apply excessive force or breaking contact with the tissue, whilst also
characterising the visible tissue. In this paper, we propose a method for the
identification of the visible tissue, which enables the analysis of ultrasound
probe and tissue contact via the detection of acoustic shadow and construction
of confidence maps of the perceptual salience. Detailed validation with both in
vivo and phantom data is performed. First, we show that our technique is
capable of achieving state of the art acoustic shadow scan line classification
- with an average binary classification accuracy on unseen data of 0.87.
Second, we show that our framework for constructing confidence maps is able to
produce an ideal response to a probe's pose that is being oriented in and out
of optimality - achieving an average RMSE across five scans of 0.174. The
performance evaluation justifies the potential clinical value of the method
which can be used both to assist clinical training and optimise robot-assisted
ultrasound tissue scanning
Detecting the Sensing Area of a Laparoscopic Probe in Minimally Invasive Cancer Surgery
In surgical oncology, it is challenging for surgeons to identify lymph nodes and completely resect cancer even with pre-operative imaging systems like PET and CT, because of the lack of reliable intraoperative visualization tools. Endoscopic radio-guided cancer detection and resection has recently been evaluated whereby a novel tethered laparoscopic gamma detector is used to localize a preoperatively injected radiotracer. This can both enhance the endoscopic imaging and complement preoperative nuclear imaging data. However, gamma activity visualization is challenging to present to the operator because the probe is non-imaging and it does not visibly indicate the activity origination on the tissue surface. Initial failed attempts used segmentation or geometric methods, but led to the discovery that it could be resolved by leveraging high-dimensional image features and probe position information. To demonstrate the effectiveness of this solution, we designed and implemented a simple regression network that successfully addressed the problem. To further validate the proposed solution, we acquired and publicly released two datasets captured using a custom-designed, portable stereo laparoscope system. Through intensive experimentation, we demonstrated that our method can successfully and effectively detect the sensing area, establishing a new performance benchmark. Code and data are available at https://github.com/br0202/Sensing_area_detection.git
Simultaneous Depth Estimation and Surgical Tool Segmentation in Laparoscopic Images
Surgical instrument segmentation and depth estimation are crucial steps to improve autonomy in robotic surgery. Most recent works treat these problems separately, making the deployment challenging. In this paper, we propose a unified framework for depth estimation and surgical tool segmentation in laparoscopic images. The network has an encoder-decoder architecture and comprises two branches for simultaneously performing depth estimation and segmentation. To train the network end to end, we propose a new multi-task loss function that effectively learns to estimate depth in an unsupervised manner, while requiring only semi-ground truth for surgical tool segmentation. We conducted extensive experiments on different datasets to validate these findings. The results showed that the end-to-end network successfully improved the state-of-the-art for both tasks while reducing the complexity during their deployment