7 research outputs found
Autonomous Tissue Scanning under Free-Form Motion for Intraoperative Tissue Characterisation
In Minimally Invasive Surgery (MIS), tissue scanning with imaging probes is
required for subsurface visualisation to characterise the state of the tissue.
However, scanning of large tissue surfaces in the presence of deformation is a
challenging task for the surgeon. Recently, robot-assisted local tissue
scanning has been investigated for motion stabilisation of imaging probes to
facilitate the capturing of good quality images and reduce the surgeon's
cognitive load. Nonetheless, these approaches require the tissue surface to be
static or deform with periodic motion. To eliminate these assumptions, we
propose a visual servoing framework for autonomous tissue scanning, able to
deal with free-form tissue deformation. The 3D structure of the surgical scene
is recovered and a feature-based method is proposed to estimate the motion of
the tissue in real-time. A desired scanning trajectory is manually defined on a
reference frame and continuously updated using projective geometry to follow
the tissue motion and control the movement of the robotic arm. The advantage of
the proposed method is that it does not require the learning of the tissue
motion prior to scanning and can deal with free-form deformation. We deployed
this framework on the da Vinci surgical robot using the da Vinci Research Kit
(dVRK) for Ultrasound tissue scanning. Since the framework does not rely on
information from the Ultrasound data, it can be easily extended to other
probe-based imaging modalities.Comment: 7 pages, 5 figures, ICRA 202
Motion-Aware Mosaicing for Confocal Laser Endomicroscopy
International audienceProbe-based Confocal Laser Endomicroscopy (pCLE) provides physicians with real-time access to histological information during standard endoscopy procedures, through high-resolution cellular imaging of internal tissues. Earlier work on mosaicing has enhanced the potential of this imaging modality by meeting the need to get a complete representation of the imaged region. However, with approaches, the dynamic information, which may be of clinical interest, is lost. In this study, we propose a new mosaic construction algorithm for pCLE sequences based on a min-cut optimization and gradient-domain composition. Its main advantage is that the motion of some structures within the tissue such as blood cells in capillaries, is taken into account. This allows physicians to get both a sharper static representation and a dynamic representation of the imaged tissue. Results on 16 sequences acquired in vivo on six different organs demonstrate the clinical relevance of our approach
High-resolution fluorescence endomicroscopy for rapid evaluation of breast cancer margins
Breast cancer is a major public health problem world-wide and the second leading cause of cancer-related female deaths. Breast conserving surgery (BCS), in the form of wide local excision (WLE), allows complete tumour resection while maintaining acceptable cosmesis. It is the recommended treatment for a large number of patients with early stage disease or, in more advanced cases, following neoadjuvant chemotherapy. About 30% of patients undergoing BCS require one or more re-operative interventions, mainly due to the presence of positive margins. The standard of care for surgical margin assessment is post-operative examination of histopathological tissue sections. However, this process is invasive, introduces sampling errors and does not provide real-time assessment of the tumour status of radial margins. The objective of this thesis is to improve intra-operative assessment of margin status by performing optical biopsy in breast tissue.
This thesis presents several technical and clinical developments related to confocal fluorescence endomicroscopy systems for real-time characterisation of different breast morphologies. The imaging systems discussed employ flexible fibre-bundle based imaging probes coupled to high-speed line-scan confocal microscope set-up. A preliminary study on 43 unfixed breast specimens describes the development and testing of line-scan confocal laser endomicroscope (LS-CLE) to image and classify different breast pathologies. LS-CLE is also demonstrated to assess the intra-operative tumour status of whole WLE specimens and surgical excisions with high diagnostic accuracy. A third study demonstrates the development and testing of a bespoke LS-CLE system with methylene blue (MB), an US Food and Drug Administration (FDA) approved fluorescent agent, and integration with robotic scanner to enable large-area in vivo imaging of breast cancer.
The work also addresses three technical issues which limit existing fibre-bundle based fluorescence endomicroscopy systems: i) Restriction to use single fluorescence agent due to low-speed, single excitation and single fluorescence spectral band imaging systems; ii) Limited Field of view (FOV) of fibre-bundle endomicroscopes due to small size of the fibre tip and iii) Limited spatial resolution of fibre-bundle endomicroscopes due to the spacing between the individual fibres leading to fibre-pixelation effects.
Details of design and development of a high-speed dual-wavelength LS-CLE system suitable for high-resolution multiplexed imaging are presented. Dual-wavelength imaging is achieved by sequentially switching between 488 nm and 660 nm laser sources for alternate frames, avoiding spectral bleed-through, and providing an effective frame rate of 60 Hz. A combination of hand-held or robotic scanning with real-time video mosaicking, is demonstrated to enable large-area imaging while still maintaining microscopic resolution. Finally, a miniaturised piezoelectric transducer-based fibre-shifting endomicroscope is developed to enhance the resolution over conventional fibre-bundle based imaging systems. The fibre-shifting endomicroscope provides a two-fold improvement in resolution and coupled to a high-speed LS-CLE scanning system, provides real-time imaging of biological samples at 30 fps. These investigations furthered the utility and applications of the fibre-bundle based fluorescence systems for rapid imaging and diagnosis of cancer margins.Open Acces
Automated analysis of confocal laser endomicroscopy images to detect head and neck cancer
Den weltweiten Goldstandard zur DignitĂ€tsbestimmung auffĂ€lliger Schleimhaut-lĂ€sionen des oberen Aerodigestivtraktes (OADT) stellt die invasive Entnahme von Gewebeproben zur Begutachtung durch einen Pathologen dar. LĂ€sst sich histologisch ein maligner Tumor nachweisen, handelt es sich in ĂŒber 90% der FĂ€lle um ein Plattenepithelkarzinom (PEC) der SchleimhĂ€ute (Pai und Westra 2009). Die visuelle und endoskopische Untersuchung erfolgt aktuell sowohl ambulant als auch wĂ€hrend Tumoroperationen im klinischen Alltag nur mit WeiĂlicht. Eine langjĂ€hrige klinische Erfahrung und genaue Kenntnis der Anatomie sind daher zwingend notwendig, da eine frĂŒhzeitige Diagnose entscheidend fĂŒr die Behandlungsstrategie und die Chancen auf Heilung der Patienten ist. Es handelt sich hier um eine stark untersucherabhĂ€ngige Methode, die keine unmittelbare histologische Aussage zu SchleimhautverĂ€nderungen im OADT treffen kann (Ambrosch 1996). Deshalb werden seit Jahrzehnten weltweit verschiedene innovative optische Bildgebungsverfahren in der Hals-, Nasen- und Ohrenheilkunde (HNO-Heilkunde) zur besseren Detektion und Abgrenzung von Tumoren entwickelt. Das ideale Ziel der einzelnen Verfahren ist non-invasiv und in Echtzeit im Sinne einer âoptischen Biopsieâ wĂ€hrend ambulanter Untersuchungen oder bei Operationen definitive Aussagen ĂŒber GewebeverĂ€nderungen zu treffen (Volgger et al. 2013a, Arens et al. 2016). Bisher wird noch kein optisches Diagnoseverfahren im klinischen Alltag angewendet (Betz et al. 2016). Eine relativ neue Technik stellt in diesem Zusammenhang die konfokale
Endomikroskopie (CLE) dar. Im Vergleich zu anderen Fachdisziplinen wie beispielsweise der Gastroenterologie wurden in der HNO-Heilkunde nur wenige Arbeiten publiziert, die die CLE zur Erkennung von PEC verwendet (Abbaci et al. 2014, Goetz et al. 2011). Es wurde bisher gezeigt, dass diese optische Technik zu diesem Zweck ein gewisses Potential besitzt. Quantitativ messbare Kriterien, die eine eindeutige Unterscheidung zwischen Tumorgewebe und gesunder Schleimhaut ermöglich, wurden aber noch nicht bestimmt (Thong et al. 2012, Volgger et al. 2013a). UnabhĂ€ngig von einander kommen verschiedene Studien zu dem Schluss, dass bei der Betrachtung von CLE-Aufnahmen Unterschiede in der Architektur der ZellverbĂ€nde und in der ZellgröĂe von Tumoren im Vergleich zu gesunder Schleimhaut auffallend sind (Pogorzelski et al. 2012, Haxel et al. 2010). In der zugrunde liegenden publizierten Orginalarbeit wird unseres Wissens der weltweit erste automatisierte Bilderkennungsalgorithmus zur Detektion von PEC im OADT anhand von CLEBilder vorgestellt. Die vorgelegte Arbeit ist zudem die weltweit erste Publikation, die quantitativ messbare Bilddaten in CLE-Bildern erhebt. Sie beweist, dass sowohl die Architektur der oberflĂ€chlichen ZellverbĂ€nde als auch die ZellgröĂe in CLE-Bilder valide Kriterien sind, anhand derer ein PEC von gesunder Schleimhaut unterschieden werden kann. DarĂŒber hinaus wurden bei der Studie indirekt zahlreiche Daten ĂŒber die generelle ZellgröĂe und Gewebestruktur von PEC und gesunder Schleimhaut des OADT erhoben. Die prospektive Observationsstudie wurde in der Klinik fĂŒr Hals-Nasen-Ohrenheilkunde am UniversitĂ€tsklinikum Jena durchgefĂŒhrt. Teilnehmer der Studie waren 12 Patienten mit klinischem Verdacht eines PEC. Die CLE-Bilder wurden nach intravenöser (i. v.) Applikation von Fluorescein in vivo wĂ€hrend diagnostischer Panendoskopien aufgezeichnet. An allen untersuchten SchleimhautlĂ€sionen wurden direkt im Anschluss Biopsien entnommen. Zwei Gruppen mit einerseits histologischem Nachweis eines PEC (Tumorgruppe n=5) und andererseits mit gesunder Schleimhaut (Kontrollgruppe n=7) wurden gebildet. Die Auswertung der CLE-Aufnahmen sowie die Annotation relevanter Bildsequenzen und Bildareale erfolgte mit medizinischem Expertenwissen. Darauf aufbauend wurden im nĂ€chsten Schritt mit Methoden der digitalen Bilderkennung quantitativ messbare Bilddaten identifiziert. Die Analyse mit spezifischen Bilderkennungsverfahren (âautomated cell border segmentation, distance mapâ) ergab statistische Werte der ZellgröĂen in den beiden Gruppen. Anhand dieser Informationen erfolgte das Training des Algorithmus mit der âleave-two-patients-outâ- Methode (Hyperlink zum öffentlich zugĂ€nglichen technischen Report: http://www.inf-cv.unijena. de/microscopyanalysis). Unser Algorithmus ist in der Lage mit einer SpezifitĂ€t von 0.85 ± 0.14 und einer SensitivitĂ€t von 0.72 ± 0.13 CLE-Bilder von PEC von gesunder Schleimhaut zu unterscheiden. Um die Aussagen des Algorithmus korrekt zu bewerten ist bei der Anwendung
dieses optischen Verfahrens medizinisches Expertenwissen notwendig. Die Weiterentwicklung zur âonlineâ-Anwendung im Sinne einer âoptischen Biopsieâ als ErgĂ€nzung zur WeiĂlichtuntersuchung erscheint realistisch, wenn gröĂere klinische Studien folgen
Robotic manipulators for single access surgery
This thesis explores the development of cooperative robotic manipulators for enhancing surgical precision and patient outcomes in single-access surgery and, specifically, Transanal Endoscopic Microsurgery (TEM). During these procedures, surgeons manipulate a heavy set of instruments via a mechanical clamp inserted in the patientâs body through a surgical port, resulting in imprecise movements, increased patient risks, and increased operating time. Therefore, an articulated robotic manipulator with passive joints is initially introduced, featuring built-in position and force sensors in each joint and electronic joint brakes for instant lock/release capability.
The articulated manipulator concept is further improved with motorised joints, evolving into an active tool holder. The joints allow the incorporation of advanced robotic capabilities such as ultra-lightweight gravity compensation and hands-on kinematic reconfiguration, which can optimise the placement of the tool holder in the operating theatre.
Due to the enhanced sensing capabilities, the application of the active robotic manipulator was further explored in conjunction with advanced image guidance approaches such as endomicroscopy. Recent advances in probe-based optical imaging such as confocal endomicroscopy is making inroads in clinical uses. However, the challenging manipulation of imaging probes hinders their practical adoption. Therefore, a combination of the fully cooperative robotic manipulator with a high-speed scanning endomicroscopy instrument is presented, simplifying the incorporation of optical biopsy techniques in routine surgical workflows.
Finally, another embodiment of a cooperative robotic manipulator is presented as an input interface to control a highly-articulated robotic instrument for TEM. This master-slave interface alleviates the drawbacks of traditional master-slave devices, e.g., using clutching mechanics to compensate for the mismatch between slave and master workspaces, and the lack of intuitive manipulation feedback, e.g. joint limits, to the user. To address those drawbacks a joint-space robotic manipulator is proposed emulating the kinematic structure of the flexible robotic instrument under control.Open Acces
Building large mosaics of confocal edomicroscopic images using visual servoing
Probe-based Confocal Laser Endomicroscopy (pCLE) provides real-time microscopic images of tissues contacted by a small probe that can be inserted in vivo through a minimally invasive access. Mosaicking consists in sweeping the probe in contact with a tissue to be imaged while collecting the video stream, and process the images to assemble them in a large mosaic. While most of the literature in this field has focused on image processing, little attention has been paid so far to the way the probe motion can be controlled. This is a crucial issue since the precision of the probe trajectory control drastically influences the quality of the final mosaic. Robotically controlled motion has the potential of providing enough precision to perform mosaicking. In this paper, we emphasize the difficulties of implementing such an approach. Firstly, probe-tissue contacts generate deformations that prevent from properly controlling the image trajectory. Secondly, in the context of minimally invasive procedures targeted by our research, robotic devices are likely to exhibit limited quality of the distal probe motion control at the microscopic scale. To cope with these problems visual servoing from real-time endomicroscopy images is proposed in this paper. It is implemented on two different devices (a high-accuracy industrial robot and a prototype minimally invasive device). Experiments on different kinds of environments (printed paper and ex vivo tissues) show that quality of the visually servoed probe motion is sufficient to build mosaics with minimal distortion in spite of disturbances