7 research outputs found

    Autonomous Tissue Scanning under Free-Form Motion for Intraoperative Tissue Characterisation

    Full text link
    In Minimally Invasive Surgery (MIS), tissue scanning with imaging probes is required for subsurface visualisation to characterise the state of the tissue. However, scanning of large tissue surfaces in the presence of deformation is a challenging task for the surgeon. Recently, robot-assisted local tissue scanning has been investigated for motion stabilisation of imaging probes to facilitate the capturing of good quality images and reduce the surgeon's cognitive load. Nonetheless, these approaches require the tissue surface to be static or deform with periodic motion. To eliminate these assumptions, we propose a visual servoing framework for autonomous tissue scanning, able to deal with free-form tissue deformation. The 3D structure of the surgical scene is recovered and a feature-based method is proposed to estimate the motion of the tissue in real-time. A desired scanning trajectory is manually defined on a reference frame and continuously updated using projective geometry to follow the tissue motion and control the movement of the robotic arm. The advantage of the proposed method is that it does not require the learning of the tissue motion prior to scanning and can deal with free-form deformation. We deployed this framework on the da Vinci surgical robot using the da Vinci Research Kit (dVRK) for Ultrasound tissue scanning. Since the framework does not rely on information from the Ultrasound data, it can be easily extended to other probe-based imaging modalities.Comment: 7 pages, 5 figures, ICRA 202

    Motion-Aware Mosaicing for Confocal Laser Endomicroscopy

    Get PDF
    International audienceProbe-based Confocal Laser Endomicroscopy (pCLE) provides physicians with real-time access to histological information during standard endoscopy procedures, through high-resolution cellular imaging of internal tissues. Earlier work on mosaicing has enhanced the potential of this imaging modality by meeting the need to get a complete representation of the imaged region. However, with approaches, the dynamic information, which may be of clinical interest, is lost. In this study, we propose a new mosaic construction algorithm for pCLE sequences based on a min-cut optimization and gradient-domain composition. Its main advantage is that the motion of some structures within the tissue such as blood cells in capillaries, is taken into account. This allows physicians to get both a sharper static representation and a dynamic representation of the imaged tissue. Results on 16 sequences acquired in vivo on six different organs demonstrate the clinical relevance of our approach

    High-resolution fluorescence endomicroscopy for rapid evaluation of breast cancer margins

    Get PDF
    Breast cancer is a major public health problem world-wide and the second leading cause of cancer-related female deaths. Breast conserving surgery (BCS), in the form of wide local excision (WLE), allows complete tumour resection while maintaining acceptable cosmesis. It is the recommended treatment for a large number of patients with early stage disease or, in more advanced cases, following neoadjuvant chemotherapy. About 30% of patients undergoing BCS require one or more re-operative interventions, mainly due to the presence of positive margins. The standard of care for surgical margin assessment is post-operative examination of histopathological tissue sections. However, this process is invasive, introduces sampling errors and does not provide real-time assessment of the tumour status of radial margins. The objective of this thesis is to improve intra-operative assessment of margin status by performing optical biopsy in breast tissue. This thesis presents several technical and clinical developments related to confocal fluorescence endomicroscopy systems for real-time characterisation of different breast morphologies. The imaging systems discussed employ flexible fibre-bundle based imaging probes coupled to high-speed line-scan confocal microscope set-up. A preliminary study on 43 unfixed breast specimens describes the development and testing of line-scan confocal laser endomicroscope (LS-CLE) to image and classify different breast pathologies. LS-CLE is also demonstrated to assess the intra-operative tumour status of whole WLE specimens and surgical excisions with high diagnostic accuracy. A third study demonstrates the development and testing of a bespoke LS-CLE system with methylene blue (MB), an US Food and Drug Administration (FDA) approved fluorescent agent, and integration with robotic scanner to enable large-area in vivo imaging of breast cancer. The work also addresses three technical issues which limit existing fibre-bundle based fluorescence endomicroscopy systems: i) Restriction to use single fluorescence agent due to low-speed, single excitation and single fluorescence spectral band imaging systems; ii) Limited Field of view (FOV) of fibre-bundle endomicroscopes due to small size of the fibre tip and iii) Limited spatial resolution of fibre-bundle endomicroscopes due to the spacing between the individual fibres leading to fibre-pixelation effects. Details of design and development of a high-speed dual-wavelength LS-CLE system suitable for high-resolution multiplexed imaging are presented. Dual-wavelength imaging is achieved by sequentially switching between 488 nm and 660 nm laser sources for alternate frames, avoiding spectral bleed-through, and providing an effective frame rate of 60 Hz. A combination of hand-held or robotic scanning with real-time video mosaicking, is demonstrated to enable large-area imaging while still maintaining microscopic resolution. Finally, a miniaturised piezoelectric transducer-based fibre-shifting endomicroscope is developed to enhance the resolution over conventional fibre-bundle based imaging systems. The fibre-shifting endomicroscope provides a two-fold improvement in resolution and coupled to a high-speed LS-CLE scanning system, provides real-time imaging of biological samples at 30 fps. These investigations furthered the utility and applications of the fibre-bundle based fluorescence systems for rapid imaging and diagnosis of cancer margins.Open Acces

    Automated analysis of confocal laser endomicroscopy images to detect head and neck cancer

    Get PDF
    Den weltweiten Goldstandard zur DignitĂ€tsbestimmung auffĂ€lliger Schleimhaut-lĂ€sionen des oberen Aerodigestivtraktes (OADT) stellt die invasive Entnahme von Gewebeproben zur Begutachtung durch einen Pathologen dar. LĂ€sst sich histologisch ein maligner Tumor nachweisen, handelt es sich in ĂŒber 90% der FĂ€lle um ein Plattenepithelkarzinom (PEC) der SchleimhĂ€ute (Pai und Westra 2009). Die visuelle und endoskopische Untersuchung erfolgt aktuell sowohl ambulant als auch wĂ€hrend Tumoroperationen im klinischen Alltag nur mit Weißlicht. Eine langjĂ€hrige klinische Erfahrung und genaue Kenntnis der Anatomie sind daher zwingend notwendig, da eine frĂŒhzeitige Diagnose entscheidend fĂŒr die Behandlungsstrategie und die Chancen auf Heilung der Patienten ist. Es handelt sich hier um eine stark untersucherabhĂ€ngige Methode, die keine unmittelbare histologische Aussage zu SchleimhautverĂ€nderungen im OADT treffen kann (Ambrosch 1996). Deshalb werden seit Jahrzehnten weltweit verschiedene innovative optische Bildgebungsverfahren in der Hals-, Nasen- und Ohrenheilkunde (HNO-Heilkunde) zur besseren Detektion und Abgrenzung von Tumoren entwickelt. Das ideale Ziel der einzelnen Verfahren ist non-invasiv und in Echtzeit im Sinne einer „optischen Biopsie“ wĂ€hrend ambulanter Untersuchungen oder bei Operationen definitive Aussagen ĂŒber GewebeverĂ€nderungen zu treffen (Volgger et al. 2013a, Arens et al. 2016). Bisher wird noch kein optisches Diagnoseverfahren im klinischen Alltag angewendet (Betz et al. 2016). Eine relativ neue Technik stellt in diesem Zusammenhang die konfokale Endomikroskopie (CLE) dar. Im Vergleich zu anderen Fachdisziplinen wie beispielsweise der Gastroenterologie wurden in der HNO-Heilkunde nur wenige Arbeiten publiziert, die die CLE zur Erkennung von PEC verwendet (Abbaci et al. 2014, Goetz et al. 2011). Es wurde bisher gezeigt, dass diese optische Technik zu diesem Zweck ein gewisses Potential besitzt. Quantitativ messbare Kriterien, die eine eindeutige Unterscheidung zwischen Tumorgewebe und gesunder Schleimhaut ermöglich, wurden aber noch nicht bestimmt (Thong et al. 2012, Volgger et al. 2013a). UnabhĂ€ngig von einander kommen verschiedene Studien zu dem Schluss, dass bei der Betrachtung von CLE-Aufnahmen Unterschiede in der Architektur der ZellverbĂ€nde und in der ZellgrĂ¶ĂŸe von Tumoren im Vergleich zu gesunder Schleimhaut auffallend sind (Pogorzelski et al. 2012, Haxel et al. 2010). In der zugrunde liegenden publizierten Orginalarbeit wird unseres Wissens der weltweit erste automatisierte Bilderkennungsalgorithmus zur Detektion von PEC im OADT anhand von CLEBilder vorgestellt. Die vorgelegte Arbeit ist zudem die weltweit erste Publikation, die quantitativ messbare Bilddaten in CLE-Bildern erhebt. Sie beweist, dass sowohl die Architektur der oberflĂ€chlichen ZellverbĂ€nde als auch die ZellgrĂ¶ĂŸe in CLE-Bilder valide Kriterien sind, anhand derer ein PEC von gesunder Schleimhaut unterschieden werden kann. DarĂŒber hinaus wurden bei der Studie indirekt zahlreiche Daten ĂŒber die generelle ZellgrĂ¶ĂŸe und Gewebestruktur von PEC und gesunder Schleimhaut des OADT erhoben. Die prospektive Observationsstudie wurde in der Klinik fĂŒr Hals-Nasen-Ohrenheilkunde am UniversitĂ€tsklinikum Jena durchgefĂŒhrt. Teilnehmer der Studie waren 12 Patienten mit klinischem Verdacht eines PEC. Die CLE-Bilder wurden nach intravenöser (i. v.) Applikation von Fluorescein in vivo wĂ€hrend diagnostischer Panendoskopien aufgezeichnet. An allen untersuchten SchleimhautlĂ€sionen wurden direkt im Anschluss Biopsien entnommen. Zwei Gruppen mit einerseits histologischem Nachweis eines PEC (Tumorgruppe n=5) und andererseits mit gesunder Schleimhaut (Kontrollgruppe n=7) wurden gebildet. Die Auswertung der CLE-Aufnahmen sowie die Annotation relevanter Bildsequenzen und Bildareale erfolgte mit medizinischem Expertenwissen. Darauf aufbauend wurden im nĂ€chsten Schritt mit Methoden der digitalen Bilderkennung quantitativ messbare Bilddaten identifiziert. Die Analyse mit spezifischen Bilderkennungsverfahren („automated cell border segmentation, distance map“) ergab statistische Werte der ZellgrĂ¶ĂŸen in den beiden Gruppen. Anhand dieser Informationen erfolgte das Training des Algorithmus mit der „leave-two-patients-out“- Methode (Hyperlink zum öffentlich zugĂ€nglichen technischen Report: http://www.inf-cv.unijena. de/microscopyanalysis). Unser Algorithmus ist in der Lage mit einer SpezifitĂ€t von 0.85 ± 0.14 und einer SensitivitĂ€t von 0.72 ± 0.13 CLE-Bilder von PEC von gesunder Schleimhaut zu unterscheiden. Um die Aussagen des Algorithmus korrekt zu bewerten ist bei der Anwendung dieses optischen Verfahrens medizinisches Expertenwissen notwendig. Die Weiterentwicklung zur „online“-Anwendung im Sinne einer „optischen Biopsie“ als ErgĂ€nzung zur Weißlichtuntersuchung erscheint realistisch, wenn grĂ¶ĂŸere klinische Studien folgen

    Robotic manipulators for single access surgery

    Get PDF
    This thesis explores the development of cooperative robotic manipulators for enhancing surgical precision and patient outcomes in single-access surgery and, specifically, Transanal Endoscopic Microsurgery (TEM). During these procedures, surgeons manipulate a heavy set of instruments via a mechanical clamp inserted in the patient’s body through a surgical port, resulting in imprecise movements, increased patient risks, and increased operating time. Therefore, an articulated robotic manipulator with passive joints is initially introduced, featuring built-in position and force sensors in each joint and electronic joint brakes for instant lock/release capability. The articulated manipulator concept is further improved with motorised joints, evolving into an active tool holder. The joints allow the incorporation of advanced robotic capabilities such as ultra-lightweight gravity compensation and hands-on kinematic reconfiguration, which can optimise the placement of the tool holder in the operating theatre. Due to the enhanced sensing capabilities, the application of the active robotic manipulator was further explored in conjunction with advanced image guidance approaches such as endomicroscopy. Recent advances in probe-based optical imaging such as confocal endomicroscopy is making inroads in clinical uses. However, the challenging manipulation of imaging probes hinders their practical adoption. Therefore, a combination of the fully cooperative robotic manipulator with a high-speed scanning endomicroscopy instrument is presented, simplifying the incorporation of optical biopsy techniques in routine surgical workflows. Finally, another embodiment of a cooperative robotic manipulator is presented as an input interface to control a highly-articulated robotic instrument for TEM. This master-slave interface alleviates the drawbacks of traditional master-slave devices, e.g., using clutching mechanics to compensate for the mismatch between slave and master workspaces, and the lack of intuitive manipulation feedback, e.g. joint limits, to the user. To address those drawbacks a joint-space robotic manipulator is proposed emulating the kinematic structure of the flexible robotic instrument under control.Open Acces

    Building large mosaics of confocal edomicroscopic images using visual servoing

    Get PDF
    Probe-based Confocal Laser Endomicroscopy (pCLE) provides real-time microscopic images of tissues contacted by a small probe that can be inserted in vivo through a minimally invasive access. Mosaicking consists in sweeping the probe in contact with a tissue to be imaged while collecting the video stream, and process the images to assemble them in a large mosaic. While most of the literature in this field has focused on image processing, little attention has been paid so far to the way the probe motion can be controlled. This is a crucial issue since the precision of the probe trajectory control drastically influences the quality of the final mosaic. Robotically controlled motion has the potential of providing enough precision to perform mosaicking. In this paper, we emphasize the difficulties of implementing such an approach. Firstly, probe-tissue contacts generate deformations that prevent from properly controlling the image trajectory. Secondly, in the context of minimally invasive procedures targeted by our research, robotic devices are likely to exhibit limited quality of the distal probe motion control at the microscopic scale. To cope with these problems visual servoing from real-time endomicroscopy images is proposed in this paper. It is implemented on two different devices (a high-accuracy industrial robot and a prototype minimally invasive device). Experiments on different kinds of environments (printed paper and ex vivo tissues) show that quality of the visually servoed probe motion is sufficient to build mosaics with minimal distortion in spite of disturbances
    corecore