Smart Surgical Microscope based on Optical Coherence Domain Reflectometry

Department of Biomedical EngineeringOver the several decades, there have been clinical needs that requires advanced technologies in medicine. Optical coherence tomography (OCT), one of the newly emerged medical imaging devices, provides non-invasive cross-sectional images in high resolution which is mainly used in ophthalmology. However, due to the limited penetration depth of 1-2 mm in bio-samples, there is a limit to be widely used. In order to easily integrate with existing medical tools and be convenient to users, it is necessary that the sample unit of OCT should be compact and simple. In this study, we developed high-speed swept-source OCT (SS-OCT) for advanced screening of otolaryngology. Synchronized signal sampling with a high-speed digitizer using a clock signal from a swept laser source, its trigger signal is also used to synchronize with the movement of the scanning mirror. The SS-OCT system can reliably provide high-throughput images, and two-axis scanning of galvano mirrors enables real-time acquisition of 3D data. Graphic processing unit (GPU) can performs high-speed data processing through parallel programming, and can also implement perspective projection 3D OCT visualization with optimal ray casting techniques. In the Clinical Study of Otolaryngology, OCT was applied to identify the microscopic extrathyroidal extension (mETE) of papillary thyroid cancer (PTC). As a result to detect the mETE of around 60% in conventional ultrasonography, it could be improved to 84.1% accuracy in our study. The detection ratio of the mETE was calculated by the pathologist analyzing the histologic image. In chapter 3, we present a novel study using combined OCT system integrated with a conventional surgical microscope. In the current set-up of surgical microscope, only two-dimensional microscopic images through the eyepiece view are provided to the surgeon. Thus, image-guided surgery, which provides real-time image information of the tissues or the organs, has been developed as an advanced surgical technique. This study illustrate newly designed optical set-up of smart surgical microscope that combined sample arm of the OCT with an existing microscope. Specifically, we used a beam projector to overlay OCT images on existing eyepiece views, and demonstrated augmented reality images. In chapter 4, in order to develop novel microsurgical instruments, optical coherence domain reflectometry (OCDR) was applied. Introduces smart surgical forceps using OCDR as a sensor that provides high-speed, high-resolution distance information in the tissue. To attach the sensor to the forceps, the lensed fiber which is a small and high sensitivity sensor was fabricated and the results are shown to be less affected by the tilt angle. In addition, the piezo actuator compensates the hand tremor, resulting in a reduction in the human hand tremor of 5 to 15 Hz. Finally, M-mode OCT needle is proposed for microsurgery guidance in ophthalmic surgery. Stepwise transitional core (STC) fiber was applied as a sensor to measure information within the tissue and attached to a 26 gauge needle. It shows the modified OCT system and the position-guided needle design of the sample stage and shows the algorithm flowchart of M-mode OCT imaging software. The developed M-mode OCT needle has been applied to animal studies using rabbit eyes and demonstrates the big-bubble deep anterior lamellar keratoplasty (DALK) surgery for corneal transplantation. Through this study, we propose a novel microsurgical instrument for lamellar keratoplasty and evaluate its feasibility with conventional regular OCT system images. In conclusion, for fundamental study required new augmented reality guided surgery with smart surgical microscope, it is expected that OCT combined with surgical microscope can be widely used. We demonstrated a novel microsurgical instrument to share with light source and the various optical components. Acquired information throughout our integrated system would be a key method to meet a wide range of different clinical needs in the real world.ope

Development and preliminary results of bimanual smart micro-surgical system using a ball-lens coupled OCT distance sensor

Bimanual surgery enhances surgical effectiveness and is required to successfully accomplish complex microsurgical tasks. The essential advantage is the ability to simultaneously grasp tissue with one hand to provide counter traction or exposure, while dissecting with the other. Towards enhancing the precision and safety of bimanual microsurgery we present a bimanual SMART micro-surgical system for a preliminary ex-vivo study. To the best of our knowledge, this is the first demonstration of a handheld bimanual microsurgical system. The essential components include a ball-lens coupled common-path swept source optical coherence tomography sensor. This system effectively suppresses asynchronous hand tremor using two PZT motors in feedback control loop and efficiently assists ambidextrous tasks. It allows precise bimanual dissection of biological tissues with a reduction in operating time as compared to the same tasks performed with conventional onehanded approaches. © 2016 Optical Society of America.1

Oblique injection depth correction by two parallel OCT sensors guided handheld SMART injector

We present a SMART injector with two parallel common-path optical coherence tomography fibers to enable angle measurements and injection depth corrections for oblique subretinal injection. The two optical fibers are attached to opposite sides of a 33 G needle with known offsets and designed to pass through a 23 G trocar that has an inner diameter of 0.65 mm. By attaching a SMART system to a rotational stage, the measured angles are calibrated for minimal error from reference angles. A commercial eye model was used to evaluate the control performance, and injection experiments were performed on a phantom made of agarose gel and a porcine eye. © 2021 Optical Society of America under the terms of the OSA Open Access Publishing Agreement.1

Optical Coherence Tomography Distal Sensor Based Handheld Microsurgical Tools

Microsurgery is typically differentiated from a general surgery in that it requires a precise sub-millimeter manipulation that could only be achievable under optical magnification. For instance, microsurgeons use surgical microscopes to view surgical sites and train themselves several years to acquire surgical skills to perform the delicate procedures. However, such microsurgical approach imposes considerable physical stress and mental fatigue on the surgeons and these could be sources for surgical risks and complications. For these reasons, a variety of robotic based surgical guidance methods have been developed and studied with the hope of providing safer and more precise microsurgery. These robotic arm based systems have been developed to provide precise tool movement and to remove physiological hand tremor, which is one of the main limiting factors that prevents precise tool manipulation. In another approaches use simpler system that adds robotic functions to existing handheld surgical tools. It is a hybrid system that incorporates the advantages of conventional manual system and robot-assist system. The advantages of such hybrid handheld systems include portability, disposability, and elimination of the large robotic-assist systems in complex surgical environment. The most critical benefit of the hybrid handheld system is its ease of use since it allows surgeons to manipulate tools mostly using their hand. However due to the imprecise nature of tool control using hands, tool tracking is more critical in handheld microsurgical tool systems than that of robotic arm systems. In general, the accuracy of the tool control is largely determined by the resolution of the sensors and the actuators. Therefore, it is essential to develop a real-time high resolution sensor in order to develop a practical microsurgical tools. For this reason, a novel intuitive targeting and tracking scheme that utilizes a common-path swept source optical coherence tomography (CP-SSOCT) distal sensor was developed integrated with handheld microsurgical tools. To achieve micron-order precision control, a reliable and accurate OCT distal sensing method was developed. The method uses a prediction algorithm is necessary to compensate for the system delay associated with the computational, mechanical and electronic latencies. Due to the multi-layered structure of retina, it was also necessary to develop effective surface detection methods rather than simple peak detection. The OCT distal sensor was integrated into handheld motion-guided micro-forceps system for highly accurate depth controlled epiretinal membranectomy. A touch sensor and two motors were used in the forceps design to minimize the motion artifact induced by squeezing, and to independently control the depth guidance of the tool-tip and the grasping action. We also built a depth guided micro-injector system that enables micro-injection with precise injection depth control. For these applications, a smart motion monitoring and a guiding algorithm were developed to provide precise and intuitive freehand control. Finally, phantom and ex-vivo bovine eye experiments were performed to evaluate the performance of the proposed OCT distal sensor and validate the effectiveness of the depth-guided micro-forceps and micro-injector over the freehand performance

Intraoperative Fourier domain optical coherence tomography for microsurgery guidance and assessment

In this dissertation, advanced high-speed Fourier domain optical coherence tomography (FD-OCT)systems were investigated and developed. Several real-time, high resolution functional Spectral-domain OCT (SD-OCT) systems capable of imaging and sensing blood flow and motion were designed and developed. The system were designed particularly for microsurgery guidance and assessment. The systems were tested for their ability to assessing microvascular anastomosis and vulnerable plaque development. An all fiber-optic common-path optical coherence tomography (CP-OCT) system capable of measuring high-resolution optical distances, was built and integrated into di fferent imaging modalities. First, a novel non-contact accurate in-vitro intra-ocular lens power measurement method was proposed and validated based on CP-OCT. Second, CP-OCT was integrated with a ber bundle based confocal microscope to achieve motion-compensated imaging. Distance between the probe and imaged target was monitored by the CP-OCT system in real-time.The distance signal from the CP-OCT system was routed to a high speed, high resolution linear motor to compensate for the axial motion of the sample in a closed-loop control. Finally a motion-compensated hand-held common-path Fourier domain optical coherence tomography probe was developed for image-guided intervention. Both phantom and ex vivo models were used to test and evaluate the probe. As the data acquisition speed of current OCT systems continue to increase, the means to process the data in real-time are in critically needed. Previous graphics processing unit accelerated OCT signal processing methods have shown their potential to achieve real-time imaging. In this dissertation, algorithms to perform real-time reference A-line subtraction and saturation artifact removal were proposed, realized and integrated into previously developed FD-OCT system CPU-GPU heterogeneous structure. Fourier domain phase resolved Doppler OCT (PRDOCT) system capable of real-time simultaneous structure and flow imaging based on dual GPUs was also developed and implemented. Finally, systematic experiments were conducted to validate the system for surgical applications. FD-OCT system was used to detect atherosclerotic plaque and drug effi ciency test in mouse model. Application of PRDOCT for both suture and cu ff based microvascular anastomosis guidance and assessment was extensively stuided in rodent model

Développement et validation de sondes en fibre optique miniaturisées pour le guidage intra-opératoire d’interventions intraoculaires

Les procédures chirurgicales intraoculaires sont des procédures difficiles par la précision qu’elles demandent, on parle de microchirurgie, mais aussi par la difficulté et la faible qualité de visualisation des tissus à traiter. En effet, dans la plupart des procédures intraoculaires le chirurgien utilise uniquement un microscope ophtalmologique qui ne permet la visualisation des tissus que par la pupille du patient et offre une perception limitée de la profondeur. La Tomographie en Cohérence Optique (OCT) fournit des images en profondeur des tissus sains de manière non invasive, elle est utilisée couramment en diagnostic ophtalmologique et est de plus en plus utilisée intra-opérativement. Dans cette thèse nous allons présenter deux systèmes OCT intra-opératifs qui visent à assister les chirurgiens sur deux procédures intraoculaires, la vitrectomie et l’injection sous-rétinienne. Pour ces deux projets nous avons utilisé le matériel chirurgical utilisé cliniquement pour plusieurs raisons : s’assurer d’utiliser des outils adéquats (dimensions, efficacité, sécurité) pour la procédure, garder des outils que les chirurgiens utilisent régulièrement et avec lesquels ils sont familiers et limiter les coûts de développement. Pour le système OCT nous avons utilisé des sondes OCT en fibre optique car elles sont flexibles, bon marché et de petit diamètre. Leur focalisation peut également être modifiée dépendamment de l’application avec une fibre optique GRIN à leur extrémité pour augmenter le signal OCT. Nous avons ainsi attaché à ces outils chirurgicaux des sondes OCT en fibre optique. Pour le projet portant sur les injections sous-rétiniennes il a fallu dans un premier temps développer des sondes OCT avec des diamètres plus petits que ceux existant. Pour ce faire nous avons développé une méthode permettant de réduire le diamètre des sondes avec de l’acide fluorhydrique et grâce à un design permettant de conserver les propriétés optiques des sondes. Ce travail est présenté dans le premier article. Le second article présente un système permettant de guider les injections sous-rétiniennes. L’injection sous-rétinienne est une intervention chirurgicale de haute précision visant à restaurer et/ou préserver la vision des patients souffrant de maladies rétiniennes. Néanmoins, l’injection sous-rétinienne reste à la limite des capacités physiologiques humaines en raison des tremblements de la main et peut être compromise par le reflux du médicament si l’injection n’est pas assez profonde dans la rétine. Nous avons développé un système pour guider l’injection avec un micromanipulateur et donner des informations précises sur la profondeur au chirurgien avec l’OCT intra-opératif. Après avoir miniaturisé une sonde OCT en fibre optique avec la méthode présentée dans l’article 1 nous avons pu l’insérer dans une canule utilisée cliniquement. La sonde couplée à un système OCT que nous avons développé acquiert un signal A-scan qui va permettre de connaitre la distance entre la canule et la rétine mais aussi de sélectionner la profondeur de l’injection dans les couches rétiniennes. La canule est attachée à un micromanipulateur qui assure son déplacement dans l’œil. Une image M-scan est construite avec le signal OCT et le chirurgien peut directement sélectionner sur l’image la profondeur de l’injection. Nous avons développé l’interface sur Labview. Après avoir sélectionné la cible de l’injection le programme de guidage va déplacer la canule et injecter le volume adéquat grâce à une pompe contrôlable. Nous avons validé notre système de guidage sur des yeux de porcs ex-vivo. Sur 40 injections 38 présentaient un décollement rétinien ciblé et localisé, preuve de la réussie de l’injection rétinienne ce qui représente un taux de succès de 95% (CI : 83.1 – 99.4). Nous avons aussi grâce à un algorithme de traitement de l’image calculé le volume présent sous la rétine après l’injection que nous avons comparé au volume injecté. Nous avons ainsi trouvé que 75% du volume initialement injecté se retrouve bien sous la rétine. Le troisième article présente un système permettant d’arrêter automatiquement le vitrecteur lors d’une vitrectomie pour réduire les dommages accidentels sur la rétine. La survenue de déchirures rétiniennes iatrogèniques dans la vitrectomie par la pars plane est une complication qui compromet l’efficacité globale de la chirurgie. Un certain nombre de déchirures rétiniennes iatrogènes se produisent lorsque la rétine est coupée accidentellement par le vitrecteur. Nous avons développé un vitrecteur intelligent capable de détecter en temps réel une coupure rétinienne accidentelle et de désactiver rapidement la machine de vitrectomie pour les prévenir. Ce vitrecteur intelligent est composé d’une sonde OCT attachée au vitrecteur et va avoir comme rôle de détecter si le vitrecteur aspire la rétine et va endommager ces tissus sains. La sonde OCT agit comme un détecteur de présence devant l’ouverture du vitrecteur, ceci en comparant un signal de référence avec le signal en direct. Cette comparaison de signal OCT va commander un bras robotique pour actionner la pédale d’arrêt du vitrecteur. Ainsi le chirurgien n’a pas besoin d’interpréter un signal, la décision d’arrêt du vitrecteur dû à la présence de la rétine est prise automatiquement. Ceci va permettre de réduire grandement, de 300 ms à 29 ms, le délai de la prise de décision d’arrêt du vitrecteur précédemment limité par le temps de réaction du chirurgien. Nous avons développé les sondes OCT, le système OCT ainsi que l’algorithme d’arrêt automatique de ce système. Nous avons validé sur des yeux porcins in-vivo, deux chirurgiens ont utilisé notre système en essayant d’endommager les tissus rétiniens. 70% (CI : 56.39 – 82.02) des tentatives de dommages rétiniens des chirurgiens furent atténuées ou empêchées par notre système. Ce projet a abouti au dépôt d’un brevet ("Smart Vitrector", Provisional patent application, US 63109040).Intraocular surgical procedures are difficult procedures because of the precision they require, they are often referred as microsurgery, but also by the little information available to the surgeon. In most intraocular procedures the surgeon only uses an ophthalmic microscope which allows visualization of tissue just through the patient’s pupil and offers limited depth perception. Optical Coherence Tomography (OCT) provides in-depth images of healthy tissue in a non-invasive manner, is commonly used in ophthalmologic diagnostics, and is increasingly used intraoperatively. In this thesis we will present two intraoperative OCT systems that aim to assist surgeons with two intraocular procedures, vitrectomy and subretinal injection. For these two projects we used the surgical equipment used clinically for several reasons : to make sure to use adequate tools (dimensions, efficiency, safety) for the procedure, to keep tools that surgeons use regularly and with which they are familiar and limit development costs. For the OCT system we used fiber optic OCT probes as they are flexible, cheap and small in diameter. Their focus can also be modified, depending the application, with a GRIN fiber at their tip to increase the OCT signal. We have attached optical fiber OCT probes to these surgical tools. For the subretinal injections project it was first necessary to develop OCT probes with smaller diameters than existing ones. To do this, we have developed a method to reduce the diameter of the probes with hydrofluoric acid and a design to maintain the optical properties of the probes. This work is presented in the first article. The second article presents a system for guiding subretinal injections. Subretinal injection of drugs is a challenging surgical intervention aiming to restore and/or preserve the vision of patients suffering from retinal diseases. Nevertheless, the subretinal injection remains at the edge of human physiological capacity because of hand tremor and can be mitigated by drug reflux if the injection is not deep enough in the retina. We developed a system to guide the injection with a micromanipulator and give precise depth information to the surgeon with intraoperative OCT. To do so we first miniaturized an optical fiber OCT probe with the method presented in article 1, we were able to insert it into a cannula used clinically. The probe coupled to an OCT system that we have developed acquires an A-scan signal which enables to know the distance between the cannula and the retina but also to select the depth of the injection into the retinal layers. The cannula is attached to a micromanipulator that moves it inside the eye. An M-scan image is built with the OCT signal and the surgeon can directly select on the image the depth of the injection. We developed the interface on Labview. After selecting the injection target, the guidance program will move the cannula and inject the appropriate volume using a controllable pump.We have validated our guidance system on pig eyes ex-vivo. Out of 40 injections, 38 presented a retinal detachment, proof of a successful retinal injection, which represents a success rate of 95% (CI : 83.1 – 99.4). Thanks to an image processing algorithm, we also calculated the bleb volume under the retina after the injection, which we compared to the initial injected volume. We have found that 75% of the injected volume ends in the subretinal space. The third article presents for automatically stopping the vitrector during a vitrectomy. The occurrence of iatrogenic retinal breaks in pars plana vitrectomy is a complication that compromises the overall efficacy of the surgery. A subset of iatrogenic retinal break occurs when the retina is cut accidentally by the vitrector. We developed a smart vitrector that can detect in real-time potential accidental retinal cut and activate promptly a vitrectomy machine to prevent them. To do so an OCT probe is attached to the vitrector and will have the role of detecting if the vitrector sucks the retina and will damage these healthy tissues. The OCT probe acts as a presence detector in front of the vitrector opening, by comparing a reference signal with the live signal. This OCT signal comparison will control a robotic arm to operate the vitrector stop pedal. Thus, the surgeon does not need to interpret a signal, the decision to stop the vitrector due to the presence of the retina is taken automatically. This will greatly reduce, from 300 ms to 29 ms, the delay to stop the vitrector previously limited by the reaction time of the surgeon. We have developed the OCT probes, the OCT system, and the automatic shutdown algorithm for this system. We validated our system on in-vivo porcine eyes, two surgeons used the modified vitrector trying to damage retinal tissue. 70% (CI : 56.39 – 82.02) of surgeons’ retinal damage attempts were mitigated or prevented by our system. This project resulted in a patent ("Smart Vitrector", Provisional patent application, US 63109040)

From teleoperation to autonomous robot-assisted microsurgery: A survey

Robot-assisted microsurgery (RAMS) has many benefits compared to traditional microsurgery. Microsurgical platforms with advanced control strategies, high-quality micro-imaging modalities and micro-sensing systems are worth developing to further enhance the clinical outcomes of RAMS. Within only a few decades, microsurgical robotics has evolved into a rapidly developing research field with increasing attention all over the world. Despite the appreciated benefits, significant challenges remain to be solved. In this review paper, the emerging concepts and achievements of RAMS will be presented. We introduce the development tendency of RAMS from teleoperation to autonomous systems. We highlight the upcoming new research opportunities that require joint efforts from both clinicians and engineers to pursue further outcomes for RAMS in years to come

The Advanced Applications For Optical Coherence Tomography In Skin Imaging

Optical coherence tomography (OCT), based on the principle of interferometry, is a fast and non-invasive imaging modality, which has been approved by FDA for dermatologic applications. OCT has high spatial resolution up to micrometer scale compared to traditional ultrasound imaging. In addition, OCT can provide real-time cross-sectional images with 1 to 2 mm penetration depth, which makes it an ideal imaging technique to assess the skin micro-morphology and pathology without any tissue removal. Many studies have investigated the possibilities of using OCT to evaluate dermatologic conditions, such as skin cancer, dermatitis, psoriasis, and skin damages. Hence, OCT has tremendous potential to provide skin histological and pathological information and assist differential diagnosis of various skin diseases. In this study, we used a swept-source OCT with 1305 nm central wavelength to explore its advanced applications in dermatology. This dissertation consists of four major research projects. First, we explored the feasibility of OCT imaging for assisting real-time visualization in skin biopsy. We showed that OCT could be used to guide and track a needle insertion in mouse skin in real-time. The structure of skin and the movement of needle can be clearly seen on the OCT images without any time delay during the procedures. Next, we tested the concept of performing the punch biopsy using OCT hand-held probe attached to a piercing tip in a phantom. We proved that using the OCT is a reliable technique to delineate the margin of lesion in phantom. And it is possible to perform the punch biopsy with the OCT probe. Second, we tested the performance of contrast-enhanced OCT in melanoma detection in an in vitro study. Melanoma is the most lethal type of skin cancer. Early detection could significantly improve the long-term survival rate of patients. In this initial study, a contrast agent (Gal3-USGNPs) is developed by conjugating melanoma biomarker (Gal3) to ultra-small gold nanoparticles (USGNPs). We showed that the contrast agent can differentiate B16 melanoma cells from normal skin keratinocytes in vitro. To avoid systemic administration of USGNPs, the third project continues to explore the enhanced topical delivery of USGNPs. In this study, we used OCT to monitor the topical delivery of nanoparticles on pig skin over time. And the diffusion and penetration of USGNPs in skin can be improved by applying chemical and physical enhancers such as DMSO and sonophoresis. Finally, in addition to image the cross-sectional structure of skin, we also aim to extract quantitative information from OCT images. The skin optical properties such as attenuation coefficient can be measured from OCT images. We measured and compared the skin attenuation coefficient in the skin of forehead and lateral hip, the skin of three different age groups, and the skin of three different Fitzpatrick types. The statistical analysis showed that epidermis has much higher attenuation coefficient than dermis. And the skin type V & VI have a relatively lower attenuation coefficient than the other skin types. These studies could aid the detection of skin cancer using imaging techniques and provide some new insights into the future applications of OCT in dermatology

Optical coherence tomography (OCT) - guided ophthalmic therapy

In this work, we demonstrate OCT-based guidance of two ophthalmic therapies, subretinal injection and selective retina therapy (SRT). Firstly, the “SMART,” a hand-held robotic surgical device actively guided by a common-path OCT (CP-OCT) distal sensor, improves in two aspects for being applied to subretinal injection: (i) A high-performance fiber probe based on high index epoxy lensed-fiber to enhance the CP-OCT retinal image quality; (ii) Automated retinal layer identification and tracking : retinal layer boundaries are tracked using convolutional neural network (CNN)-based segmentation for accurate subretinal injection guidance. It is shown that high index epoxy lensed-fiber probe improves the SNR and retinal image quality of the CP-OCT system. We propose and implement real-time retinal boundary tracking of A-scan OCT images using CNNs for accurate localization of a surgical tool tip. Unwanted axial motions of the surgical tools are compensated by a piezo-electric linear motor based on the retinal boundary tracking. A CNN-based CP-OCT distal sensor successfully tracks retinal boundaries, especially the PR/CH boundary for subretinal injection, and automatically guides the needle’s axial position in real-time. The micro-scale depth targeting accuracy of our system shows its promising possibility for clinical application. We also propose and demonstrate SRT monitoring based on speckle variance OCT (svOCT) for dosimetry control. M-scans of a phantom, ex vivo bovine iris, and ex vivo bovine retina are obtained by a swept-source OCT system during laser pulses irradiation. SvOCT images are calculated as interframe intensity variance of the sequence, and they show abrupt speckle variance change induced by laser pulse irradiation. The axially averaged svOCT signals show a sharp peak corresponding to each laser pulse, and the peak values are proportional to irradiated laser pulse energy. For the ex vivo retinal study, microscopic images of treated spots are obtained before and after removing the upper neural retinal layer to assess the damage in both RPE and neural layers. Spatial and temporal temperature distributions in the retina are numerically calculated in a 2D retinal model using COMSOL Multiphysics. We find that the svOCT peak values have a reliable correlation with the degree of retinal lesion formation. The temperature at the neural retina and RPE is estimated from the svOCT peak values using numerically calculated temperature, which is consistent with the observed lesion creation