Spectrally encoded fiber-based structured lighting probe for intraoperative 3D imaging

Three dimensional quantification of organ shape and structure during minimally invasive surgery (MIS) could enhance precision by allowing the registration of multi-modal or pre-operative image data (US/MRI/CT) with the live optical image. Structured illumination is one technique to obtain 3D information through the projection of a known pattern onto the tissue, although currently these systems tend to be used only for macroscopic imaging or open procedures rather than in endoscopy. To account for occlusions, where a projected feature may be hidden from view and/or confused with a neighboring point, a flexible multispectral structured illumination probe has been developed that labels each projected point with a specific wavelength using a supercontinuum laser. When imaged by a standard endoscope camera they can then be segmented using their RGB values, and their 3D coordinates calculated after camera calibration. The probe itself is sufficiently small (1.7 mm diameter) to allow it to be used in the biopsy channel of commonly used medical endoscopes. Surgical robots could therefore also employ this technology to solve navigation and visualization problems in MIS, and help to develop advanced surgical procedures such as natural orifice translumenal endoscopic surgery

Spatial frequency domain imaging towards improved detection of gastrointestinal cancers

Early detection and treatment of gastrointestinal cancers has been shown to drastically improve patients survival rates. However, wide population based screening for gastrointestinal cancers is not feasible due to its high cost, risk of potential complications, and time consuming nature. This thesis forms the proposal for the development of a cost-effective, minimally invasive device to return quantitative tissue information for gastrointestinal cancer detection in-vivo using spatial frequency domain imaging (SFDI). SFDI is a non-invasive imaging technique which can return close to real time maps of absorption and reduced scattering coefficients by projecting a 2D sinusoidal pattern onto a sample of interest. First a low-cost, conventional bench top system was constructed to characterise tissue mimicking phantoms. Phantoms were fabricated with specific absorption and reduced scattering coefficients, mimicking the variation in optical properties typically seen in healthy, cancerous, and pre-cancerous oesophageal tissue. The system shows accurate retrieval of absorption and reduced scattering coefficients of 19% and 11% error respectively. However, this bench top system consists of a bulky projector and is therefore not feasible for in-vivo imaging. For SFDI systems to be feasible for in-vivo imaging, they are required to be miniaturised. Many conditions must be considered when doing this such as various illumination conditions, lighting conditions and system geometries. Therefore to aid in the miniaturisation of the bench top system, an SFDI system was simulated in the open-source ray tracing software Blender, where the capability to simulate these conditions is possible. A material of tunable absorption and scattering properties was characterised such that the specific absorption and reduced scattering coefficients of the material were known. The simulated system shows capability in detecting optical properties of typical gastrointestinal conditions in an up-close, planar geometry, as well in a non-planar geometry of a tube simulating a lumen. Optical property imaging in the non-planar, tubular geometry was done with the use of a novel illumination pattern, developed for this work. Finally, using the knowledge gained from the simulation model, the bench top system was miniaturised to a 3 mm diameter prototype. The novel use of a fiber array producing the necessary interfering fringe patterns replaced the bulky projector. The system showed capability to image phantoms simulating typical gastrointestinal conditions at two wavelengths (515 and 660 nm), measuring absorption and reduced scattering coefficients with 15% and 6% accuracy in comparison to the bench top system for the fabricated phantoms. It is proposed that this system may be used for cost-effective, minimally invasive, quantitative imaging of the gastrointestinal tract in-vivo, providing enhanced contrast for difficult to detect cancers

Advanced endoscopic techniques and optical diagnosis in the lower gastrointestinal tract

Inflammatory bowel disease describes debilitating chronic diseases (Ulcerative colitis and Crohn’s Disease) of the gastrointestinal tract that requires many patients to take long term medications, have an increased risk of malignancy and can often lead to major abdominal surgery. With the advent of modern therapies, in particular biological therapies, clinicians and patients can increasingly achieve mucosal healing. Mucosal healing is associated with favourable outcomes such as reduced hospitalisation, colectomy and fewer courses of steroids. However, the definition of mucosal healing was based on endoscopic scoring systems developed in a previous generation of endoscopic technology. Novel endoscopic technology, such as Virtual Electronic Chromoendoscopy, can gain more accurate assessments of inflammatory activity, closing the gap to histological activity. The term optical diagnosis refers to the ability of an endoscopist to accurately predict the histology of an endoscopic finding, albeit in predicting inflammatory activity in inflammatory bowel disease or in predicting the histology of a colorectal polyp. With ever improving endoscopic technology successful optical diagnosis is increasingly possible. The PICaSSO score is a score that was developed using Virtual Electronic Chromoendoscopy and is the first to define endoscopic features in keeping with mucosal healing. This thesis includes a large multicentre international prospective study which investigates the correlation of the PICaSSO score and other established endoscopic scores (MES and UCEIS) against multiple histological indices (RHI, NHI, ECAP, Geboes, and Villanacci) and prospectively assessed outcomes at 6 and 12 months. There was strong correlation between PICaSSO and histology scores, significantly superior to correlation coefficients of MES and UCEIS with histology scores. A PICaSSO score of ≤3 detected histologic remission by RHI with AUROC 0.90 (95% CI 0.86-0.94). PICaSSO score ≤3 predicted better outcomes than PICaSSO >3. The next study moves from optical characterisation to molecular characterisation of inflammatory activity using Raman Spectroscopy. Raman Spectroscopy (RS) describes the scattering of inelastic light giving spectra that are highly specific for individual molecules. This study aimed to establish spectral changes before and after treatment and whether Raman Spectroscopy can accurately differentiate between inflammation and MH. Reductions in intensity at 1003cm-1 and 1252cm-1 when a reduction in inflammation was seen post-treatment and when MH was present. MH was associated with an increase in intensity at 1304cm-1. A trained neural network differentiated MH from active inflammation with high sensitivity, specificity, PPV, NPV and accuracy in UC and CD. Raman Spectroscopy may represent an additional tool in the assessment of mucosal healing in IBD. To implement optical diagnosis in practice there needs to be robust training. The next study presents a randomised controlled study comparing the performance of self-training vs. didactic training on the diagnostic accuracy of diminutive/small colonic polyp histological prediction by trainees using established polyp classification tools. The study showed self-learning can achieve results similar to didactic training which, could enable widespread implementation of optical diagnosis in clinical practice. Following this study, I present a meta-analysis and systematic review of optical diagnosis training in small/diminutive colorectal polyps. Optical diagnosis training is effective in improving accuracy of histology prediction in colorectal polyps and didactic and computer-based training show comparable effectiveness in improving diagnostic accuracy. The final study presented is implementing optical diagnosis training and setting quality standards in IBD surveillance endoscopy. A significant improvement in quality was seen in the proportion of procedures using dye-based chromoendoscopy, use of polyp classification tools and an increase lesion detection

Real-time tissue viability assessment using near-infrared light

Despite significant advances in medical imaging technologies, there currently exist no tools to effectively assist healthcare professionals during surgical procedures. In turn, procedures remain subjective and dependent on experience, resulting in avoidable failure and significant quality of care disparities across hospitals. Optical techniques are gaining popularity in clinical research because they are low cost, non-invasive, portable, and can retrieve both fluorescence and endogenous contrast information, providing physiological information relative to perfusion, oxygenation, metabolism, hydration, and sub-cellular content. Near-infrared (NIR) light is especially well suited for biological tissue and does not cause tissue damage from ionizing radiation or heat. My dissertation has been focused on developing rapid imaging techniques for mapping endogenous tissue constituents to aid surgical guidance. These techniques allow, for the first time, video-rate quantitative acquisition over a large field of view (> 100 cm2) in widefield and endoscopic implementations. The optical system analysis has been focused on the spatial-frequency domain for its ease of quantitative measurements over large fields of view and for its recent development in real-time acquisition, single snapshot of optical properties (SSOP) imaging. Using these methods, this dissertation provides novel improvements and implementations to SSOP, including both widefield and endoscopic instrumentations capable of video-rate acquisition of optical properties and sample surface profile maps. In turn, these measures generate profile-corrected maps of hemoglobin concentration that are highly beneficial for perfusion and overall tissue viability. Also utilizing optical property maps, a novel technique for quantitative fluorescence imaging was also demonstrated, showing large improvement over standard and ratiometric methods. To enable real-time feedback, rapid processing algorithms were designed using lookup tables that provide a 100x improvement in processing speed. Finally, these techniques were demonstrated in vivo to investigate their ability for early detection of tissue failure due to ischemia. Both pre-clinical studies show endogenous contrast imaging can provide early measures of future tissue viability. The goal of this work has been to provide the foundation for real-time imaging systems that provide tissue constituent quantification for tissue viability assessments.2018-01-09T00:00:00

Development of Optical Devices for Digital Medicine

Department of Biomedical EngineeringAdvances of technology have made a revolution that interconnects industrial devices and fuses the boundaries of digital, physical and biological spaces. These technologies such as cloud computing, 3D printing technology, big data, internet of things (IOT), artificial intelligence (AI), and maturity of system integrations have been improved every year, changing our daily life quickly in intelligent and convenient ways. In this days, these explosions of technology, changing the way we live and think, is referred to 4th industrial revolution. As we know, every industry is affected by the new waves of technologies, digitalization and connectivity, and the biomedical or medical field is no exception. Healthcare fields have benefited mostly from recent technical improvements, revolutionizing the medical systems in many terms in cost-effective ways. Particularly, ???digital medicine??? has been recently came into the limelight as one of the uprising fields. In digital medicine, traditional medical devices and diagnostic programs have become miniaturized, digitalized, and automated. As taking advantages of digital medicine, specific fields related to digital pathology, point-of-care (POC) diagnostics, and application of deep learning or machine learning technologies have shown the great potentials not only in biomedical academia but also in the revenues of their markets. It allows to connect devices, hospital equipment, and to accelerate efficiencies in health service such as diagnosis, and to reduce the cost of services. Moreover, interconnection between advanced technologies has been improved the access of healthcare to the places where hospital or medical services are limited. Furthermore, artificial intelligence has shown promising results related to disease screening especially using medical images. Although fields in digital medicine are prospering, still there are limitations that needs to be overcome in order to provide further advanced health services to patients in the various situations. In digital pathology, improvements of microscopic technologies, internets, and storage capabilities have reduced the time-consuming processes. The simple transformation of microscopic image to digital have successfully alternated many limitations in the analogue histopathology workflow to efficient and cost saving ways. However, tissue staining is currently referred as one of the bottleneck that makes workflow still lengthy, labor-intensive, and costly. In the POC diagnostic fields, various digitalized portable smartphone-based diagnostic devices have been introduced as alternatives to conventional medical services. These devices have provided the quality assurance of diagnostics by taking advantages of sharing, and quantitative analysis of digital information. However, most of these works have been focused on replacing diagnostic process which mostly done in laboratory settings. As medical imaging devices and trained clinicians or practitioners are limited, there are also high demands on clinical imaging-based diagnostics in developing countries. In this thesis, computational microscope using patterned NIR illumination was developed for label-free quantitative differential phase tissue imaging to bypass the staining process of the pathology workflow. This system overcame the limitations found in the conventional quantitative differential phase contrast in a LED array microscope, allowing to captured light scattering and absorbing specimen while maintaining weak object approximation. Moreover, portable endoscope system was developed integrating the additive production technologies (3D printing), ICT, and optics for POC diagnostics. This innovative POC endoscope demonstrated comparable imaging capability to that of commercialized clinical endoscope system. Furthermore, deep learning and machine learning models have been trained and applied to each devices, respectively. Generative adversarial network (GAN) was applied to our NIR-based QPI system to virtually stain the label-free QPI which look comparable to image that is captured from bright field microscope using labeled tissue. Lastly, POC automated cervical cancer screening system was developed utilizing smartphone-based endoscope system as well as training the machine learning algorithm. 3-5% of acetic acid was applied to the suspicious lesion and its reaction was captured before and after application using smartphone endoscope. This screening system enables to extract the features of cancers and informs the possibility of cancer from endoscopic images.clos

3D Textured Surface Reconstruction from Endoscopic Video

Endoscopy enables high-resolution visualization of tissue texture and is a critical step in many clinical workflows, including diagnosis of infections, tumors or diseases and treatment planning for cancers. This includes my target problems of radiation treatment planning in the nasopharynx and pre-cancerous polyps screening and treatment in colonoscopy. However, an endoscopic video does not provide its information in 3D space, making it difficult to use for tumor localization, and it is inefficient to review. In addition, when there are incomplete camera observations of the organ surface, full surface coverage cannot be guaranteed in an endoscopic procedure, and unsurveyed regions can hardly be noticed in a continuous first-person perspective. This dissertation introduces a new imaging approach that we call endoscopography: an endoscopic video is reconstructed into a full 3D textured surface, which we call an endoscopogram. In this dissertation, I present two endoscopography techniques. One method is a combination of a frame-by-frame algorithmic 3D reconstruction method and a groupwise deformable surface registration method. My contribution is the innovative combination of the two methods that improves the temporal consistency of the frame-by-frame 3D reconstruction algorithm and eliminates the manual intervention that was needed in the deformable surface registration method. The combined method reconstructs an endoscopogram in an offline manner, and the information contained in the tissue texture in the endoscopogram can be transferred to a 3D image such as CT through a surface-to-surface registration. Then, through an interactive tool, the physician can draw directly on the endoscopogram surface to specify a tumor, which then can be automatically transferred to CT slices to aid tumor localization. The second method is a novel deep-learning-driven dense SLAM (simultaneous localization and mapping) system, called RNN-SLAM, that in real time can produce an endoscopogram with display of the unsurveyed regions. In particular, my contribution is the deep learning system in the RNN-SLAM, called RNN-DP. RNN-DP is a novel multi-view dense depth map and odometry estimation method that uses Recurrent Neural Networks (RNN) and trains utilizing multi-view image reprojection and forward-backward flow-consistency losses.Doctor of Philosoph

Role of image-enhanced endoscopy in the assessment of inflammatory bowel disease.

Abstract Introduction: Ulcerative colitis (UC) which is a form of inflammatory bowel disease (IBD) is characterised by a relapsing and remitting disease course. Clinical disease activity indices (DAIs) are used to assess the severity of the disease activity relying solely on the clinical symptomatology of the patients. Non-invasive biomarkers help in assessment and possibly predicting the disease relapse. Although faecal calprotectin (FCP) is one such biomarker that is extensively researched, its accuracy in assessment and prediction of relapse is only modest. Similarly endoscopy in IBD with white light examination (WLE) alone is not accurate in either the assessment of disease activity or the prediction of disease course. Narrow band imaging (NBI) allows examination of the vasculature and pit pattern of the mucosa in greater detail than WLE. Patients with colonic IBD also have a higher risk of developing dysplasia or colorectal cancer (CRC). Chromoendoscopy (CE) provides a contrast enhancement and aids in highlighting the dysplastic areas. Aims: Primary aim of the research is to assess the role of advanced endoscopy, NBI and Chromoendoscopy (CE) in assessment of disease activity and dysplasia detection respectively in UC. The secondary aim is to assess the role of DAIs in assessment of disease activity, their correlation with endoscopic & histological markers and overall outcomes during the follow up period. Methods We performed two different experiments using advanced endoscopic techniques for this research project; one is in assessment of inflammatory activity and second is in detection of dysplasia in UC. We performed retrospective analysis of our practice to identify if white light alone predicts relapse in patients with quiescent UC. Based on our findings we devised a prospective observational study to look at the effect of adding NBI to WLE in assessment of disease activity in patients with UC of varying grades of severity. As newer generation of NBI (H290 series of Olympus KeyMed®) endoscopes were being introduced into the UK market at the time of the study, we compared the effect of NBI in three generations of endoscope (Q240, H260 and H290 series). We also assessed the use of Raman spectroscopy in endoscopic and histological assessment of inflammation in UC. In another retrospective study we looked at the uptake of chromoendoscopy in surveillance colonoscopies in UC. A randomised controlled study (RCT) was also designed to compare high definition WLE (HDWLE) to high definition CE (HDCE) in detecting dysplasia in UC surveillance. As part of relapse-prediction work we also conducted a meta-analysis of published RCTs on FCP to analyse its predictive capability in IBD. Results: In the retrospective analysis, we found that the presence of either Mayo Endoscopic Subscore >1 or Geboes score ≥2.1, increases the risk of relapse up to 6 times in the subsequent twelve months period. In our comparative study of NBI in three different generations of endoscopes, we demonstrated that NBI is superior to WLE in the assessment of the presence of blood. We also noticed a significant improvement in NBI in the newer generation of endoscopes (H290 and H260) compared to the earlier endoscopes (Q240). From the meta-analysis of RCTs we found that the FCP can predict disease flare with an accuracy of up to 75% only. In the observational study we determined that addition of NBI to WLE did not provide additional value in either assessment of disease activity or predicting relapse. Among the clinical disease activity indices (DAIs), the simple clinical colitis index or Walmsley index with score of ≥3 correlated well with endoscopy and histological findings. From the Raman spectroscopy study we identified the intensities of peaks (carotenoid and the phospholipids) that were statistically significantly different between the Raman spectra of the inflamed and quiescent colonic tissue. In our second retrospective analysis CE was found to be superior to WLE in detecting all dysplastic lesions and the detection of endoscopically visible flat non-polypoid lesions. However CE was performed only in one third of the study population. In the RCT we found that HDCE has an incremental yield of about 12.7% with a NNT of about 8, suggesting that HDCE would detect one additional patient with a dysplastic lesion for every 8 patients on whom this procedure is done. Conclusion: The thesis has shown that endoscopic biomarkers and FCP do not reliably predict relapse in UC. Addition of NBI does not confer added benefit in assessment of disease activity. HDCE is superior to HDWLE and should be adapted as a standard practice in surveillance of dysplasia in UC

Towards Quantitative Endoscopy with Vision Intelligence

In this thesis, we work on topics related to quantitative endoscopy with vision-based intelligence. Specifically, our works revolve around the topic of video reconstruction in endoscopy, where many challenges exist, such as texture scarceness, illumination variation, multimodality, etc., and these prevent prior works from working effectively and robustly. To this end, we propose to combine the strength of expressivity of deep learning approaches and the rigorousness and accuracy of non-linear optimization algorithms to develop a series of methods to confront such challenges towards quantitative endoscopy. We first propose a retrospective sparse reconstruction method that can estimate a high-accuracy and density point cloud and high-completeness camera trajectory from a monocular endoscopic video with state-of-the-art performance. To enable this, replacing the role of a hand-crafted local descriptor, a deep image feature descriptor is developed to boost the feature matching performance in a typical sparse reconstruction algorithm. A retrospective surface reconstruction pipeline is then proposed to estimate a textured surface model from a monocular endoscopic video, where self-supervised depth and descriptor learning and surface fusion technique is involved. We show that the proposed method performs superior to a popular dense reconstruction method and the estimate reconstructions are in good agreement with the surface models obtained from CT scans. To align video-reconstructed surface models with pre-operative imaging such as CT, we introduce a global point cloud registration algorithm that is robust to resolution mismatch that often happens in such multi-modal scenarios. Specifically, a geometric feature descriptor is developed where a novel network normalization technique is used to help a 3D network produce more consistent and distinctive geometric features for samples with different resolutions. The proposed geometric descriptor achieves state-of-the-art performance, based on our evaluation. Last but not least, a real-time SLAM system that estimates a surface geometry and camera trajectory from a monocular endoscopic video is developed, where deep representations for geometry and appearance and non-linear factor graph optimization are used. We show that the proposed SLAM system performs favorably compared with a state-of-the-art feature-based SLAM system