Development of Optical Devices for Digital Medicine

Department of Biomedical EngineeringAdvances of technology have made a revolution that interconnects industrial devices and fuses the boundaries of digital, physical and biological spaces. These technologies such as cloud computing, 3D printing technology, big data, internet of things (IOT), artificial intelligence (AI), and maturity of system integrations have been improved every year, changing our daily life quickly in intelligent and convenient ways. In this days, these explosions of technology, changing the way we live and think, is referred to 4th industrial revolution. As we know, every industry is affected by the new waves of technologies, digitalization and connectivity, and the biomedical or medical field is no exception. Healthcare fields have benefited mostly from recent technical improvements, revolutionizing the medical systems in many terms in cost-effective ways. Particularly, ???digital medicine??? has been recently came into the limelight as one of the uprising fields. In digital medicine, traditional medical devices and diagnostic programs have become miniaturized, digitalized, and automated. As taking advantages of digital medicine, specific fields related to digital pathology, point-of-care (POC) diagnostics, and application of deep learning or machine learning technologies have shown the great potentials not only in biomedical academia but also in the revenues of their markets. It allows to connect devices, hospital equipment, and to accelerate efficiencies in health service such as diagnosis, and to reduce the cost of services. Moreover, interconnection between advanced technologies has been improved the access of healthcare to the places where hospital or medical services are limited. Furthermore, artificial intelligence has shown promising results related to disease screening especially using medical images. Although fields in digital medicine are prospering, still there are limitations that needs to be overcome in order to provide further advanced health services to patients in the various situations. In digital pathology, improvements of microscopic technologies, internets, and storage capabilities have reduced the time-consuming processes. The simple transformation of microscopic image to digital have successfully alternated many limitations in the analogue histopathology workflow to efficient and cost saving ways. However, tissue staining is currently referred as one of the bottleneck that makes workflow still lengthy, labor-intensive, and costly. In the POC diagnostic fields, various digitalized portable smartphone-based diagnostic devices have been introduced as alternatives to conventional medical services. These devices have provided the quality assurance of diagnostics by taking advantages of sharing, and quantitative analysis of digital information. However, most of these works have been focused on replacing diagnostic process which mostly done in laboratory settings. As medical imaging devices and trained clinicians or practitioners are limited, there are also high demands on clinical imaging-based diagnostics in developing countries. In this thesis, computational microscope using patterned NIR illumination was developed for label-free quantitative differential phase tissue imaging to bypass the staining process of the pathology workflow. This system overcame the limitations found in the conventional quantitative differential phase contrast in a LED array microscope, allowing to captured light scattering and absorbing specimen while maintaining weak object approximation. Moreover, portable endoscope system was developed integrating the additive production technologies (3D printing), ICT, and optics for POC diagnostics. This innovative POC endoscope demonstrated comparable imaging capability to that of commercialized clinical endoscope system. Furthermore, deep learning and machine learning models have been trained and applied to each devices, respectively. Generative adversarial network (GAN) was applied to our NIR-based QPI system to virtually stain the label-free QPI which look comparable to image that is captured from bright field microscope using labeled tissue. Lastly, POC automated cervical cancer screening system was developed utilizing smartphone-based endoscope system as well as training the machine learning algorithm. 3-5% of acetic acid was applied to the suspicious lesion and its reaction was captured before and after application using smartphone endoscope. This screening system enables to extract the features of cancers and informs the possibility of cancer from endoscopic images.clos

Methods and application areas of endoscopic optical coherence tomography

We review the current state of research in endoscopic optical coherence tomography (OCT). We first survey the range of available endoscopic optical imaging techniques. We then discuss the various OCT-based endoscopic methods that have thus far been developed. We compare the different endoscopic OCT methods in terms of their scan performance. Next, we examine the application range of endoscopic OCT methods. In particular, we look at the reported utility of the methods in digestive, intravascular, respiratory, urinary and reproductive systems. We highlight two additional applications—biopsy procedures and neurosurgery—where sufficiently compact OCT-based endoscopes can have significant clinical impacts

Towards Developing Computer Vision Algorithms and Architectures for Real-world Applications

abstract: Computer vision technology automatically extracts high level, meaningful information from visual data such as images or videos, and the object recognition and detection algorithms are essential in most computer vision applications. In this dissertation, we focus on developing algorithms used for real life computer vision applications, presenting innovative algorithms for object segmentation and feature extraction for objects and actions recognition in video data, and sparse feature selection algorithms for medical image analysis, as well as automated feature extraction using convolutional neural network for blood cancer grading. To detect and classify objects in video, the objects have to be separated from the background, and then the discriminant features are extracted from the region of interest before feeding to a classifier. Effective object segmentation and feature extraction are often application specific, and posing major challenges for object detection and classification tasks. In this dissertation, we address effective object flow based ROI generation algorithm for segmenting moving objects in video data, which can be applied in surveillance and self driving vehicle areas. Optical flow can also be used as features in human action recognition algorithm, and we present using optical flow feature in pre-trained convolutional neural network to improve performance of human action recognition algorithms. Both algorithms outperform the state-of-the-arts at their time. Medical images and videos pose unique challenges for image understanding mainly due to the fact that the tissues and cells are often irregularly shaped, colored, and textured, and hand selecting most discriminant features is often difficult, thus an automated feature selection method is desired. Sparse learning is a technique to extract the most discriminant and representative features from raw visual data. However, sparse learning with \textit{L1} regularization only takes the sparsity in feature dimension into consideration; we improve the algorithm so it selects the type of features as well; less important or noisy feature types are entirely removed from the feature set. We demonstrate this algorithm to analyze the endoscopy images to detect unhealthy abnormalities in esophagus and stomach, such as ulcer and cancer. Besides sparsity constraint, other application specific constraints and prior knowledge may also need to be incorporated in the loss function in sparse learning to obtain the desired results. We demonstrate how to incorporate similar-inhibition constraint, gaze and attention prior in sparse dictionary selection for gastroscopic video summarization that enable intelligent key frame extraction from gastroscopic video data. With recent advancement in multi-layer neural networks, the automatic end-to-end feature learning becomes feasible. Convolutional neural network mimics the mammal visual cortex and can extract most discriminant features automatically from training samples. We present using convolutinal neural network with hierarchical classifier to grade the severity of Follicular Lymphoma, a type of blood cancer, and it reaches 91\% accuracy, on par with analysis by expert pathologists. Developing real world computer vision applications is more than just developing core vision algorithms to extract and understand information from visual data; it is also subject to many practical requirements and constraints, such as hardware and computing infrastructure, cost, robustness to lighting changes and deformation, ease of use and deployment, etc.The general processing pipeline and system architecture for the computer vision based applications share many similar design principles and architecture. We developed common processing components and a generic framework for computer vision application, and a versatile scale adaptive template matching algorithm for object detection. We demonstrate the design principle and best practices by developing and deploying a complete computer vision application in real life, building a multi-channel water level monitoring system, where the techniques and design methodology can be generalized to other real life applications. The general software engineering principles, such as modularity, abstraction, robust to requirement change, generality, etc., are all demonstrated in this research.Dissertation/ThesisDoctoral Dissertation Computer Science 201

Surface analysis and visualization from multi-light image collections

Multi-Light Image Collections (MLICs) are stacks of photos of a scene acquired with a fixed viewpoint and a varying surface illumination that provides large amounts of visual and geometric information. Over the last decades, a wide variety of methods have been devised to extract information from MLICs and have shown its use in different application domains to support daily activities. In this thesis, we present methods that leverage a MLICs for surface analysis and visualization. First, we provide background information: acquisition setup, light calibration and application areas where MLICs have been successfully used for the research of daily analysis work. Following, we discuss the use of MLIC for surface visualization and analysis and available tools used to support the analysis. Here, we discuss methods that strive to support the direct exploration of the captured MLIC, methods that generate relightable models from MLIC, non-photorealistic visualization methods that rely on MLIC, methods that estimate normal map from MLIC and we point out visualization tools used to do MLIC analysis. In chapter 3 we propose novel benchmark datasets (RealRTI, SynthRTI and SynthPS) that can be used to evaluate algorithms that rely on MLIC and discusses available benchmark for validation of photometric algorithms that can be also used to validate other MLIC-based algorithms. In chapter 4, we evaluate the performance of different photometric stereo algorithms using SynthPS for cultural heritage applications. RealRTI and SynthRTI have been used to evaluate the performance of (Neural)RTI method. Then, in chapter 5, we present a neural network-based RTI method, aka NeuralRTI, a framework for pixel-based encoding and relighting of RTI data. In this method using a simple autoencoder architecture, we show that it is possible to obtain a highly compressed representation that better preserves the original information and provides increased quality of virtual images relighted from novel directions, particularly in the case of challenging glossy materials. Finally, in chapter 6, we present a method for the detection of crack on the surface of paintings from multi-light image acquisitions and that can be used as well on single images and conclude our presentation

Parallel centerline extraction on the GPU

Centerline extraction is important in a variety of visualization applications including shape analysis, geometry processing, and virtual endoscopy. Centerlines allow accurate measurements of length along winding tubular structures, assist automatic virtual navigation, and provide a path-planning system to control the movement and orientation of a virtual camera. However, efficiently computing centerlines with the desired accuracy has been a major challenge. Existing centerline methods are either not fast enough or not accurate enough for interactive application to complex 3D shapes. Some methods based on distance mapping are accurate, but these are sequential algorithms which have limited performance when running on the CPU. To our knowledge, there is no accurate parallel centerline algorithm that can take advantage of modern many-core parallel computing resources, such as GPUs, to perform automatic centerline extraction from large data volumes at interactive speed and with high accuracy. In this paper, we present a new parallel centerline extraction algorithm suitable for implementation on a GPU to produce highly accurate, 26-connected, one-voxel-thick centerlines at interactive speed. The resulting centerlines are as accurate as those produced by a state-of-the-art sequential CPU method [40], while being computed hundreds of times faster. Applications to fly through path planning and virtual endoscopy are discussed. Experimental results demonstrating centeredness, robustness and efficiency are presented

Discrete Wavelet Transforms

The discrete wavelet transform (DWT) algorithms have a firm position in processing of signals in several areas of research and industry. As DWT provides both octave-scale frequency and spatial timing of the analyzed signal, it is constantly used to solve and treat more and more advanced problems. The present book: Discrete Wavelet Transforms: Algorithms and Applications reviews the recent progress in discrete wavelet transform algorithms and applications. The book covers a wide range of methods (e.g. lifting, shift invariance, multi-scale analysis) for constructing DWTs. The book chapters are organized into four major parts. Part I describes the progress in hardware implementations of the DWT algorithms. Applications include multitone modulation for ADSL and equalization techniques, a scalable architecture for FPGA-implementation, lifting based algorithm for VLSI implementation, comparison between DWT and FFT based OFDM and modified SPIHT codec. Part II addresses image processing algorithms such as multiresolution approach for edge detection, low bit rate image compression, low complexity implementation of CQF wavelets and compression of multi-component images. Part III focuses watermaking DWT algorithms. Finally, Part IV describes shift invariant DWTs, DC lossless property, DWT based analysis and estimation of colored noise and an application of the wavelet Galerkin method. The chapters of the present book consist of both tutorial and highly advanced material. Therefore, the book is intended to be a reference text for graduate students and researchers to obtain state-of-the-art knowledge on specific applications

Methods and application areas of endoscopic optical coherence tomography

We review the current state of research in endoscopic optical coherence tomography (OCT). We first survey the range of available endoscopic optical imaging techniques. We then discuss the various OCT-based endoscopic methods that have thus far been developed. We compare the different endoscopic OCT methods in terms of their scan performance. Next, we examine the application range of endoscopic OCT methods. In particular, we look at the reported utility of the methods in digestive, intravascular, respiratory, urinary and reproductive systems. We highlight two additional applications—biopsy procedures and neurosurgery—where sufficiently compact OCT-based endoscopes can have significant clinical impacts