Spinning Pupil Aberration Measurement for anisoplanatic deconvolution

The aberrations in an optical microscope are commonly measured and corrected at one location in the field of view, within the so-called isoplanatic patch. Full-field correction is desirable for high-resolution imaging of large specimens. Here we present a novel wavefront detector, based on pupil sampling with sub-apertures, which measures the aberrated wavefront phase at each position of the specimen. Based on this measurement, we propose a region-wise deconvolution that provides an anisoplanatic reconstruction of the sample image. Our results indicate that the measurement and correction of the aberrations can be performed in a wide-field fluorescence microscope over its entire field of view.Comment: 5 pages, 4 figure

Deep Generative Modeling Based Retinal Image Analysis

In the recent past, deep learning algorithms have been widely used in retinal image analysis (fundus and OCT) to perform tasks like segmentation and classification. But to build robust and highly efficient deep learning models amount of the training images, the quality of the training images is extremely necessary. The quality of an image is also an extremely important factor for the clinical diagnosis of different diseases. The main aim of this thesis is to explore two relatively under-explored area of retinal image analysis, namely, the retinal image quality enhancement and artificial image synthesis. In this thesis, we proposed a series of deep generative modeling based algorithms to perform these above-mentioned tasks. From a mathematical perspective, the generative model is a statistical model of the joint probability distribution between an observable variable and a target variable. The generative adversarial network (GAN), variational auto-encoder(VAE) are some popular generative models. Generative models can be used to generate new samples from a given distribution. The OCT images have inherent speckle noise in it, fundus images do not suffer from noises in general, but the newly developed tele-ophthalmoscope devices produce images with relatively low spatial resolution and blur. Different GAN based algorithms were developed to generate corresponding high-quality images fro its low-quality counterpart. A combination of residual VAE and GAN was implemented to generate artificial retinal fundus images with their corresponding artificial blood vessel segmentation maps. This will not only help to generate new training images as many as needed but also will help to reduce the privacy issue of releasing personal medical data

Deep learning-based diagnostic system for malignant liver detection

Cancer is the second most common cause of death of human beings, whereas liver cancer is the fifth most common cause of mortality. The prevention of deadly diseases in living beings requires timely, independent, accurate, and robust detection of ailment by a computer-aided diagnostic (CAD) system. Executing such intelligent CAD requires some preliminary steps, including preprocessing, attribute analysis, and identification. In recent studies, conventional techniques have been used to develop computer-aided diagnosis algorithms. However, such traditional methods could immensely affect the structural properties of processed images with inconsistent performance due to variable shape and size of region-of-interest. Moreover, the unavailability of sufficient datasets makes the performance of the proposed methods doubtful for commercial use. To address these limitations, I propose novel methodologies in this dissertation. First, I modified a generative adversarial network to perform deblurring and contrast adjustment on computed tomography (CT) scans. Second, I designed a deep neural network with a novel loss function for fully automatic precise segmentation of liver and lesions from CT scans. Third, I developed a multi-modal deep neural network to integrate pathological data with imaging data to perform computer-aided diagnosis for malignant liver detection. The dissertation starts with background information that discusses the proposed study objectives and the workflow. Afterward, Chapter 2 reviews a general schematic for developing a computer-aided algorithm, including image acquisition techniques, preprocessing steps, feature extraction approaches, and machine learning-based prediction methods. The first study proposed in Chapter 3 discusses blurred images and their possible effects on classification. A novel multi-scale GAN network with residual image learning is proposed to deblur images. The second method in Chapter 4 addresses the issue of low-contrast CT scan images. A multi-level GAN is utilized to enhance images with well-contrast regions. Thus, the enhanced images improve the cancer diagnosis performance. Chapter 5 proposes a deep neural network for the segmentation of liver and lesions from abdominal CT scan images. A modified Unet with a novel loss function can precisely segment minute lesions. Similarly, Chapter 6 introduces a multi-modal approach for liver cancer variants diagnosis. The pathological data are integrated with CT scan images to diagnose liver cancer variants. In summary, this dissertation presents novel algorithms for preprocessing and disease detection. Furthermore, the comparative analysis validates the effectiveness of proposed methods in computer-aided diagnosis

RFormer: Transformer-based Generative Adversarial Network for Real Fundus Image Restoration on A New Clinical Benchmark

Ophthalmologists have used fundus images to screen and diagnose eye diseases. However, different equipments and ophthalmologists pose large variations to the quality of fundus images. Low-quality (LQ) degraded fundus images easily lead to uncertainty in clinical screening and generally increase the risk of misdiagnosis. Thus, real fundus image restoration is worth studying. Unfortunately, real clinical benchmark has not been explored for this task so far. In this paper, we investigate the real clinical fundus image restoration problem. Firstly, We establish a clinical dataset, Real Fundus (RF), including 120 low- and high-quality (HQ) image pairs. Then we propose a novel Transformer-based Generative Adversarial Network (RFormer) to restore the real degradation of clinical fundus images. The key component in our network is the Window-based Self-Attention Block (WSAB) which captures non-local self-similarity and long-range dependencies. To produce more visually pleasant results, a Transformer-based discriminator is introduced. Extensive experiments on our clinical benchmark show that the proposed RFormer significantly outperforms the state-of-the-art (SOTA) methods. In addition, experiments of downstream tasks such as vessel segmentation and optic disc/cup detection demonstrate that our proposed RFormer benefits clinical fundus image analysis and applications. The dataset, code, and models are publicly available at https://github.com/dengzhuo-AI/Real-FundusComment: IEEE J-BHI 2022; The First Benchmark and First Transformer-based Method for Real Clinical Fundus Image Restoratio

Gaze Guidance through Peripheral Stimuli

Guiding gaze using near-peripheral vision improves gaze-contingent-display efficiency by reducing display response latency. We propose a new approach for controlling exploration of static displays through near-peripheral stimuli, and report results of an evaluation of its effectiveness. 10 participants viewed full screen displays of 60 blurred pictures (Gaussian filtering). As soon as a fixation (first strategy) or a gaze sample (second strategy) was detected next to the current stimulus, the area surrounding it was deblurred. An image was totally deblurred when all stimuli had thus attracted the user's gaze. Stimuli are blinking deblurred circles (radius: 1 deg visual angle). They appear in predefined positions on the screen, one at a time. For each picture, successive stimulus positions on the screen reproduce observed gaze patterns. The current stimulus is visible only if the visual angle between its position on the screen and the position of the user's current fixation is superior to 8 deg (to avoid users noticing it) and inferior to 14 deg. (near-periphery upper limit). Eye movements are detected through an ASL-H6 eye-tracker (120 Hz). Stimulus saliency is estimated, for each picture and stimulus, from contrast ratio and sum of squared differences between blurred and deblurred area around the stimulus

Improving Depth Perception in Immersive Media Devices by Addressing Vergence-Accommodation Conflict

: Recently, immersive media devices have seen a boost in popularity. However, many problems still remain. Depth perception is a crucial part of how humans behave and interact with their environment. Convergence and accommodation are two physiological mechanisms that provide important depth cues. However, when humans are immersed in virtual environments, they experience a mismatch between these cues. This mismatch causes users to feel discomfort while also hindering their ability to fully perceive object distances. To address the conflict, we have developed a technique that encompasses inverse blurring into immersive media devices. For the inverse blurring, we utilize the classical Wiener deconvolution approach by proposing a novel technique that is applied without the need for an eye-tracker and implemented in a commercial immersive media device. The technique's ability to compensate for the vergence-accommodation conflict was verified through two user studies aimed at reaching and spatial awareness, respectively. The two studies yielded a statistically significant 36% and 48% error reduction in user performance to estimate distances, respectively. Overall, the work done demonstrates how visual stimuli can be modified to allow users to achieve a more natural perception and interaction with the virtual environment

Detection and evaluation of distorted frames in retinal image data

Diplomová práca sa zaoberá detekciou a hodnotením skreslených snímok v retinálnych obrazových dátach. Teoretická časť obsahuje stručné zhrnutie anatómie oka a metód hodnotenia kvality obrazov všeobecne, ako aj konkrétne hodnotenie retinálnych obrazov. Praktická časť bola vypracovaná v programovacom jazyku Python. Obsahuje predspracovanie dostupných retinálnych obrazov za účelom vytvorenia vhodného datasetu. Ďalej je navrhnutá metóda hodnotenia troch typov šumu v skreslených retinálnych obrazoch, presnejšie pomocou Inception-ResNet-v2 modelu. Táto metóda nebola prijateľná a navrhnutá bola teda iná metóda pozostávajúca z dvoch krokov - klasifikácie typu šumu a následného hodnotenia úrovne daného šumu. Pre klasifikáciu typu šumu bolo využité filtrované Fourierove spektrum a na hodnotenie obrazu boli využité príznaky extrahované pomocou ResNet50, ktoré vstupovali do regresného modelu. Táto metóda bola ďalej rozšírená ešte o krok detekcie zašumených snímok v retinálnych sekvenciách.The master's thesis deals with detection and evaluation of distorted frames in retinal image data. The theoretical part contains brief summary of eye anatomy and methods for image quality assessment generally, and also particularly on retinal images. The practical part is carried out in programming language Python. It contains preprocessing of the available retinal images in order to create an appropriate dataset. Further a method for evaluation of three types of blur in distorted retinal images is proposed, specifically Inception-ResNet-v2 model. This method is not feasible and thus another method consisting of two steps is designed - classification of the type of blur and subsequently evaluation of the particular blur level. Filtered Fourier spectrum is used to classify the type of blur and features extracted by ResNet50 serve as the input for regression model. This method is further extended with initial step of detection of blurred frames in retinal sequences.