Deep learning network to correct axial and coronal eye motion in 3D OCT retinal imaging

Optical Coherence Tomography (OCT) is one of the most important retinal imaging technique. However, involuntary motion artifacts still pose a major challenge in OCT imaging that compromises the quality of downstream analysis, such as retinal layer segmentation and OCT Angiography. We propose deep learning based neural networks to correct axial and coronal motion artifacts in OCT based on a single volumetric scan. The proposed method consists of two fully-convolutional neural networks that predict Z and X dimensional displacement maps sequentially in two stages. The experimental result shows that the proposed method can effectively correct motion artifacts and achieve smaller error than other methods. Specifically, the method can recover the overall curvature of the retina, and can be generalized well to various diseases and resolutions

Optimization and learning based video coding

The complexity of video coding standards has increased significantly from H.262/MPEG-2 to H.264/AVC in order to increase coding efficiency. Complexity mainly was increased more by architecture than by algorithms: One 16x16 MB type in MPEG-2 was partitioned into various MB types such as 16x16, 8x16, 8x8, 4x4. Half pixel accuracy motion estimation was extended to support quarter pixel accuracy, and various simple directional filters were applied for intra prediction. In this dissertation, we consider optimization and learning methods to solve video coding problems. In our approaches, complexity is mainly increased by algorithms to improve coding efficiency. Especially, we apply these methods for the Rate-Distortion (RD) optimization problem in H.264 and intra prediction as a new video coding scheme because they are highly related with numerical optimization and regression theories. For the RD optimization problem, we propose a general framework with consideration of temporal prediction dependency using the primal-dual decomposition and subgradient projection methods. As a result, optimality conditions among the Lagrange multipliers $\lambda$ are derived for the optimal bit allocation. The proposed method is compared with the Rate Control (RC) algorithm in the reference software model (JM model) of H.264. In order to reduce the complexity of the proposed method, an adaptive Lagrange multiplier selection method is proposed in the RC algorithm using the Classification-Maximization (CM) algorithm. In addition, two variations of the CM algorithm, that is, Relaxed CM (RCM) and Incremental CM (ICM) are proposed to improve the performance and avoid iterations. We compare [lambda] of the proposed adaptive Lagrange multiplier selection methods with ones of the JM model and the greedy search. Finally, we propose a new video coding scheme using learning methods. In particular, learning methods such as support vector regression and locally weighted learning are applied for intra prediction by means of batch and online learning. We present that online learning based intra prediction is better for video coding because of limited training time and nonstationary video sequences even though batch learning based intra prediction can achieve significant improvement in low- motion sequences. Experimental results show that online learning based video coding is promising for future video codin

Optimization and learning based video coding

The complexity of video coding standards has increased significantly from H.262/MPEG-2 to H.264/AVC in order to increase coding efficiency. Complexity mainly was increased more by architecture than by algorithms: One 16x16 MB type in MPEG-2 was partitioned into various MB types such as 16x16, 8x16, 8x8, 4x4. Half pixel accuracy motion estimation was extended to support quarter pixel accuracy, and various simple directional filters were applied for intra prediction. In this dissertation, we consider optimization and learning methods to solve video coding problems. In our approaches, complexity is mainly increased by algorithms to improve coding efficiency. Especially, we apply these methods for the Rate-Distortion (RD) optimization problem in H.264 and intra prediction as a new video coding scheme because they are highly related with numerical optimization and regression theories. For the RD optimization problem, we propose a general framework with consideration of temporal prediction dependency using the primal-dual decomposition and subgradient projection methods. As a result, optimality conditions among the Lagrange multipliers $\lambda$ are derived for the optimal bit allocation. The proposed method is compared with the Rate Control (RC) algorithm in the reference software model (JM model) of H.264. In order to reduce the complexity of the proposed method, an adaptive Lagrange multiplier selection method is proposed in the RC algorithm using the Classification-Maximization (CM) algorithm. In addition, two variations of the CM algorithm, that is, Relaxed CM (RCM) and Incremental CM (ICM) are proposed to improve the performance and avoid iterations. We compare [lambda] of the proposed adaptive Lagrange multiplier selection methods with ones of the JM model and the greedy search. Finally, we propose a new video coding scheme using learning methods. In particular, learning methods such as support vector regression and locally weighted learning are applied for intra prediction by means of batch and online learning. We present that online learning based intra prediction is better for video coding because of limited training time and nonstationary video sequences even though batch learning based intra prediction can achieve significant improvement in low- motion sequences. Experimental results show that online learning based video coding is promising for future video codin

JOINT MOTION CORRECTION AND 3D SEGMENTATION WITH GRAPH-ASSISTED NEURAL NETWORKS FOR RETINAL OCT.

Optical Coherence Tomography (OCT) is a widely used non-invasive high resolution 3D imaging technique for biological tissues and plays an important role in ophthalmology. OCT retinal layer segmentation is a fundamental image processing step for OCT-Angiography projection, and disease analysis. A major problem in retinal imaging is the motion artifacts introduced by involuntary eye movements. In this paper, we propose neural networks that jointly correct eye motion and retinal layer segmentation utilizing 3D OCT information, so that the segmentation among neighboring B-scans would be consistent. The experimental results show both visual and quantitative improvements by combining motion correction and 3D OCT layer segmentation comparing to conventional and deep-learning based 2D OCT layer segmentation

A Convolutional Neural Network Pipeline For Multi-Temporal Retinal Image Registration

A sequence of images is usually captured to observe the change of health status in medical diagnosis. However, an image sequence taken over year usually suffers from severe deformation, making it time-consuming for physicians to match corresponding patterns. In this paper, we propose a coarse-to-fine pipeline for retinal image registration based on convolutional neural network. By leveraging the three components of the pipeline: feature matching, outlier rejection, and local registration, we recover the deformation and accurately align multi-temporal image sequences. Experimental results show that the proposed network is robust to severe deformation as well as illumination and contrast variations. With the proposed registration pipeline, the change of image patterns over time can be identified through visual analysis

Learning to Correct Axial Motion in Oct for 3D Retinal Imaging

Optical Coherence Tomography (OCT) is a powerful technique for non-invasive 3D imaging of biological tissues at high resolution that has revolutionized retinal imaging. A major challenge in OCT imaging is the motion artifacts introduced by involuntary eye movements. In this paper, we propose a convolutional neural network that learns to correct axial motion in OCT based on a single volumetric scan. The proposed method is able to correct large motion, while preserving the overall curvature of the retina. The experimental results show significant improvements in visual quality as well as overall error compared to the conventional methods in both normal and disease cases