Robust Non-Rigid Registration with Reweighted Position and Transformation Sparsity

Non-rigid registration is challenging because it is ill-posed with high degrees of freedom and is thus sensitive to noise and outliers. We propose a robust non-rigid registration method using reweighted sparsities on position and transformation to estimate the deformations between 3-D shapes. We formulate the energy function with position and transformation sparsity on both the data term and the smoothness term, and define the smoothness constraint using local rigidity. The double sparsity based non-rigid registration model is enhanced with a reweighting scheme, and solved by transferring the model into four alternately-optimized subproblems which have exact solutions and guaranteed convergence. Experimental results on both public datasets and real scanned datasets show that our method outperforms the state-of-the-art methods and is more robust to noise and outliers than conventional non-rigid registration methods.Comment: IEEE Transactions on Visualization and Computer Graphic

해부학적 유도 PET 재구성: 매끄럽지 않은 사전 함수부터 딥러닝 접근까지

학위논문 (박사) -- 서울대학교 대학원 : 의과대학 의과학과, 2021. 2. 이재성.Advances in simultaneous positron emission tomography/magnetic resonance imaging (PET/MRI) technology have led to an active investigation of the anatomy-guided regularized PET image reconstruction algorithm based on MR images. Among the various priors proposed for anatomy-guided regularized PET image reconstruction, Bowsher’s method based on second-order smoothing priors sometimes suffers from over-smoothing of detailed structures. Therefore, in this study, we propose a Bowsher prior based on the l1 norm and an iteratively reweighting scheme to overcome the limitation of the original Bowsher method. In addition, we have derived a closed solution for iterative image reconstruction based on this non-smooth prior. A comparison study between the original l2 and proposed l1 Bowsher priors were conducted using computer simulation and real human data. In the simulation and real data application, small lesions with abnormal PET uptake were better detected by the proposed l1 Bowsher prior methods than the original Bowsher prior. The original l2 Bowsher leads to a decreased PET intensity in small lesions when there is no clear separation between the lesions and surrounding tissue in the anatomical prior. However, the proposed l1 Bowsher prior methods showed better contrast between the tumors and surrounding tissues owing to the intrinsic edge-preserving property of the prior which is attributed to the sparseness induced by l1 norm, especially in the iterative reweighting scheme. Besides, the proposed methods demonstrated lower bias and less hyper-parameter dependency on PET intensity estimation in the regions with matched anatomical boundaries in PET and MRI. Moreover, based on the formulation of l1 Bowsher prior, the unrolled network containing the conventional maximum-likelihood expectation-maximization (ML-EM) module was also proposed. The convolutional layers successfully learned the distribution of anatomically-guided PET images and the EM module corrected the intermediate outputs by comparing them with sinograms. The proposed unrolled network showed better performance than ordinary U-Net, where the regional uptake is less biased and deviated. Therefore, these methods will help improve the PET image quality based on the anatomical side information.양전자방출단층촬영 / 자기공명영상 (PET/MRI) 동시 획득 기술의 발전으로 MR 영상을 기반으로 한 해부학적 사전 함수로 정규화 된 PET 영상 재구성 알고리즘에 대한 심도있는 평가가 이루어졌다. 해부학 기반으로 정규화 된 PET 이미지 재구성을 위해 제안 된 다양한 사전 중 2차 평활화 사전함수에 기반한 Bowsher의 방법은 때때로 세부 구조의 과도한 평활화로 어려움을 겪는다. 따라서 본 연구에서는 원래 Bowsher 방법의 한계를 극복하기 위해 l1 norm에 기반한 Bowsher 사전 함수와 반복적인 재가중치 기법을 제안한다. 또한, 우리는 이 매끄럽지 않은 사전 함수를 이용한 반복적 이미지 재구성에 대해 닫힌 해를 도출했다. 원래 l2와 제안 된 l1 Bowsher 사전 함수 간의 비교 연구는 컴퓨터 시뮬레이션과 실제 데이터를 사용하여 수행되었다. 시뮬레이션 및 실제 데이터에서 비정상적인 PET 흡수를 가진 작은 병변은 원래 Bowsher 이전보다 제안 된 l1 Bowsher 사전 방법으로 더 잘 감지되었다. 원래의 l2 Bowsher는 해부학적 영상에서 병변과 주변 조직 사이에 명확한 분리가 없을 때 작은 병변에서의 PET 강도를 감소시킨다. 그러나 제안 된 l1 Bowsher 사전 방법은 특히 반복적 재가중치 기법에서 l1 노름에 의해 유도된 희소성에 기인한 특성으로 인해 종양과 주변 조직 사이에 더 나은 대비를 보여주었다. 또한 제안된 방법은 PET과 MRI의 해부학적 경계가 일치하는 영역에서 PET 강도 추정에 대한 편향이 더 낮고 하이퍼 파라미터 종속성이 적음을 보여주었다. 또한, l1Bowsher 사전 함수의 닫힌 해를 기반으로 기존의 ML-EM (maximum-likelihood expectation-maximization) 모듈을 포함하는 펼쳐진 네트워크도 제안되었다. 컨볼루션 레이어는 해부학적으로 유도 재구성된 PET 이미지의 분포를 성공적으로 학습했으며, EM 모듈은 중간 출력들을 사이노그램과 비교하여 결과 이미지가 잘 들어맞게 수정했다. 제안된 펼쳐진 네트워크는 지역의 흡수선량이 덜 편향되고 편차가 적어, 일반 U-Net보다 더 나은 성능을 보여주었다. 따라서 이러한 방법들은 해부학적 정보를 기반으로 PET 이미지 품질을 향상시키는 데 유용할 것이다.Chapter 1. Introduction 1 1.1. Backgrounds 1 1.1.1. Positron Emission Tomography 1 1.1.2. Maximum a Posterior Reconstruction 1 1.1.3. Anatomical Prior 2 1.1.4. Proposed l_1 Bowsher Prior 3 1.1.5. Deep Learning for MR-less Application 4 1.2. Purpose of the Research 4 Chapter 2. Anatomically-guided PET Reconstruction Using Bowsher Prior 6 2.1. Backgrounds 6 2.1.1. PET Data Model 6 2.1.2. Original Bowsher Prior 7 2.2. Methods and Materials 8 2.2.1. Proposed l_1 Bowsher Prior 8 2.2.2. Iterative Reweighting 13 2.2.3. Computer Simulations 15 2.2.4. Human Data 16 2.2.5. Image Analysis 17 2.3. Results 19 2.3.1. Simulation with Brain Phantom 19 2.3.2.Human Data 20 2.4. Discussions 25 Chapter 3. Deep Learning Approach for Anatomically-guided PET Reconstruction 31 3.1. Backgrounds 31 3.2. Methods and Materials 33 3.2.1. Douglas-Rachford Splitting 33 3.2.2. Network Architecture 34 3.2.3. Dataset and Training Details 35 3.2.4. Image Analysis 36 3.3. Results 37 3.4. Discussions 38 Chapter 4. Conclusions 40 Bibliography 41 Abstract in Korean (국문 초록) 52Docto

Image Deblurring and Super-resolution by Adaptive Sparse Domain Selection and Adaptive Regularization

As a powerful statistical image modeling technique, sparse representation has been successfully used in various image restoration applications. The success of sparse representation owes to the development of l1-norm optimization techniques, and the fact that natural images are intrinsically sparse in some domain. The image restoration quality largely depends on whether the employed sparse domain can represent well the underlying image. Considering that the contents can vary significantly across different images or different patches in a single image, we propose to learn various sets of bases from a pre-collected dataset of example image patches, and then for a given patch to be processed, one set of bases are adaptively selected to characterize the local sparse domain. We further introduce two adaptive regularization terms into the sparse representation framework. First, a set of autoregressive (AR) models are learned from the dataset of example image patches. The best fitted AR models to a given patch are adaptively selected to regularize the image local structures. Second, the image non-local self-similarity is introduced as another regularization term. In addition, the sparsity regularization parameter is adaptively estimated for better image restoration performance. Extensive experiments on image deblurring and super-resolution validate that by using adaptive sparse domain selection and adaptive regularization, the proposed method achieves much better results than many state-of-the-art algorithms in terms of both PSNR and visual perception.Comment: 35 pages. This paper is under review in IEEE TI

Remove Cosine Window from Correlation Filter-based Visual Trackers: When and How

Correlation filters (CFs) have been continuously advancing the state-of-the-art tracking performance and have been extensively studied in the recent few years. Most of the existing CF trackers adopt a cosine window to spatially reweight base image to alleviate boundary discontinuity. However, cosine window emphasizes more on the central region of base image and has the risk of contaminating negative training samples during model learning. On the other hand, spatial regularization deployed in many recent CF trackers plays a similar role as cosine window by enforcing spatial penalty on CF coefficients. Therefore, we in this paper investigate the feasibility to remove cosine window from CF trackers with spatial regularization. When simply removing cosine window, CF with spatial regularization still suffers from small degree of boundary discontinuity. To tackle this issue, binary and Gaussian shaped mask functions are further introduced for eliminating boundary discontinuity while reweighting the estimation error of each training sample, and can be incorporated with multiple CF trackers with spatial regularization. In comparison to the counterparts with cosine window, our methods are effective in handling boundary discontinuity and sample contamination, thereby benefiting tracking performance. Extensive experiments on three benchmarks show that our methods perform favorably against the state-of-the-art trackers using either handcrafted or deep CNN features. The code is publicly available at https://github.com/lifeng9472/Removing_cosine_window_from_CF_trackers.Comment: 13 pages, 7 figures, submitted to IEEE Transactions on Image Processin

Fast Fiber Orientation Estimation in Diffusion MRI from kq-Space Sampling and Anatomical Priors

High spatio-angular resolution diffusion MRI (dMRI) has been shown to provide accurate identification of complex fiber configurations, albeit at the cost of long acquisition times. We propose a method to recover intra-voxel fiber configurations at high spatio-angular resolution relying on a kq-space under-sampling scheme to enable accelerated acquisitions. The inverse problem for reconstruction of the fiber orientation distribution (FOD) is regularized by a structured sparsity prior promoting simultaneously voxelwise sparsity and spatial smoothness of fiber orientation. Prior knowledge of the spatial distribution of white matter, gray matter and cerebrospinal fluid is also assumed. A minimization problem is formulated and solved via a forward-backward convex optimization algorithmic structure. Simulations and real data analysis suggest that accurate FOD mapping can be achieved from severe kq-space under-sampling regimes, potentially enabling high spatio-angular dMRI in the clinical setting.Comment: 10 pages, 5 figures, Supplementary Material