Multi-start Method with Prior Learning for Image Registration

We propose an efficient image registration strategy that is based on learned prior distributions of transformation parameters. These priors are used to constrain a finite- time multi-start optimization method. Motivation for this approach comes from the fact that standard affine brain image registration methods, especially those based on gradient descent optimization alone, are affected by the initial search position. While global optimization methods can resolve this problem, they are are often very time consuming. Our goal is to build an explicit prior model of the gap between a typical registration solution and the solution gained by a global optimization method. We use this learned prior model to restrict randomized search in the relevant parameter space surrounding the initial solution. Global optimization in this restricted parameter space provides, in finite time, results that are superior to both gradient descent and the general multi-start strategy. The performance of our method is illustrated on a data set of 67 elderly and neurodegenerative brains. Our novel learning strategy and the associated registration method are shown to outperform other approaches. Theoretical, synthetic and real-world examples illustrate this improvement

Automatic image registration using evolutionary algorithm

Image registration is a process to obtain the spatial transformation of an input image to a reference image by which similarity measure is optimized between the two images. Mutual information is a similarity measure based on information theory used in the process of image registration. Mutual information compares the statistical dependency between images. Registration based on mutual information is robust and could use for a large class of mono modality and multimodality images. In this work we use mutual information as the similarity measure .There is a requirement to finding the global maxima of similarity measure, for this we use two algorithm simple genetic algorithm and share genetic algorithm and compare the result of these algorithm. In these optimization technique require several decision to made during implementation, such as encoding, selection method and evolution operator. In this work we use two selection method roulette-wheel method and tournament selection method. Result indicates that these optimization techniques can be used for efficient image registratio

NePhi\texttt{NePhi}: Neural Deformation Fields for Approximately Diffeomorphic Medical Image Registration

This work proposes $\texttt{NePhi}$, a neural deformation model which results in approximately diffeomorphic transformations. In contrast to the predominant voxel-based approaches, $\texttt{NePhi}$ represents deformations functionally which allows for memory-efficient training and inference. This is of particular importance for large volumetric registrations. Further, while medical image registration approaches representing transformation maps via multi-layer perceptrons have been proposed, $\texttt{NePhi}$ facilitates both pairwise optimization-based registration $\textit{as well as}$ learning-based registration via predicted or optimized global and local latent codes. Lastly, as deformation regularity is a highly desirable property for most medical image registration tasks, $\texttt{NePhi}$ makes use of gradient inverse consistency regularization which empirically results in approximately diffeomorphic transformations. We show the performance of $\texttt{NePhi}$ on two 2D synthetic datasets as well as on real 3D lung registration. Our results show that $\texttt{NePhi}$ can achieve similar accuracies as voxel-based representations in a single-resolution registration setting while using less memory and allowing for faster instance-optimization

A novel automated approach of multi-modality retinal image registration and fusion

Biomedical image registration and fusion are usually scene dependent, and require intensive computational effort. A novel automated approach of feature-based control point detection and area-based registration and fusion of retinal images has been successfully designed and developed. The new algorithm, which is reliable and time-efficient, has an automatic adaptation from frame to frame with few tunable threshold parameters. The reference and the to-be-registered images are from two different modalities, i.e. angiogram grayscale images and fundus color images. The relative study of retinal images enhances the information on the fundus image by superimposing information contained in the angiogram image. Through the thesis research, two new contributions have been made to the biomedical image registration and fusion area. The first contribution is the automatic control point detection at the global direction change pixels using adaptive exploratory algorithm. Shape similarity criteria are employed to match the control points. The second contribution is the heuristic optimization algorithm that maximizes Mutual-Pixel-Count (MPC) objective function. The initially selected control points are adjusted during the optimization at the sub-pixel level. A global maxima equivalent result is achieved by calculating MPC local maxima with an efficient computation cost. The iteration stops either when MPC reaches the maximum value, or when the maximum allowable loop count is reached. To our knowledge, it is the first time that the MPC concept has been introduced into biomedical image fusion area as the measurement criteria for fusion accuracy. The fusion image is generated based on the current control point coordinates when the iteration stops. The comparative study of the presented automatic registration and fusion scheme against Centerline Control Point Detection Algorithm, Genetic Algorithm, RMSE objective function, and other existing data fusion approaches has shown the advantage of the new approach in terms of accuracy, efficiency, and novelty

Forward-Backward Splitting in Deformable Image Registration: A Demons Approach

Efficient non-linear image registration implementations are key for many biomedical imaging applications. By using the classical demons approach, the associated optimization problem is solved by an alternate optimization scheme consisting of a gradient descent step followed by Gaussian smoothing. Despite being simple and powerful, the solution of the underlying relaxed formulation is not guaranteed to minimize the original global energy. Implicitly, however, this second step can be recast as the proximal map of the regularizer. This interpretation introduces a parallel to the more general Forward-Backward Splitting (FBS) scheme consisting of a forward gradient descent and proximal step. By shifting entirely to FBS, we can take advantage of the recent advances in FBS methods and solve the original, non-relaxed deformable registration problem for any type of differentiable similarity measure and convex regularization associated with a tractable proximal operator. Additionally, global convergence to a critical point is guaranteed under weak restrictions. For the first time in the context of image registration, we show that Tikhonov regularization breaks down to the simple use of B-Spline filtering in the proximal step. We demonstrate the versatility of FBS by encoding spatial transformation as displacement fields or free-form B-Spline deformations. We use state-of-the-art FBS solvers and compare their performance against the classical demons, the recently proposed inertial demons and the conjugate gradient optimizer. Numerical experiments performed on both synthetic and clinical data show the advantage of FBS in image registration in terms of both convergence and accuracy

Towards 3D ultrasound image based soft tissue tracking: a transrectal ultrasound prostate image alignment system

The emergence of real-time 3D ultrasound (US) makes it possible to consider image-based tracking of subcutaneous soft tissue targets for computer guided diagnosis and therapy. We propose a 3D transrectal US based tracking system for precise prostate biopsy sample localisation. The aim is to improve sample distribution, to enable targeting of unsampled regions for repeated biopsies, and to make post-interventional quality controls possible. Since the patient is not immobilized, since the prostate is mobile and due to the fact that probe movements are only constrained by the rectum during biopsy acquisition, the tracking system must be able to estimate rigid transformations that are beyond the capture range of common image similarity measures. We propose a fast and robust multi-resolution attribute-vector registration approach that combines global and local optimization methods to solve this problem. Global optimization is performed on a probe movement model that reduces the dimensionality of the search space and thus renders optimization efficient. The method was tested on 237 prostate volumes acquired from 14 different patients for 3D to 3D and 3D to orthogonal 2D slices registration. The 3D-3D version of the algorithm converged correctly in 96.7% of all cases in 6.5s with an accuracy of 1.41mm (r.m.s.) and 3.84mm (max). The 3D to slices method yielded a success rate of 88.9% in 2.3s with an accuracy of 1.37mm (r.m.s.) and 4.3mm (max)

Progressive Refinement Imaging

This paper presents a novel technique for progressive online integration of uncalibrated image sequences with substantial geometric and/or photometric discrepancies into a single, geometrically and photometrically consistent image. Our approach can handle large sets of images, acquired from a nearly planar or infinitely distant scene at different resolutions in object domain and under variable local or global illumination conditions. It allows for efficient user guidance as its progressive nature provides a valid and consistent reconstruction at any moment during the online refinement process. // Our approach avoids global optimization techniques, as commonly used in the field of image refinement, and progressively incorporates new imagery into a dynamically extendable and memory‐efficient Laplacian pyramid. Our image registration process includes a coarse homography and a local refinement stage using optical flow. Photometric consistency is achieved by retaining the photometric intensities given in a reference image, while it is being refined. Globally blurred imagery and local geometric inconsistencies due to, e.g. motion are detected and removed prior to image fusion. // We demonstrate the quality and robustness of our approach using several image and video sequences, including handheld acquisition with mobile phones and zooming sequences with consumer cameras

An Orthogonal Learning Differential Evolution Algorithm for Remote Sensing Image Registration

We introduce an area-based method for remote sensing image registration. We use orthogonal learning differential evolution algorithm to optimize the similarity metric between the reference image and the target image. Many local and global methods have been used to achieve the optimal similarity metric in the last few years. Because remote sensing images are usually influenced by large distortions and high noise, local methods will fail in some cases. For this reason, global methods are often required. The orthogonal learning (OL) strategy is efficient when searching in complex problem spaces. In addition, it can discover more useful information via orthogonal experimental design (OED). Differential evolution (DE) is a heuristic algorithm. It has shown to be efficient in solving the remote sensing image registration problem. So orthogonal learning differential evolution algorithm (OLDE) is efficient for many optimization problems. The OLDE method uses the OL strategy to guide the DE algorithm to discover more useful information. Experiments show that the OLDE method is more robust and efficient for registering remote sensing images

LATITUDE: Robotic Global Localization with Truncated Dynamic Low-pass Filter in City-scale NeRF

Neural Radiance Fields (NeRFs) have made great success in representing complex 3D scenes with high-resolution details and efficient memory. Nevertheless, current NeRF-based pose estimators have no initial pose prediction and are prone to local optima during optimization. In this paper, we present LATITUDE: Global Localization with Truncated Dynamic Low-pass Filter, which introduces a two-stage localization mechanism in city-scale NeRF. In place recognition stage, we train a regressor through images generated from trained NeRFs, which provides an initial value for global localization. In pose optimization stage, we minimize the residual between the observed image and rendered image by directly optimizing the pose on tangent plane. To avoid convergence to local optimum, we introduce a Truncated Dynamic Low-pass Filter (TDLF) for coarse-to-fine pose registration. We evaluate our method on both synthetic and real-world data and show its potential applications for high-precision navigation in large-scale city scenes. Codes and data will be publicly available at https://github.com/jike5/LATITUDE.Comment: 7 pages, 6 figures, submitted to ICRA 202