Approximate Nearest Neighbor Fields in Video

We introduce RIANN (Ring Intersection Approximate Nearest Neighbor search), an algorithm for matching patches of a video to a set of reference patches in real-time. For each query, RIANN finds potential matches by intersecting rings around key points in appearance space. Its search complexity is reversely correlated to the amount of temporal change, making it a good fit for videos, where typically most patches change slowly with time. Experiments show that RIANN is up to two orders of magnitude faster than previous ANN methods, and is the only solution that operates in real-time. We further demonstrate how RIANN can be used for real-time video processing and provide examples for a range of real-time video applications, including colorization, denoising, and several artistic effects.Comment: A CVPR 2015 oral pape

A novel image inpainting framework based on multilevel image pyramids

Image inpainting is the art of manipulating an image so that it is visually unrecognizable way. A considerable amount of research has been done in this area over the last few years. However, the state of art techniques does suffer from computational complexities and plausible results. This paper proposes a multi-level image pyramid-based image inpainting algorithm. The image inpainting algorithm starts with the coarsest level of the image pyramid and overpainting information is transferred to the subsequent levels until the bottom level gets inpainted. The search strategy used in the algorithm is based on hashing the coherent information in an image which makes the search fast and accurate. Also, the search space is constrained based on the propagated information thereby reducing the complexity of the algorithm. Compared to other inpainting methods; the proposed algorithm inpaints the target region with better plausibility and human vision conformation. Experimental results show that the proposed algorithm achieves better results as compared to other inpainting techniques

SubPatch: Random kd-tree on a sub-sampled patch set for nearest neighbor field estimation

We propose a new method to compute the approximate nearest-neighbors field (ANNF) between image pairs using random kd-tree and patch set sub-sampling. By exploiting image coherence we demonstrate that it is possible to reduce the number of patches on which we compute the ANNF, while maintaining high overall accuracy on the final result. Information on missing patches is then recovered by interpolation and propagation of good matches. The introduction of the sub-sampling factor on patch sets also allows for setting the desired trade off between accuracy and speed, providing a flexibility that lacks in state-of-the-art methods. Tests conducted on a public database prove that our algorithm achieves superior performance with respect to PatchMatch (PM) and Coherence Sensitivity Hashing (CSH) algorithms in a comparable computational time

Image Inpainting by Hyperbolic Selection of Pixels for Two Dimensional Bicubic Interpolations

Image inpainting is a restoration process which has numerous applications. Restoring of scanned old images with scratches, or removing objects in images are some of inpainting applications. Different approaches have been used for implementation of inpainting algorithms. Interpolation approaches only consider one direction for this purpose. In this paper we present a new perspective to image inpainting. We consider multiple directions and apply both one-dimensional and two-dimensional bicubic interpolations. Neighboring pixels are selected in a hyperbolic formation to better preserve corner pixels. We compare our work with recent inpainting approaches to show our superior results

Efficient Large-scale Approximate Nearest Neighbor Search on the GPU

We present a new approach for efficient approximate nearest neighbor (ANN) search in high dimensional spaces, extending the idea of Product Quantization. We propose a two-level product and vector quantization tree that reduces the number of vector comparisons required during tree traversal. Our approach also includes a novel highly parallelizable re-ranking method for candidate vectors by efficiently reusing already computed intermediate values. Due to its small memory footprint during traversal, the method lends itself to an efficient, parallel GPU implementation. This Product Quantization Tree (PQT) approach significantly outperforms recent state of the art methods for high dimensional nearest neighbor queries on standard reference datasets. Ours is the first work that demonstrates GPU performance superior to CPU performance on high dimensional, large scale ANN problems in time-critical real-world applications, like loop-closing in videos

Feature Match for Medical Images

This paper represents an algorithm for Feature Match, a summed up estimated approximate nearest neighbor field (ANNF) calculation system, between a source and target image. The proposed calculation can estimate ANNF maps between any image sets, not as a matter of course related. This generalization is accomplished through proper spatial-range changes. To register ANNF maps, worldwide shading adjustment is connected as a reach change on the source picture. Image patches from the pair of pictures are approximated utilizing low-dimensional elements, which are utilized alongside KD-tree to appraise the ANNF map. This ANNF guide is further enhanced in view of picture coherency and spatial changes. The proposed generalization, empowers to handle a more extensive scope of vision applications, which have not been handled utilizing the ANNF structure. Here one such application is outlined namely: optic plate discovery .This application manages restorative imaging, where optic circles are found in retinal pictures utilizing a sound optic circle picture as regular target picture. ANNF mappings is used in this application and is shown experimentally that the proposed approaches are faster and accurate, compared with the state-of the-art techniques