Segmentation by transduction

International audienceThis paper addresses the problem of segmenting an image into regions consistent with user-supplied seeds (e.g., a sparse set of broad brush strokes). We view this task as a statistical transductive inference, in which some pixels are already associated with given zones and the remaining ones need to be classified. Our method relies on the Laplacian graph regularizer, a powerful manifold learning tool that is based on the estimation of variants of the Laplace-Beltrami operator and is tightly related to diffusion processes. Segmentation is modeled as the task of finding matting coefficients for unclassified pixels given known matting coefficients for seed pixels. The proposed algorithm essentially relies on a high margin assumption in the space of pixel characteristics. It is simple, fast, and accurate, as demonstrated by qualitative results on natural images and a quantitative comparison with state-of-the-art methods on the Microsoft GrabCut segmentation database

Single image defocus estimation by modified gaussian function

© 2019 John Wiley & Sons, Ltd. This article presents an algorithm to estimate the defocus blur from a single image. Most of the existing methods estimate the defocus blur at edge locations, which further involves the reblurring process. For this purpose, existing methods use the traditional Gaussian function in the phase of reblurring but it is found that the traditional Gaussian kernel is sensitive to the edges and can cause loss of edges information. Hence, there are more chances of missing spatially varying blur at edge locations. We offer the repeated averaging filters as an alternative to the traditional Gaussian function, which is more effective, and estimate the spatially varying defocus blur at edge locations. By using repeated averaging filters, a blur sparse map is computed. The obtained sparse map is propagated by integration of superpixels segmentation and transductive inference to estimate full defocus blur map. Our adopted method of repeated averaging filters has less computational time of defocus blur map estimation and has better visual estimates of the final defocus recovered map. Moreover, it has surpassed many previous state-of-the-art proposed systems in terms of quantative analysis

Fast Preprocessing for Robust Face Sketch Synthesis

Exemplar-based face sketch synthesis methods usually meet the challenging problem that input photos are captured in different lighting conditions from training photos. The critical step causing the failure is the search of similar patch candidates for an input photo patch. Conventional illumination invariant patch distances are adopted rather than directly relying on pixel intensity difference, but they will fail when local contrast within a patch changes. In this paper, we propose a fast preprocessing method named Bidirectional Luminance Remapping (BLR), which interactively adjust the lighting of training and input photos. Our method can be directly integrated into state-of-the-art exemplar-based methods to improve their robustness with ignorable computational cost.Comment: IJCAI 2017. Project page: http://www.cs.cityu.edu.hk/~yibisong/ijcai17_sketch/index.htm

Transductive Segmentation of Textured Meshes

International audienceThis paper addresses the problem of segmenting a textured mesh into objects or object classes, consistently with user-supplied seeds. We view this task as transductive learning and use the flexibility of kernel-based weights to incorporate a various number of diverse features. Our method combines a Laplacian graph regularizer that enforces spatial coherence in label propagation and an SVM classifier that ensures dissemination of the seeds characteristics. Our interactive framework allows to easily specify classes seeds with sketches drawn on the mesh and potentially refine the segmentation. We obtain qualitatively good segmentations on several architectural scenes and show the applicability of our method to outliers removing

Context-Aware Image Matting for Simultaneous Foreground and Alpha Estimation

Natural image matting is an important problem in computer vision and graphics. It is an ill-posed problem when only an input image is available without any external information. While the recent deep learning approaches have shown promising results, they only estimate the alpha matte. This paper presents a context-aware natural image matting method for simultaneous foreground and alpha matte estimation. Our method employs two encoder networks to extract essential information for matting. Particularly, we use a matting encoder to learn local features and a context encoder to obtain more global context information. We concatenate the outputs from these two encoders and feed them into decoder networks to simultaneously estimate the foreground and alpha matte. To train this whole deep neural network, we employ both the standard Laplacian loss and the feature loss: the former helps to achieve high numerical performance while the latter leads to more perceptually plausible results. We also report several data augmentation strategies that greatly improve the network's generalization performance. Our qualitative and quantitative experiments show that our method enables high-quality matting for a single natural image. Our inference codes and models have been made publicly available at https://github.com/hqqxyy/Context-Aware-Matting.Comment: This is the camera ready version of ICCV2019 pape

A discrete graph Laplacian for signal processing

In this thesis we exploit diffusion processes on graphs to effect two fundamental problems of image processing: denoising and segmentation. We treat these two low-level vision problems on the pixel-wise level under a unified framework: a graph embedding. Using this framework opens us up to the possibilities of exploiting recently introduced algorithms from the semi-supervised machine learning literature. We contribute two novel edge-preserving smoothing algorithms to the literature. Furthermore we apply these edge-preserving smoothing algorithms to some computational photography tasks. Many recent computational photography tasks require the decomposition of an image into a smooth base layer containing large scale intensity variations and a residual layer capturing fine details. Edge-preserving smoothing is the main computational mechanism in producing these multi-scale image representations. We, in effect, introduce a new approach to edge-preserving multi-scale image decompositions. Where as prior approaches such as the Bilateral filter and weighted-least squares methods require multiple parameters to tune the response of the filters our method only requires one. This parameter can be interpreted as a scale parameter. We demonstrate the utility of our approach by applying the method to computational photography tasks that utilise multi-scale image decompositions. With minimal modification to these edge-preserving smoothing algorithms we show that we can extend them to produce interactive image segmentation. As a result the operations of segmentation and denoising are conducted under a unified framework. Moreover we discuss how our method is related to region based active contours. We benchmark our proposed interactive segmentation algorithms against those based upon energy-minimisation, specifically graph-cut methods. We demonstrate that we achieve competitive performance

Efficient Representation Learning With Graph Neural Networks

Graph neural networks (GNNs) have emerged as the dominant paradigm for graph representation learning, igniting widespread interest in utilizing sophisticated GNNs for diverse computer vision tasks in various domains, including visual SLAM, 3D object recognition and segmentation, as well as visual perception with event cameras. However, the applications of these GNNs often rely on cumbersome GNN architectures for favorable performance, posing challenges for real-time interaction, particularly in edge computing scenarios. This is particularly relevant in cases such as autonomous driving, where timely responses are crucial for handling complex traffic conditions. The objective of this thesis is to contribute to the advancement of learning efficient representations using lightweight GNNs, enabling their effective deployment in resource-constrained environments. To achieve this goal, the thesis explores various efficient learning schemes, focusing on four key aspects: the data side, the model side, the data-model side, and the application side. In terms of data-driven efficient learning, the thesis proposes an adaptive data modification scheme that allows a pre-trained model to be repurposed for multiple designated downstream tasks in a resource-efficient manner, without the need for re-training or fine-tuning. For model-centric efficiency, the thesis introduces a multi-talented and lightweight architecture, without accessing human annotations, that can integrate the expertise of the pre-trained complex GNNs specializing in different tasks. Furthermore, the thesis explores a dedicated binarization scheme on the data-model side that converts both input data and model parameters into 1-bit representations, resulting in lightweight 1-bit architectures. Finally, the thesis investigates an application-specific efficient learning scheme that models the style transfer process as message passing in GNNs, enabling efficient semi-parametric stylization