STEFANN: Scene Text Editor using Font Adaptive Neural Network

Textual information in a captured scene plays an important role in scene interpretation and decision making. Though there exist methods that can successfully detect and interpret complex text regions present in a scene, to the best of our knowledge, there is no significant prior work that aims to modify the textual information in an image. The ability to edit text directly on images has several advantages including error correction, text restoration and image reusability. In this paper, we propose a method to modify text in an image at character-level. We approach the problem in two stages. At first, the unobserved character (target) is generated from an observed character (source) being modified. We propose two different neural network architectures - (a) FANnet to achieve structural consistency with source font and (b) Colornet to preserve source color. Next, we replace the source character with the generated character maintaining both geometric and visual consistency with neighboring characters. Our method works as a unified platform for modifying text in images. We present the effectiveness of our method on COCO-Text and ICDAR datasets both qualitatively and quantitatively.Comment: Accepted in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 202

Automatic image segmentation by dynamic region growth and multiresolution merging

Image segmentation is a fundamental task in many computer vision applications. We present a novel unsupervised color image segmentation algorithm named GSEG, which exploits the information obtained from detecting edges in color images. By using a color gradient detection technique, pixels without edges are clustered and labeled individually to identify the image content. Elements that contain higher gradient density are included by a dynamic generation of clusters as the segmentation progresses. By quantizing the colors in the image and extracting texture information from the neighborhood entropy of each pixel, the proposed method obtains accurate models of texture that are highly effective to merge regions that belong to the same object. Experimental results for various image scenarios in comparison with state-of-the-art segmentation techniques demonstrate the performance advantages of the proposed method

On Adaptive Image Segmentation of Remotely Sensed Imagery

A critical step in object-oriented geospatial analysis (OBIA) is image segmentation. A single set of parameters is often not effective segmenting an image. To solve this problem, an adaptive approach to image segmentation has been proposed, which utilizes segments determined from a lower-spatial resolution image as the context to analyse a corresponding image at a higher-spatial resolution to create multiple sets of segmentation parameters to address the needs of different parts of the image. However, due to inherent differences in perceptions of a scene at different spatial resolutions and co-registration, segment boundaries from the low spatial resolution image need to be adjusted before they are applied to the high-spatial resolution image. This is a non-trivial task due to considerations such as noise, image complexity, and determining appropriate boundaries. Accordingly, an innovative method was developed. Adjustments were executed for each boundary pixel based on the minimization of an energy function characterizing local homogeneity. Adjustments are based on a structure which rewarded movement towards edges, and superior changes towards homogeneity. The adjusted segments act as the basis for the determination of segmentation parameters through a variogram based method. The developed method was tested on a set of Quickbird, and ASTER images, from a study area in Ontario, Canada. Results showed that the adjusted segmentation boundaries obtained from the lower resolution imagery were aligned well with the features in the Quickbird imagery, and segmentation maps determined using the adaptive segmentation method were superior to those created by a non-adaptive approach. This work will allow users to more easily and quickly segment large high resolution images

Color Separation for Image Segmentation

Image segmentation is a fundamental problem in computer vision that has drawn intensive research attention during the past few decades, resulting in a variety of segmentation algorithms. Segmentation is often formulated as a Markov random field (MRF) and the solution corresponding to the maximum a posteriori probability (MAP) is found using energy minimiza- tion framework. Many standard segmentation techniques rely on foreground and background appearance models given a priori. In this case the corresponding energy can be efficiently op- timized globally. If the appearance models are not known, the energy becomes NP-hard, and many methods resort to iterative schemes that jointly optimize appearance and segmentation. Such algorithms can only guarantee local minimum. Here we propose a new energy term explicitly measuring L1 distance between the object and background appearance models that can be globally maximized in one graph cut. Our method directly tries to minimize the appearance overlap between the segments. We show that in many applications including interactive segmentation, shape matching, segmentation from stereo pairs and saliency segmentation our simple term makes NP-hard segmentation functionals unnecessary and renders good segmentation performance both qualitatively and quantitatively

Segmentation of Melanoma Skin Lesion Using Perceptual Color Difference Saliency with Morphological Analysis

The prevalence of melanoma skin cancer disease is rapidly increasing as recorded death cases of its patients continue to annually escalate. Reliable segmentation of skin lesion is one essential requirement of an efficient noninvasive computer aided diagnosis tool for accelerating the identification process of melanoma. This paper presents a new algorithm based on perceptual color difference saliency along with binary morphological analysis for segmentation of melanoma skin lesion in dermoscopic images. The new algorithm is compared with existing image segmentation algorithms on benchmark dermoscopic images acquired from public corpora. Results of both qualitative and quantitative evaluations of the new algorithm are encouraging as the algorithm performs excellently in comparison with the existing image segmentation algorithms