STEFANN: Scene Text Editor using Font Adaptive Neural Network

Textual information in a captured scene plays an important role in scene interpretation and decision making. Though there exist methods that can successfully detect and interpret complex text regions present in a scene, to the best of our knowledge, there is no significant prior work that aims to modify the textual information in an image. The ability to edit text directly on images has several advantages including error correction, text restoration and image reusability. In this paper, we propose a method to modify text in an image at character-level. We approach the problem in two stages. At first, the unobserved character (target) is generated from an observed character (source) being modified. We propose two different neural network architectures - (a) FANnet to achieve structural consistency with source font and (b) Colornet to preserve source color. Next, we replace the source character with the generated character maintaining both geometric and visual consistency with neighboring characters. Our method works as a unified platform for modifying text in images. We present the effectiveness of our method on COCO-Text and ICDAR datasets both qualitatively and quantitatively.Comment: Accepted in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 202

Forensic research on detecting seam carving in digital images

Digital images have been playing an important role in our daily life for the last several decades. Naturally, image editing technologies have been tremendously developed due to the increasing demands. As a result, digital images can be easily manipulated on a personal computer or even a cellphone for many purposes nowadays, so that the authenticity of digital images becomes an important issue. In this dissertation research, four machine learning based forensic methods are presented to detect one of the popular image editing techniques, called ‘seam carving’. To reveal seam carving applied to uncompressed images from the perspective of energy distribution change, an energy based statistical model is proposed as the first work in this dissertation. Features measured global energy of images, remaining optimal seams, and noise level are extracted from four local derivative pattern (LDP) domains instead of from the original pixel domain to heighten the energy change caused by seam carving. A support vector machine (SVM) based classifier is employed to determine whether an image has been seam carved or not. In the second work, an advanced feature model is presented for seam carving detection by investigating the statistical variation among neighboring pixels. Comprised with three types of statistical features, i.e., LDP features, Markov features, and SPAM features, the powerful feature model significantly improved the state-of-the-art accuracy in detecting low carving rate seam carving. After the feature selection by utilizing SVM based recursive feature elimination (SVM-RFE), with a small amount of features selected from the proposed model the overall performance is further improved. Combining above mentioned two works, a hybrid feature model is then proposed as the third work to further boost the accuracy in detecting seam carving at low carving rate. The proposed model consists of two sets of features, which capture energy change and neighboring relationship variation respectively, achieves remarkable performance on revealing seam carving, especially low carving rate seam carving, in digital images. Besides these three hand crafted feature models, a deep convolutional neural network is designed for seam carving detection. It is the first work that successfully utilizes deep learning technology to solve this forensic problem. The experimental works demonstrate their much more improved performance in the cases where the amount of seam carving is not serious. Although these four pieces of work move the seam carving detection ahead substantially, future research works with more advanced statistical model or deep neural network along this line are expected

Image Resizing using Seam Carving

Image resizing has become more necessary with the increased popularity of cell phones, tablets and other electronic devices with varying screen sizes. This paper presents methods for resizing images and videos while attempting to preserve the important content of that image or video. An algorithm called seam carving can expand or reduce the size of an image while typically maintaining quality and content. Seam carving is not always effective however and there have been recent developments and modifications on this algorithm. This paper presents two advancements on seam carving, one that optimizes image retargeting on images with many repeated objects or patterns. The other applies the method of seam carving to video resizing

Digital image forensics via meta-learning and few-shot learning

Digital images are a substantial portion of the information conveyed by social media, the Internet, and television in our daily life. In recent years, digital images have become not only one of the public information carriers, but also a crucial piece of evidence. The widespread availability of low-cost, user-friendly, and potent image editing software and mobile phone applications facilitates altering images without professional expertise. Consequently, safeguarding the originality and integrity of digital images has become a difficulty. Forgers commonly use digital image manipulation to transmit misleading information. Digital image forensics investigates the irregular patterns that might result from image alteration. It is crucial to information security. Over the past several years, machine learning techniques have been effectively used to identify image forgeries. Convolutional Neural Networks(CNN) are a frequent machine learning approach. A standard CNN model could distinguish between original and manipulated images. In this dissertation, two CNN models are introduced to recognize seam carving and Gaussian filtering. Training a conventional CNN model for a new similar image forgery detection task, one must start from scratch. Additionally, many types of tampered image data are challenging to acquire or simulate. Meta-learning is an alternative learning paradigm in which a machine learning model gets experience across numerous related tasks and uses this expertise to improve its future learning performance. Few-shot learning is a method for acquiring knowledge from few data. It can classify images with as few as one or two examples per class. Inspired by meta-learning and few-shot learning, this dissertation proposed a prototypical networks model capable of resolving a collection of related image forgery detection problems. Unlike traditional CNN models, the proposed prototypical networks model does not need to be trained from scratch for a new task. Additionally, it drastically decreases the quantity of training images

Content Aware Video Retargeting using Seam Carving

Video retargeting method achieves high - quality resizing to arbitrary aspect ratios for complex videos containing diverse camera and dynamic motions . Video retargeting from a full - resolution video to a lower resolution display will inevitably cause information loss. While retargeting the video the important contents must also be preserved. Seam carving works well for images without straight lines or regular patterns like landscape images but may cause distortions if used for images with straight lines. Our approach combines Seam Carving method along with Hough transform to preserve the origi nality of the video

Image resizing with minimum distortion

Displays became cheap and were combined with many devices, like camera, mobile, and so on…, so there has been an increased interest on resizing methods to make the image suitable and fill any screen size. Common and known methods like cropping or resampling can cause undesirable effects such as: losses in information or distortion in perception. Recently, content-aware image resizing methods have been proposed to get rid of these problems and produce exceptional results. Seam-carving produced by Avidan and Shamir has gained attention as an effective solution. This paper discussed about this method and used it to resize (minimize and maximize) four colored images vertically and horizontally respectively, and maintained the main features of the images by deleting or repeating only the uninfluenced features. The energy map was calculated that described the basic and influential details of the image using energy function. But instead of gradient function (as in Avidan and Shamir) entropy function was used to compute the energy of the images. A vertical or a horizontal seam of pixels with minimum energy values was either deleted or inserted to resize the image. Good results were obtained especially when the image contains spaces within its details. The work was programmed using Matlab2018a