Colorization of Multispectral Image Fusion using Convolutional Neural Network approach

The proposed technique  offers a significant advantage in enhancing multiband nighttime imagery for surveillance and navigation purposes., The multi-band image data set comprises visual  and infrared  motion sequences with various military and civilian surveillance scenarios which include people that are stationary, walking or running, Vehicles and buildings or other man-made structures. Colorization method led to provide superior discrimination, identification of objects (Lesions), faster reaction times and an increased scene understanding than monochrome fused image. The guided filtering approach is used to decompose the source images hence they are divided into two parts: approximation part and detail content part further the weighted-averaging method is used to fuse the approximation part. The multi-layer features are extracted from the detail content part using the VGG-19 network. Finally, the approximation part and detail content part will be combined to reconstruct the fused image. The proposed approach has offers better outcomes equated to prevailing state-of-the-art techniques in terms of quantitative and qualitative parameters. In future, propose technique will help Battlefield monitoring, Defence for situation awareness, Surveillance, Target tracking and Person authentication

The Missing Data Encoder: Cross-Channel Image Completion\\with Hide-And-Seek Adversarial Network

Image completion is the problem of generating whole images from fragments only. It encompasses inpainting (generating a patch given its surrounding), reverse inpainting/extrapolation (generating the periphery given the central patch) as well as colorization (generating one or several channels given other ones). In this paper, we employ a deep network to perform image completion, with adversarial training as well as perceptual and completion losses, and call it the ``missing data encoder'' (MDE). We consider several configurations based on how the seed fragments are chosen. We show that training MDE for ``random extrapolation and colorization'' (MDE-REC), i.e. using random channel-independent fragments, allows a better capture of the image semantics and geometry. MDE training makes use of a novel ``hide-and-seek'' adversarial loss, where the discriminator seeks the original non-masked regions, while the generator tries to hide them. We validate our models both qualitatively and quantitatively on several datasets, showing their interest for image completion, unsupervised representation learning as well as face occlusion handling

Example-based image colorization using locality consistent sparse representation

—Image colorization aims to produce a natural looking color image from a given grayscale image, which remains a challenging problem. In this paper, we propose a novel examplebased image colorization method exploiting a new locality consistent sparse representation. Given a single reference color image, our method automatically colorizes the target grayscale image by sparse pursuit. For efficiency and robustness, our method operates at the superpixel level. We extract low-level intensity features, mid-level texture features and high-level semantic features for each superpixel, which are then concatenated to form its descriptor. The collection of feature vectors for all the superpixels from the reference image composes the dictionary. We formulate colorization of target superpixels as a dictionary-based sparse reconstruction problem. Inspired by the observation that superpixels with similar spatial location and/or feature representation are likely to match spatially close regions from the reference image, we further introduce a locality promoting regularization term into the energy formulation which substantially improves the matching consistency and subsequent colorization results. Target superpixels are colorized based on the chrominance information from the dominant reference superpixels. Finally, to further improve coherence while preserving sharpness, we develop a new edge-preserving filter for chrominance channels with the guidance from the target grayscale image. To the best of our knowledge, this is the first work on sparse pursuit image colorization from single reference images. Experimental results demonstrate that our colorization method outperforms state-ofthe-art methods, both visually and quantitatively using a user stud

IST Austria Thesis

Modern computer vision systems heavily rely on statistical machine learning models, which typically require large amounts of labeled data to be learned reliably. Moreover, very recently computer vision research widely adopted techniques for representation learning, which further increase the demand for labeled data. However, for many important practical problems there is relatively small amount of labeled data available, so it is problematic to leverage full potential of the representation learning methods. One way to overcome this obstacle is to invest substantial resources into producing large labelled datasets. Unfortunately, this can be prohibitively expensive in practice. In this thesis we focus on the alternative way of tackling the aforementioned issue. We concentrate on methods, which make use of weakly-labeled or even unlabeled data. Specifically, the first half of the thesis is dedicated to the semantic image segmentation task. We develop a technique, which achieves competitive segmentation performance and only requires annotations in a form of global image-level labels instead of dense segmentation masks. Subsequently, we present a new methodology, which further improves segmentation performance by leveraging tiny additional feedback from a human annotator. By using our methods practitioners can greatly reduce the amount of data annotation effort, which is required to learn modern image segmentation models. In the second half of the thesis we focus on methods for learning from unlabeled visual data. We study a family of autoregressive models for modeling structure of natural images and discuss potential applications of these models. Moreover, we conduct in-depth study of one of these applications, where we develop the state-of-the-art model for the probabilistic image colorization task

Dynamic Weights Equations for Converting Grayscale Image to RGB Image

طريقة تحويل الصور الملونة من نظام الوان العرض إلى الصور الرمادي هو عملية بسيطة باستخدام طريقة الأوزان الثابتة للتحويل، ولكن باستخدام نفس الأوزان لاستعادة اللون من نفس الصور ليست عملية فعالة لجميع أنواع الصور لأن الصورة الرمادية تحتوي على معلومات قليلة، وغير كافية لاجراء عملية التحويل. الفكرة الأساسية في هذا البحث هي استخدام المعادلات الرياضية المستخرجة من الصورة الرمادية في عملية التحويل ،حيث يقدم هذا البحث طريقة تلوين الصورة الرمادية باستخدام الأوزان المستمدة من خصائص الصورة الرمادية. وقد تم استخراج مقياس (الانحراف، المتوسط، والانحراف المعياري) من خصائص الصور الرمادية واعتمادها في تحديد الأوزان اللازمة لنظام الوان العرض. أثبتت هذه الطريقة نجاحها في تلوين الصور مقارنة مع الطريقة التقليدية المعتمدة على الأوزان الثابتة لتلوين الصور لأنها تعتمد على الأوزان الثابتة لتحويل جميع الصور انواع الصور الرمادية.The method of converting color images from the RGB color system to grayscale images is a simple operation by using the fixed weights method of conversion, but using the same weights to restore the color of the same images is not an effective operation of all types of images because the grayscale image contains little information and it isn't worthy of conversion operation. The basic idea in this paper is to employ the mathematics equations which extracted from the grayscale image in conversion operation, this paper presents the method of coloring the grayscale image by using the weights derived from the characteristics of the grayscale image. Skewness, Mean and Standard deviation moments have been extracted from the features of grayscale images and its adoption the determine weights of the RGB color system. This method proved its success in coloring images compared to the traditional method adoption of fixed weights for coloring images because it relies on fixed weights for converting all grayscale images

Discovery of Visual Semantics by Unsupervised and Self-Supervised Representation Learning

The success of deep learning in computer vision is rooted in the ability of deep networks to scale up model complexity as demanded by challenging visual tasks. As complexity is increased, so is the need for large amounts of labeled data to train the model. This is associated with a costly human annotation effort. To address this concern, with the long-term goal of leveraging the abundance of cheap unlabeled data, we explore methods of unsupervised "pre-training." In particular, we propose to use self-supervised automatic image colorization. We show that traditional methods for unsupervised learning, such as layer-wise clustering or autoencoders, remain inferior to supervised pre-training. In search for an alternative, we develop a fully automatic image colorization method. Our method sets a new state-of-the-art in revitalizing old black-and-white photography, without requiring human effort or expertise. Additionally, it gives us a method for self-supervised representation learning. In order for the model to appropriately re-color a grayscale object, it must first be able to identify it. This ability, learned entirely self-supervised, can be used to improve other visual tasks, such as classification and semantic segmentation. As a future direction for self-supervision, we investigate if multiple proxy tasks can be combined to improve generalization. This turns out to be a challenging open problem. We hope that our contributions to this endeavor will provide a foundation for future efforts in making self-supervision compete with supervised pre-training.Comment: Ph.D. thesi