Detection of dirt impairments from archived film sequences : survey and evaluations

Film dirt is the most commonly encountered artifact in archive restoration applications. Since dirt usually appears as a temporally impulsive event, motion-compensated interframe processing is widely applied for its detection. However, motion-compensated prediction requires a high degree of complexity and can be unreliable when motion estimation fails. Consequently, many techniques using spatial or spatiotemporal filtering without motion were also been proposed as alternatives. A comprehensive survey and evaluation of existing methods is presented, in which both qualitative and quantitative performances are compared in terms of accuracy, robustness, and complexity. After analyzing these algorithms and identifying their limitations, we conclude with guidance in choosing from these algorithms and promising directions for future research

A cockpit of multiple measures for assessing film restoration quality

In machine vision, the idea of expressing the quality of a films by a single value is very popular. Usually this value is computed by processing a set of image features with the aim of resembling as much as pos- sible a kind of human judgment of the film quality. Since human quality assessment is a complex mech- anism involving many different perceptual aspects, we believe that such approach may scarcely provide a comprehensive analysis. Especially in the field of digital movie restoration, a single score can hardly provide reliable information about the effects of the various restoring operations. For this reason we in- troduce an alternative approach, where a set of measures, describing over time basic global and local visual properties of the film frames, is computed in an unsupervised way and delivered to expert evalu- ators for checking the restoration pipeline and results. The proposed framework can be viewed as a car or airplane cockpit , whose parameters (i.e. the computed measures) are necessary to control the machine status and performance. This cockpit, which is publicly available online, would like to support the digital restoration process and its assessment

Temporal structure in spiking patterns of ganglion cells defines perceptual thresholds in rodents with subretinal prosthesis.

Subretinal prostheses are designed to restore sight in patients blinded by retinal degeneration using electrical stimulation of the inner retinal neurons. To relate retinal output to perception, we studied behavioral thresholds in blind rats with photovoltaic subretinal prostheses stimulated by full-field pulsed illumination at 20 Hz, and measured retinal ganglion cell (RGC) responses to similar stimuli ex-vivo. Behaviorally, rats exhibited startling response to changes in brightness, with an average contrast threshold of 12%, which could not be explained by changes in the average RGC spiking rate. However, RGCs exhibited millisecond-scale variations in spike timing, even when the average rate did not change significantly. At 12% temporal contrast, changes in firing patterns of prosthetic response were as significant as with 2.3% contrast steps in visible light stimulation of healthy retinas. This suggests that millisecond-scale changes in spiking patterns define perceptual thresholds of prosthetic vision. Response to the last pulse in the stimulation burst lasted longer than the steady-state response during the burst. This may be interpreted as an excitatory OFF response to prosthetic stimulation, and can explain behavioral response to decrease in illumination. Contrast enhancement of images prior to delivery to subretinal prosthesis can partially compensate for reduced contrast sensitivity of prosthetic vision

Artificial Intelligence in the Creative Industries: A Review

This paper reviews the current state of the art in Artificial Intelligence (AI) technologies and applications in the context of the creative industries. A brief background of AI, and specifically Machine Learning (ML) algorithms, is provided including Convolutional Neural Network (CNNs), Generative Adversarial Networks (GANs), Recurrent Neural Networks (RNNs) and Deep Reinforcement Learning (DRL). We categorise creative applications into five groups related to how AI technologies are used: i) content creation, ii) information analysis, iii) content enhancement and post production workflows, iv) information extraction and enhancement, and v) data compression. We critically examine the successes and limitations of this rapidly advancing technology in each of these areas. We further differentiate between the use of AI as a creative tool and its potential as a creator in its own right. We foresee that, in the near future, machine learning-based AI will be adopted widely as a tool or collaborative assistant for creativity. In contrast, we observe that the successes of machine learning in domains with fewer constraints, where AI is the `creator', remain modest. The potential of AI (or its developers) to win awards for its original creations in competition with human creatives is also limited, based on contemporary technologies. We therefore conclude that, in the context of creative industries, maximum benefit from AI will be derived where its focus is human centric -- where it is designed to augment, rather than replace, human creativity

Pixel-level Image Fusion Algorithms for Multi-camera Imaging System

This thesis work is motivated by the potential and promise of image fusion technologies in the multi sensor image fusion system and applications. With specific focus on pixel level image fusion, the process after the image registration is processed, we develop graphic user interface for multi-sensor image fusion software using Microsoft visual studio and Microsoft Foundation Class library. In this thesis, we proposed and presented some image fusion algorithms with low computational cost, based upon spatial mixture analysis. The segment weighted average image fusion combines several low spatial resolution data source from different sensors to create high resolution and large size of fused image. This research includes developing a segment-based step, based upon stepwise divide and combine process. In the second stage of the process, the linear interpolation optimization is used to sharpen the image resolution. Implementation of these image fusion algorithms are completed based on the graphic user interface we developed. Multiple sensor image fusion is easily accommodated by the algorithm, and the results are demonstrated at multiple scales. By using quantitative estimation such as mutual information, we obtain the experiment quantifiable results. We also use the image morphing technique to generate fused image sequence, to simulate the results of image fusion. While deploying our pixel level image fusion algorithm approaches, we observe several challenges from the popular image fusion methods. While high computational cost and complex processing steps of image fusion algorithms provide accurate fused results, they also makes it hard to become deployed in system and applications that require real-time feedback, high flexibility and low computation abilit

Efficient Bayesian-based Multi-View Deconvolution

Light sheet fluorescence microscopy is able to image large specimen with high resolution by imaging the sam- ples from multiple angles. Multi-view deconvolution can significantly improve the resolution and contrast of the images, but its application has been limited due to the large size of the datasets. Here we present a Bayesian- based derivation of multi-view deconvolution that drastically improves the convergence time and provide a fast implementation utilizing graphics hardware.Comment: 48 pages, 20 figures, 1 table, under review at Nature Method

Optimal sparsity allows reliable system-aware restoration of fluorescence microscopy images

Incluye: artículo, material suplementario, videos y software.Fluorescence microscopy is one of the most indispensable and informative driving forces for biological research, but the extent of observable biological phenomena is essentially determined by the content and quality of the acquired images. To address the different noise sources that can degrade these images, we introduce an algorithm for multiscale image restoration through optimally sparse representation (MIRO). MIRO is a deterministic framework that models the acquisition process and uses pixelwise noise correction to improve image quality. Our study demonstrates that this approach yields a remarkable restoration of the fluorescence signal for a wide range of microscopy systems, regardless of the detector used (e.g., electron-multiplying charge-coupled device, scientific complementary metal-oxide semiconductor, or photomultiplier tube). MIRO improves current imaging capabilities, enabling fast, low-light optical microscopy, accurate image analysis, and robust machine intelligence when integrated with deep neural networks. This expands the range of biological knowledge that can be obtained from fluorescence microscopy.We acknowledge the support of the National Institutes of Health grants R35GM124846 (to S.J.) and R01AA028527 (to C.X.), the National Science Foundation grants BIO2145235 and EFMA1830941 (to S.J.), and Marvin H. and Nita S. Floyd Research Fund (to S.J.). This research project was supported, in part, by the Emory University Integrated Cellular Imaging Microscopy Core and by PHS Grant UL1TR000454 from the Clinical and Translational Science Award Program, National Institutes of Health, and National Center for Advancing Translational Sciences.S

Multi-Directional Scratch Detection and Restoration in Digitized Images

Line scratches are common defects in old archived videos, but similar imperfections may occur in printed images, in most cases by reason of improper handling or inaccurate preservation of the support. Once an image is digitized, its defects become part of that image. Many state-of-the-art papers deal with long, thin, vertical lines in old movie frames, by exploiting both spatial and temporal information. In this paper we aim to face with a more challenging and general problem: the analysis of line scratches in still images, regardless of their orientation, color, and shape. We present a detection/restoration method to process this defect