Visual Task Performance Assessment using Complementary and Redundant Information within Fused Imagery

Image fusion is the process of combining information from a set of source images to obtain a single image with more relevant information than any individual source image. The intent of image fusion is to produce a single image that renders a better description of the scene than any of the individual source images. Information within source images can be classified as either redundant or complementary. The relevant amounts of complementary and redundant information within the source images provide an effective metric for quantifying the benefits of image fusion. Two common reasons for using image fusion for a particular task are to increase task reliability or to increase capability. It seems natural to associate reliability with redundancy of information between source bands, whereas increased capability is associated with complementary information between source bands. The basic idea is that the more redundant the information between the source images being fused, the less likely an increase in task performance can be realized using the fused imagery. Intuitively, the benefits of image fusion with regards to task performance are maximized when the source images contain large amounts of complementary information. This research introduces a new performance measure based on mutual information which, under the assumption the fused imagery has been properly prepared for human perception, can be used as a predictor of human task performance using the complementary and redundant information in fused imagery. The ability of human observers to identify targets of interest using fused imagery is evaluated using human perception experiments. In the perception experiments, imagery of the same scenes containing targets of interest, captured in different spectral bands, is fused using various fusion algortihms and shown to human observers for identification. The results of the experiments show a correlation exists between the proposed measure and human visual identification task performance. The perception experiments serve to validate the performance prediction accuracy of the new performance measure. the development of the proposed metric introduces into the image fusion community a new image fusion evaluation measure that has the potential to fill many voids within the image fusion literature

Deep learning-based image deconstruction method with maintained saliency

人間の視覚注意解析のための人工知能（AI）技術の開発に成功. 京都大学プレスリリース. 2022-09-29.Visual properties that primarily attract bottom-up attention are collectively referred to as saliency. In this study, to understand the neural activity involved in top-down and bottom-up visual attention, we aim to prepare pairs of natural and unnatural images with common saliency. For this purpose, we propose an image transformation method based on deep neural networks that can generate new images while maintaining the consistent feature map, in particular the saliency map. This is an ill-posed problem because the transformation from an image to its corresponding feature map could be many-to-one, and in our particular case, the various images would share the same saliency map. Although stochastic image generation has the potential to solve such ill-posed problems, the most existing methods focus on adding diversity of the overall style/touch information while maintaining the naturalness of the generated images. To this end, we developed a new image transformation method that incorporates higher-dimensional latent variables so that the generated images appear unnatural with less context information but retain a high diversity of local image structures. Although such high-dimensional latent spaces are prone to collapse, we proposed a new regularization based on Kullback–Leibler divergence to avoid collapsing the latent distribution. We also conducted human experiments using our newly prepared natural and corresponding unnatural images to measure overt eye movements and functional magnetic resonance imaging, and found that those images induced distinctive neural activities related to top-down and bottom-up attentional processing

Computational Single-Cell Analysis Of Confocal Fluorescence Images With Dapi-Generated Masks

Lipolysis is a metabolic pathway in which free fatty acids are mobilized from stored triglycerides. The rate-limiting enzyme in this process is adipose triglyceride lipase, which is regulated by α/β-hydrolase domain-containing protein 5 (ABHD5) via both natural and synthetic pathways. With advanced artificial neural networks, image processing methods can extract quantitative results from fluorescence images. The segmentation of complex biological images, in which regions of the image are labeled as distinct masks, is the first step in image analysis. Ilastik, a machine-learning software, performs image segmentation with a user-trained neural network and custom key feature labels. The software’s results are evaluated using a custom Python script, resulting in a new workflow that incorporates Ilastik for the construction of single-cell data from confocal fluorescence images. We analyzed multi-color fluorescence images of tissues to determine the growth and metabolism of lipid droplets. Moreover, the use of neural-network-based fluorescence image analysis to measure single-cell triglyceride storage and ABHD5 expression upon stimulation with isoproterenol, SR3420, dimethyl sulfoxide, or forskolin is reported. We demonstrate enhanced quantitative information for hypothesis testing in the assessment of single-cell behaviors and metabolic pathways

Improving the Canny Edge Detector Using Automatic Programming: Improving Hysteresis Thresholding

We have used automatic programming to improve the hysteresis thresholding stage of the popular Canny edge detector—without increasing the computational complexity or adding extra information. The F-measure has been increased by 1.8% on a test set of natural images, and a paired student-t test and a Wilcoxon signed rank test show that the improvement is statistically significant. This is the first time evolutionary computation and automatic programming has been used to improve hysteresis thresholding. The new program has introduced complex recursive patterns that make the algorithm perform better with weak edges and retain more detail. The findings provide further evidence that an automatically designed algorithm can outperform manually created algorithms on low level image analysis problems, and that automatic programming is ideally suited for inferring suitable heuristics for such problems

The Spatial Vision Tree: A Generic Pattern Recognition Engine- Scientific Foundations, Design Principles, and Preliminary Tree Design

New foundational ideas are used to define a novel approach to generic visual pattern recognition. These ideas proceed from the starting point of the intrinsic equivalence of noise reduction and pattern recognition when noise reduction is taken to its theoretical limit of explicit matched filtering. This led us to think of the logical extension of sparse coding using basis function transforms for both de-noising and pattern recognition to the full pattern specificity of a lexicon of matched filter pattern templates. A key hypothesis is that such a lexicon can be constructed and is, in fact, a generic visual alphabet of spatial vision. Hence it provides a tractable solution for the design of a generic pattern recognition engine. Here we present the key scientific ideas, the basic design principles which emerge from these ideas, and a preliminary design of the Spatial Vision Tree (SVT). The latter is based upon a cryptographic approach whereby we measure a large aggregate estimate of the frequency of occurrence (FOO) for each pattern. These distributions are employed together with Hamming distance criteria to design a two-tier tree. Then using information theory, these same FOO distributions are used to define a precise method for pattern representation. Finally the experimental performance of the preliminary SVT on computer generated test images and complex natural images is assessed

Shape index descriptors applied to texture-based galaxy analysis

A texture descriptor based on the shape index and the accompanying curvedness measure is proposed, and it is evaluated for the automated analysis of astronomical image data. A representative sample of images of low-redshift galaxies from the Sloan Digital Sky Survey (SDSS) serves as a testbed. The goal of applying texture descriptors to these data is to extract novel information about galaxies; information which is often lost in more traditional analysis. In this study, we build a regression model for predicting a spectroscopic quantity, the specific star-formation rate (sSFR). As texture features we consider multi-scale gradient orientation histograms as well as multi-scale shape index histograms, which lead to a new descriptor. Our results show that we can successfully predict spectroscopic quantities from the texture in optical multi-band images. We successfully recover the observed bi-modal distribution of galaxies into quiescent and star-forming. The state-ofthe-art for predicting the sSFR is a color-based physical model. We significantly improve its accuracy by augmenting the model with texture information. This study is the first step towards enabling the quantification of physical galaxy properties from imaging data alone.The Danish Council for Independent Research | Natural Sciences through the project "Surveying the sky using machine learning" (FNU 12-125149