"Zero-Shot" Super-Resolution using Deep Internal Learning

Deep Learning has led to a dramatic leap in Super-Resolution (SR) performance in the past few years. However, being supervised, these SR methods are restricted to specific training data, where the acquisition of the low-resolution (LR) images from their high-resolution (HR) counterparts is predetermined (e.g., bicubic downscaling), without any distracting artifacts (e.g., sensor noise, image compression, non-ideal PSF, etc). Real LR images, however, rarely obey these restrictions, resulting in poor SR results by SotA (State of the Art) methods. In this paper we introduce "Zero-Shot" SR, which exploits the power of Deep Learning, but does not rely on prior training. We exploit the internal recurrence of information inside a single image, and train a small image-specific CNN at test time, on examples extracted solely from the input image itself. As such, it can adapt itself to different settings per image. This allows to perform SR of real old photos, noisy images, biological data, and other images where the acquisition process is unknown or non-ideal. On such images, our method outperforms SotA CNN-based SR methods, as well as previous unsupervised SR methods. To the best of our knowledge, this is the first unsupervised CNN-based SR method

An image processing approach to computing distances between RNA secondary structures dot plots

<p>Abstract</p> <p>Background</p> <p>Computing the distance between two RNA secondary structures can contribute in understanding the functional relationship between them. When used repeatedly, such a procedure may lead to finding a query RNA structure of interest in a database of structures. Several methods are available for computing distances between RNAs represented as strings or graphs, but none utilize the RNA representation with dot plots. Since dot plots are essentially digital images, there is a clear motivation to devise an algorithm for computing the distance between dot plots based on image processing methods.</p> <p>Results</p> <p>We have developed a new metric dubbed 'DoPloCompare', which compares two RNA structures. The method is based on comparing dot plot diagrams that represent the secondary structures. When analyzing two diagrams and motivated by image processing, the distance is based on a combination of histogram correlations and a geometrical distance measure. We introduce, describe, and illustrate the procedure by two applications that utilize this metric on RNA sequences. The first application is the RNA design problem, where the goal is to find the nucleotide sequence for a given secondary structure. Examples where our proposed distance measure outperforms others are given. The second application locates peculiar point mutations that induce significant structural alternations relative to the wild type predicted secondary structure. The approach reported in the past to solve this problem was tested on several RNA sequences with known secondary structures to affirm their prediction, as well as on a data set of ribosomal pieces. These pieces were computationally cut from a ribosome for which an experimentally derived secondary structure is available, and on each piece the prediction conveys similarity to the experimental result. Our newly proposed distance measure shows benefit in this problem as well when compared to standard methods used for assessing the distance similarity between two RNA secondary structures.</p> <p>Conclusion</p> <p>Inspired by image processing and the dot plot representation for RNA secondary structure, we have managed to provide a conceptually new and potentially beneficial metric for comparing two RNA secondary structures. We illustrated our approach on the RNA design problem, as well as on an application that utilizes the distance measure to detect conformational rearranging point mutations in an RNA sequence.</p

From voxels to pixels and back: Self-supervision in natural-image reconstruction from fMRI

Reconstructing observed images from fMRI brain recordings is challenging. Unfortunately, acquiring sufficient "labeled" pairs of {Image, fMRI} (i.e., images with their corresponding fMRI responses) to span the huge space of natural images is prohibitive for many reasons. We present a novel approach which, in addition to the scarce labeled data (training pairs), allows to train fMRI-to-image reconstruction networks also on "unlabeled" data (i.e., images without fMRI recording, and fMRI recording without images). The proposed model utilizes both an Encoder network (image-to-fMRI) and a Decoder network (fMRI-to-image). Concatenating these two networks back-to-back (Encoder-Decoder & Decoder-Encoder) allows augmenting the training with both types of unlabeled data. Importantly, it allows training on the unlabeled test-fMRI data. This self-supervision adapts the reconstruction network to the new input test-data, despite its deviation from the statistics of the scarce training data.Comment: *First two authors contributed equally. NeurIPS 201

Mentalizing and motivation neural function during social interactions in autism spectrum disorders

AbstractAutism Spectrum Disorders (ASDs) are characterized by core deficits in social functions. Two theories have been suggested to explain these deficits: mind-blindness theory posits impaired mentalizing processes (i.e. decreased ability for establishing a representation of others' state of mind), while social motivation theory proposes that diminished reward value for social information leads to reduced social attention, social interactions, and social learning. Mentalizing and motivation are integral to typical social interactions, and neuroimaging evidence points to independent brain networks that support these processes in healthy individuals. However, the simultaneous function of these networks has not been explored in individuals with ASDs. We used a social, interactive fMRI task, the Domino game, to explore mentalizing- and motivation-related brain activation during a well-defined interval where participants respond to rewards or punishments (i.e. motivation) and concurrently process information about their opponent's potential next actions (i.e. mentalizing). Thirteen individuals with high-functioning ASDs, ages 12–24, and 14 healthy controls played fMRI Domino games against a computer-opponent and separately, what they were led to believe was a human-opponent. Results showed that while individuals with ASDs understood the game rules and played similarly to controls, they showed diminished neural activity during the human-opponent runs only (i.e. in a social context) in bilateral middle temporal gyrus (MTG) during mentalizing and right Nucleus Accumbens (NAcc) during reward-related motivation (Pcluster<0.05 FWE). Importantly, deficits were not observed in these areas when playing against a computer-opponent or in areas related to motor and visual processes. These results demonstrate that while MTG and NAcc, which are critical structures in the mentalizing and motivation networks, respectively, activate normally in a non-social context, they fail to respond in an otherwise identical social context in ASD compared to controls. We discuss implications to both the mind-blindness and social motivation theories of ASD and the importance of social context in research and treatment protocols