The troublesome kernel: why deep learning for inverse problems is typically unstable

There is overwhelming empirical evidence that Deep Learning (DL) leads to unstable methods in applications ranging from image classification and computer vision to voice recognition and automated diagnosis in medicine. Recently, a similar instability phenomenon has been discovered when DL is used to solve certain problems in computational science, namely, inverse problems in imaging. In this paper we present a comprehensive mathematical analysis explaining the many facets of the instability phenomenon in DL for inverse problems. Our main results not only explain why this phenomenon occurs, they also shed light as to why finding a cure for instabilities is so difficult in practice. Additionally, these theorems show that instabilities are typically not rare events - rather, they can occur even when the measurements are subject to completely random noise - and consequently how easy it can be to destablise certain trained neural networks. We also examine the delicate balance between reconstruction performance and stability, and in particular, how DL methods may outperform state-of-the-art sparse regularization methods, but at the cost of instability. Finally, we demonstrate a counterintuitive phenomenon: training a neural network may generically not yield an optimal reconstruction method for an inverse problem

On the existence of optimal multi-valued decoders and their accuracy bounds for undersampled inverse problems

Undersampled inverse problems occur everywhere in the sciences including medical imaging, radar, astronomy etc., yielding underdetermined linear or non-linear reconstruction problems. There are now a myriad of techniques to design decoders that can tackle such problems, ranging from optimization based approaches, such as compressed sensing, to deep learning (DL), and variants in between the two techniques. The variety of methods begs for a unifying approach to determine the existence of optimal decoders and fundamental accuracy bounds, in order to facilitate a theoretical and empirical understanding of the performance of existing and future methods. Such a theory must allow for both single-valued and multi-valued decoders, as underdetermined inverse problems typically have multiple solutions. Indeed, multi-valued decoders arise due to non-uniqueness of minimizers in optimisation problems, such as in compressed sensing, and for DL based decoders in generative adversarial models, such as diffusion models and ensemble models. In this work we provide a framework for assessing the lowest possible reconstruction accuracy in terms of worst- and average-case errors. The universal bounds bounds only depend on the measurement model $F$, the model class $\mathcal{M}_1 \subseteq \mathcal{X}$ and the noise model $\mathcal{E}$. For linear $F$ these bounds depend on its kernel, and in the non-linear case the concept of kernel is generalized for undersampled settings. Additionally, we provide multi-valued variational solutions that obtain the lowest possible reconstruction error

Candida albicans repetitive elements display epigenetic diversity and plasticity

Transcriptionally silent heterochromatin is associated with repetitive DNA. It is poorly understood whether and how heterochromatin differs between different organisms and whether its structure can be remodelled in response to environmental signals. Here, we address this question by analysing the chromatin state associated with DNA repeats in the human fungal pathogen Candida albicans. Our analyses indicate that, contrary to model systems, each type of repetitive element is assembled into a distinct chromatin state. Classical Sir2-dependent hypoacetylated and hypomethylated chromatin is associated with the rDNA locus while telomeric regions are assembled into a weak heterochromatin that is only mildly hypoacetylated and hypomethylated. Major Repeat Sequences, a class of tandem repeats, are assembled into an intermediate chromatin state bearing features of both euchromatin and heterochromatin. Marker gene silencing assays and genome-wide RNA sequencing reveals that C. albicans heterochromatin represses expression of repeat-associated coding and non-coding RNAs. We find that telomeric heterochromatin is dynamic and remodelled upon an environmental change. Weak heterochromatin is associated with telomeres at 30?°C, while robust heterochromatin is assembled over these regions at 39?°C, a temperature mimicking moderate fever in the host. Thus in C. albicans, differential chromatin states controls gene expression and epigenetic plasticity is linked to adaptation

Impact of Chromatin Structures on DNA Processing for Genomic Analyses

Chromatin has an impact on recombination, repair, replication, and evolution of DNA. Here we report that chromatin structure also affects laboratory DNA manipulation in ways that distort the results of chromatin immunoprecipitation (ChIP) experiments. We initially discovered this effect at the Saccharomyces cerevisiae HMR locus, where we found that silenced chromatin was refractory to shearing, relative to euchromatin. Using input samples from ChIP-Seq studies, we detected a similar bias throughout the heterochromatic portions of the yeast genome. We also observed significant chromatin-related effects at telomeres, protein binding sites, and genes, reflected in the variation of input-Seq coverage. Experimental tests of candidate regions showed that chromatin influenced shearing at some loci, and that chromatin could also lead to enriched or depleted DNA levels in prepared samples, independently of shearing effects. Our results suggested that assays relying on immunoprecipitation of chromatin will be biased by intrinsic differences between regions packaged into different chromatin structures - biases which have been largely ignored to date. These results established the pervasiveness of this bias genome-wide, and suggested that this bias can be used to detect differences in chromatin structures across the genome

Telomere position effect is regulated by heterochromatin-associated proteins and NkuA in Aspergillus nidulans

Gene-silencing mechanisms are being shown to be associated with an increasing number of fungal developmental processes. Telomere position effect (TPE) is a eukaryotic phenomenon resulting in gene repression in areas immediately adjacent to telomere caps. Here, TPE is shown to regulate expression of transgenes on the left arm of chromosome III and the right arm of chromosome VI in Aspergillus nidulans. Phenotypes found to be associated with transgene repression included reduction in radial growth and the absence of sexual spores; however, these pleiotropic phenotypes were remedied when cultures were grown on media with appropriate supplementation. Simple radial growth and ascosporogenesis assays provided insights into the mechanism of TPE, including a means to determine its extent. These experiments revealed that the KU70 homologue (NkuA) and the heterochromatin-associated proteins HepA, ClrD and HdaA were partially required for transgene silencing. This study indicates that TPE extends at least 30 kb on chromosome III, suggesting that this phenomenon may be important for gene regulation in subtelomeric regions of A. nidulans

Analysis of gene expression data from non-small celllung carcinoma cell lines reveals distinct sub-classesfrom those identified at the phenotype level

Microarray data from cell lines of Non-Small Cell Lung Carcinoma (NSCLC) can be used to look for differences in gene expression between the cell lines derived from different tumour samples, and to investigate if these differences can be used to cluster the cell lines into distinct groups. Dividing the cell lines into classes can help to improve diagnosis and the development of screens for new drug candidates. The micro-array data is first subjected to quality control analysis and then subsequently normalised using three alternate methods to reduce the chances of differences being artefacts resulting from the normalisation process. The final clustering into sub-classes was carried out in a conservative manner such that subclasses were consistent across all three normalisation methods. If there is structure in the cell line population it was expected that this would agree with histological classifications, but this was not found to be the case. To check the biological consistency of the sub-classes the set of most strongly differentially expressed genes was be identified for each pair of clusters to check if the genes that most strongly define sub-classes have biological functions consistent with NSCLC

Sub-Telomeric core X and Y' Elements in S.cerevisiae Suppress Extreme Variations in Gene Silencing

Telomere Position Effect (TPE) is governed by strong repression signals emitted by telomeres via the Sir2/3/4 Histone Deacetylase complex. These signals are then relayed by weak proto-silencers residing in the subtelomeric core X and Y' elements. Subtelomeres also contain Sub-Telomeric Anti-silencing Regions (STARs). In this study we have prepared telomeres built of different combinations of core X, Y' and STARs and have analyzed them in strains lacking Histone-Acetyltransferase genes as well as in cdc6-1 and Δrif1 strains. We show that core X and Y' dramatically reduce both positive and negative variations in TPE, that are caused by these mutations. We also show that the deletion of Histone-Acetyltransferase genes reduce the silencing activity of an ACS proto-silencer, but also reduce the anti-silencing activity of a STAR. We postulate that core X and Y' act as epigenetic “cushioning” cis-elements

Molecular identification of papillomavirus in ducks

Papillomaviruses infect many vertebrates, including birds. Persistent infections by some strains can cause malignant proliferation of cells (i.e. cancer), though more typically infections cause benign tumours, or may be completely subclinical. Sometimes extensive, persistent tumours are recorded– notably in chaffinches and humans. In 2016, a novel papillomavirus genotype was characterized from a duck faecal microbiome, in Bhopal, India; the sixth papillomavirus genotype from birds. Prompted by this finding, we screened 160 cloacal swabs and 968 faecal samples collected from 299 ducks sampled at Ottenby Bird Observatory, Sweden in 2015, using a newly designed real-time PCR. Twenty one samples (1.9%) from six individuals (2%) were positive. Eighteen sequences were identical to the published genotype, duck papillomavirus 1. One additional novel genotype was recovered from three samples. Both genotypes were recovered from a wild strain domestic mallard that was infected for more than 60 days with each genotype. All positive individuals were adult (P = 0.004). Significantly more positive samples were detected from swabs than faecal samples (P < 0.0001). Sample type data suggests transmission may be via direct contact, and only infrequently, via the oral-faecal route. Infection in only adult birds supports the hypothesis that this virus is sexually transmitted, though more work is required to verify this.Thanks to duck trappers at Ottenby Bird Observatory for support and sample collection, and to Abbtesaim Jawad for DNA extraction. This work was supported by the Crafoord Foundation Sweden (grants number 20160971 and 20170671). This is contribution no. 306 from Ottenby Bird Observatory

Silent but Not Static: Accelerated Base-Pair Substitution in Silenced Chromatin of Budding Yeasts

Subtelomeric DNA in budding yeasts, like metazoan heterochromatin, is gene poor, repetitive, transiently silenced, and highly dynamic. The rapid evolution of subtelomeric regions is commonly thought to arise from transposon activity and increased recombination between repetitive elements. However, we found evidence of an additional factor in this diversification. We observed a surprising level of nucleotide divergence in transcriptionally silenced regions in inter-species comparisons of Saccharomyces yeasts. Likewise, intra-species analysis of polymorphisms also revealed increased SNP frequencies in both intergenic and synonymous coding positions of silenced DNA. This analysis suggested that silenced DNA in Saccharomyces cerevisiae and closely related species had increased single base-pair substitution that was likely due to the effects of the silencing machinery on DNA replication or repair