Content-Aware Image Restoration Techniques without Ground Truth and Novel Ideas to Image Reconstruction

In this thesis I will use state-of-the-art (SOTA) image denoising methods to denoise electron microscopy (EM) data. Then, I will present NoiseVoid a deep learning based self-supervised image denoising approach which is trained on single noisy observations. Eventually, I approach the missing wedge problem in tomography and introduce a novel image encoding, based on the Fourier transform which I am using to predict missing Fourier coefficients directly in Fourier space with Fourier Image Transformer (FIT). In the next paragraphs I will summarize the individual contributions briefly. Electron microscopy is the go to method for high-resolution images in biological research. Modern scanning electron microscopy (SEM) setups are used to obtain neural connectivity maps, allowing us to identify individual synapses. However, slow scanning speeds are required to obtain SEM images of sufficient quality. In (Weigert et al. 2018) the authors show, for fluorescence microscopy, how pairs of low- and high-quality images can be obtained from biological samples and use them to train content-aware image restoration (CARE) networks. Once such a network is trained, it can be applied to noisy data to restore high quality images. With SEM-CARE I present how this approach can be directly applied to SEM data, allowing us to scan the samples faster, resulting in $40$- to $50$-fold imaging speedups for SEM imaging. In structural biology cryo transmission electron microscopy (cryo TEM) is used to resolve protein structures and describe molecular interactions. However, missing contrast agents as well as beam induced sample damage (Knapek and Dubochet 1980) prevent acquisition of high quality projection images. Hence, reconstructed tomograms suffer from low signal-to-noise ratio (SNR) and low contrast, which makes post-processing of such data difficult and often has to be done manually. To facilitate down stream analysis and manual data browsing of cryo tomograms I present cryoCARE a Noise2Noise (Lehtinen et al. 2018) based denoising method which is able to restore high contrast, low noise tomograms from sparse-view low-dose tilt-series. An implementation of cryoCARE is publicly available as Scipion (de la Rosa-Trevín et al. 2016) plugin. Next, I will discuss the problem of self-supervised image denoising. With cryoCARE I exploited the fact that modern cryo TEM cameras acquire multiple low-dose images, hence the Noise2Noise (Lehtinen et al. 2018) training paradigm can be applied. However, acquiring multiple noisy observations is not always possible e.g. in live imaging, with old cryo TEM cameras or simply by lack of access to the used imaging system. In such cases we have to fall back to self-supervised denoising methods and with Noise2Void I present the first self-supervised neural network based image denoising approach. Noise2Void is also available as an open-source Python package and as a one-click solution in Fiji (Schindelin et al. 2012). In the last part of this thesis I present Fourier Image Transformer (FIT) a novel approach to image reconstruction with Transformer networks. I develop a novel 1D image encoding based on the Fourier transform where each prefix encodes the whole image at reduced resolution, which I call Fourier Domain Encoding (FDE). I use FIT with FDEs and present proof of concept for super-resolution and tomographic reconstruction with missing wedge correction. The missing wedge artefacts in tomographic imaging originate in sparse-view imaging. Sparse-view imaging is used to keep the total exposure of the imaged sample to a minimum, by only acquiring a limited number of projection images. However, tomographic reconstructions from sparse-view acquisitions are affected by missing wedge artefacts, characterized by missing wedges in the Fourier space and visible as streaking artefacts in real image space. I show that FITs can be applied to tomographic reconstruction and that they fill in missing Fourier coefficients. Hence, FIT for tomographic reconstruction solves the missing wedge problem at its source.:Contents Summary iii Acknowledgements v 1 Introduction 1 1.1 Scanning Electron Microscopy . . . . . . . . . . . . . . . . . . . . 3 1.2 Cryo Transmission Electron Microscopy . . . . . . . . . . . . . . . 4 1.2.1 Single Particle Analysis . . . . . . . . . . . . . . . . . . . . 5 1.2.2 Cryo Tomography . . . . . . . . . . . . . . . . . . . . . . . 7 1.3 Tomographic Reconstruction . . . . . . . . . . . . . . . . . . . . . 8 1.4 Overview and Contributions . . . . . . . . . . . . . . . . . . . . . 11 2 Denoising in Electron Microscopy 15 2.1 Image Denoising . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 2.2 Supervised Image Restoration . . . . . . . . . . . . . . . . . . . . 19 2.2.1 Training and Validation Loss . . . . . . . . . . . . . . . . 19 2.2.2 Neural Network Architectures . . . . . . . . . . . . . . . . 21 2.3 SEM-CARE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 2.3.1 SEM-CARE Experiments . . . . . . . . . . . . . . . . . . 23 2.3.2 SEM-CARE Results . . . . . . . . . . . . . . . . . . . . . 25 2.4 Noise2Noise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 2.5 cryoCARE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 2.5.1 Restoration of cryo TEM Projections . . . . . . . . . . . . 27 2.5.2 Restoration of cryo TEM Tomograms . . . . . . . . . . . . 29 2.5.3 Automated Downstream Analysis . . . . . . . . . . . . . . 31 2.6 Implementations and Availability . . . . . . . . . . . . . . . . . . 32 2.7 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 2.7.1 Tasks Facilitated through cryoCARE . . . . . . . . . . . 33 3 Noise2Void: Self-Supervised Denoising 35 3.1 Probabilistic Image Formation . . . . . . . . . . . . . . . . . . . . 37 3.2 Receptive Field . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 3.3 Noise2Void Training . . . . . . . . . . . . . . . . . . . . . . . . . 39 3.3.1 Implementation Details . . . . . . . . . . . . . . . . . . . . 41 3.4 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 3.4.1 Natural Images . . . . . . . . . . . . . . . . . . . . . . . . 43 3.4.2 Light Microscopy Data . . . . . . . . . . . . . . . . . . . . 44 3.4.3 Electron Microscopy Data . . . . . . . . . . . . . . . . . . 47 3.4.4 Errors and Limitations . . . . . . . . . . . . . . . . . . . . 48 3.5 Conclusion and Followup Work . . . . . . . . . . . . . . . . . . . 50 4 Fourier Image Transformer 53 4.1 Transformers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 4.1.1 Attention Is All You Need . . . . . . . . . . . . . . . . . . 55 4.1.2 Fast-Transformers . . . . . . . . . . . . . . . . . . . . . . . 56 4.1.3 Transformers in Computer Vision . . . . . . . . . . . . . . 57 4.2 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 4.2.1 Fourier Domain Encodings (FDEs) . . . . . . . . . . . . . 57 4.2.2 Fourier Coefficient Loss . . . . . . . . . . . . . . . . . . . . 59 4.3 FIT for Super-Resolution . . . . . . . . . . . . . . . . . . . . . . . 60 4.3.1 Super-Resolution Data . . . . . . . . . . . . . . . . . . . . 60 4.3.2 Super-Resolution Experiments . . . . . . . . . . . . . . . . 61 4.4 FIT for Tomography . . . . . . . . . . . . . . . . . . . . . . . . . 63 4.4.1 Computed Tomography Data . . . . . . . . . . . . . . . . 64 4.4.2 Computed Tomography Experiments . . . . . . . . . . . . 66 4.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 5 Conclusions and Outlook 7

DenoiSeg: Joint Denoising and Segmentation

Microscopy image analysis often requires the segmentation of objects, but training data for this task is typically scarce and hard to obtain. Here we propose DenoiSeg, a new method that can be trained end-to-end on only a few annotated ground truth segmentations. We achieve this by extending Noise2Void, a self-supervised denoising scheme that can be trained on noisy images alone, to also predict dense 3-class segmentations. The reason for the success of our method is that segmentation can profit from denoising, especially when performed jointly within the same network. The network becomes a denoising expert by seeing all available raw data, while co-learning to segment, even if only a few segmentation labels are available. This hypothesis is additionally fueled by our observation that the best segmentation results on high quality (very low noise) raw data are obtained when moderate amounts of synthetic noise are added. This renders the denoising-task non-trivial and unleashes the desired co-learning effect. We believe that DenoiSeg offers a viable way to circumvent the tremendous hunger for high quality training data and effectively enables few-shot learning of dense segmentations.Comment: 10 pages, 4 figures, 2 pages supplement (4 figures

Leveraging Self-supervised Denoising for Image Segmentation

Deep learning (DL) has arguably emerged as the method of choice for the detection and segmentation of biological structures in microscopy images. However, DL typically needs copious amounts of annotated training data that is for biomedical projects typically not available and excessively expensive to generate. Additionally, tasks become harder in the presence of noise, requiring even more high-quality training data. Hence, we propose to use denoising networks to improve the performance of other DL-based image segmentation methods. More specifically, we present ideas on how state-of-the-art self-supervised CARE networks can improve cell/nuclei segmentation in microscopy data. Using two state-of-the-art baseline methods, U-Net and StarDist, we show that our ideas consistently improve the quality of resulting segmentations, especially when only limited training data for noisy micrographs are available.Comment: accepted at ISBI 202

Is a Block of the Femoral and Sciatic Nerves an Alternative to Epidural Analgesia in Sheep Undergoing Orthopaedic Hind Limb Surgery? A Prospective, Randomized, Double Blinded Experimental Trial

Peripheral nerve blocks are commonly used in human and veterinary medicine. The aim of the study was to compare the analgesic efficacy of a combined block of the femoral and sciatic nerves with an epidural injection of ropivacaine in experimental sheep undergoing orthopaedic hind limb surgery. Twenty-five sheep were assigned to two groups (peripheral nerve block; sciatic and femoral nerves (P); epidural analgesia (E)). In group P 10 mL ropivacaine 0.5% was injected around the sciatic and the femoral nerves under sonographic guidance and 10 mL NaCl 0.9% into the epidural space while in group E 10 mL ropivacaine 0.5% was injected into the epidural space and 10 mL NaCl 0.9% to the sciatic and the femoral nerves. During surgery, heart rate, respiratory rate and mean blood pressure were used as indicators of nociception. In the postoperative phase, nociception was evaluated every hour by use of a purposefully adapted pain score until the animal showed painful sensation at the surgical site. The mean duration of analgesia at the surgical wound was 6 h in group P and 8 h in group E. Mean time to standing was 4 h in group P and 7 h in group E. In conclusion time to standing was significantly shorter in group P while the duration of nociception was comparable in both groups. The peripheral nerve block can be used as an alternative to epidural analgesia in experimental sheep

Protocol for a systematic review of good surgical practice guidelines for experimental rodent surgery

OBJECTIVE: Surgery is an integral part of many experimental studies. Aseptic and minimal invasive surgical technique and optimal perioperative and post-operative care are prerequisites to achieve surgical success and best possible animal welfare outcomes. Good surgical practice cannot only improve the animal's postoperative recovery, but also study outcome and validity. There seems to be a lack of implementation of good surgical practice during rodent surgery. The aim of this systematic review is to identify, critically evaluate and compare the currently recommended standards and underlying guidelines for rodent surgery-and finally to compile a comprehensive guideline of good surgical practice for rodent surgery. SEARCH STRATEGY: PubMed, Embase and Web of Science were searched to identify guidelines published in peer-reviewed journals. To identify grey literature and unpublished guidelines, we will perform a Google search for published guidelines and search laboratory animal sciences books for relevant book chapters. Additionally, we will conduct a survey among animal researchers enquiring about the guidelines they use. SCREENING AND STUDY SELECTION: For publications retrieved by the systematic search, unique references are screened by two reviewers, first for eligibility based on title and abstract and subsequently for final inclusion based on full text. Eligibility of books is based on title and content, final inclusion based on chapter full text. Guidelines are either retrieved by Google searches or a survey. Google searches will be conducted by at least four of the authors. Thereafter, guidelines will be screened by two of the authors. DATA EXTRACTION AND SYNTHESIS: We will extract data from publications, book chapters and guidelines. Based on the extracted data, we will perform a descriptive synthesis of the bibliographical details, guideline development and endorsement, and the prevalence of individual recommendations, including subgroup analysis of the guidance per continent or country and differences between peer-reviewed versus non-peer-reviewed guidance

Physiology, behavior, and conservation

Many animal populations are in decline as a result of human activity. Conservation practitioners are attempting to prevent further declines and loss of biodiversity as well as to facilitate recovery of endangered species, and they often rely on interdisciplinary approaches to generate conservation solutions. Two recent interfaces in conservation science involve animal behavior (i.e., conservation behavior) and physiology (i.e., conservation physiology). To date, these interfaces have been considered separate entities, but from both pragmatic and biological perspectives, there is merit in better integrating behavior and physiology to address applied conservation problems and to inform resource management. Although there are some institutional, conceptual, methodological, and communication-oriented challenges to integrating behavior and physiology to inform conservation actions, most of these barriers can be overcome. Through outlining several successful examples that integrate these disciplines, we conclude that physiology and behavior can together generate meaningful data to support animal conservation and management actions. Tangentially, applied conservation and management problems can, in turn, also help advance and reinvigorate the fundamental disciplines of animal physiology and behavior by providing advanced natural experiments that challenge traditional frameworks

Increased chromosomal radiosensitivity in asymptomatic carriers of a heterozygous BRCA1 mutation

Background: Breast cancer risk increases drastically in individuals carrying a germline BRCA1 mutation. The exposure to ionizing radiation for diagnostic or therapeutic purposes of BRCA1 mutation carriers is counterintuitive, since BRCA1 is active in the DNA damage response pathway. The aim of this study was to investigate whether healthy BRCA1 mutations carriers demonstrate an increased radiosensitivity compared with healthy individuals. Methods: We defined a novel radiosensitivity indicator (RIND) based on two endpoints measured by the G2 micronucleus assay, reflecting defects in DNA repair and G2 arrest capacity after exposure to doses of 2 or 4 Gy. We investigated if a correlation between the RIND score and nonsense-mediated decay (NMD) could be established. Results: We found significantly increased radiosensitivity in the cohort of healthy BRCA1 mutation carriers compared with healthy controls. In addition, our analysis showed a significantly different distribution over the RIND scores (p = 0.034, Fisher’s exact test) for healthy BRCA1 mutation carriers compared with non-carriers: 72 % of mutation carriers showed a radiosensitive phenotype (RIND score 1–4), whereas 72 % of the healthy volunteers showed no radiosensitivity (RIND score 0). Furthermore, 28 % of BRCA1 mutation carriers had a RIND score of 3 or 4 (not observed in control subjects). The radiosensitive phenotype was similar for relatives within several families, but not for unrelated individuals carrying the same mutation. The median RIND score was higher in patients with a mutation leading to a premature termination codon (PTC) located in the central part of the gene than in patients with a germline mutation in the 5′ end of the gene. Conclusions: We show that BRCA1 mutations are associated with a radiosensitive phenotype related to a compromised DNA repair and G2 arrest capacity after exposure to either 2 or 4 Gy. Our study confirms that haploinsufficiency is the mechanism involved in radiosensitivity in patients with a PTC allele, but it suggests that further research is needed to evaluate alternative mechanisms for mutations not subjected to NMD