Adaptive Image Denoising by Targeted Databases

We propose a data-dependent denoising procedure to restore noisy images. Different from existing denoising algorithms which search for patches from either the noisy image or a generic database, the new algorithm finds patches from a database that contains only relevant patches. We formulate the denoising problem as an optimal filter design problem and make two contributions. First, we determine the basis function of the denoising filter by solving a group sparsity minimization problem. The optimization formulation generalizes existing denoising algorithms and offers systematic analysis of the performance. Improvement methods are proposed to enhance the patch search process. Second, we determine the spectral coefficients of the denoising filter by considering a localized Bayesian prior. The localized prior leverages the similarity of the targeted database, alleviates the intensive Bayesian computation, and links the new method to the classical linear minimum mean squared error estimation. We demonstrate applications of the proposed method in a variety of scenarios, including text images, multiview images and face images. Experimental results show the superiority of the new algorithm over existing methods.Comment: 15 pages, 13 figures, 2 tables, journa

Towards Understanding the Effect of Pretraining Label Granularity

In this paper, we study how pretraining label granularity affects the generalization of deep neural networks in image classification tasks. We focus on the "fine-to-coarse" transfer learning setting where the pretraining label is more fine-grained than that of the target problem. We experiment with this method using the label hierarchy of iNaturalist 2021, and observe a 8.76% relative improvement of the error rate over the baseline. We find the following conditions are key for the improvement: 1) the pretraining dataset has a strong and meaningful label hierarchy, 2) its label function strongly aligns with that of the target task, and most importantly, 3) an appropriate level of pretraining label granularity is chosen. The importance of pretraining label granularity is further corroborated by our transfer learning experiments on ImageNet. Most notably, we show that pretraining at the leaf labels of ImageNet21k produces better transfer results on ImageNet1k than pretraining at other coarser granularity levels, which supports the common practice. Theoretically, through an analysis on a two-layer convolutional ReLU network, we prove that: 1) models trained on coarse-grained labels only respond strongly to the common or "easy-to-learn" features; 2) with the dataset satisfying the right conditions, fine-grained pretraining encourages the model to also learn rarer or "harder-to-learn" features well, thus improving the model's generalization

Adaptive non-local means for multiview image denoising: Searching for the right patches via a statistical approach

ABSTRACT We present an adaptive non-local means (NLM) denoising method for a sequence of images captured by a multiview imaging system, where direct extensions of existing single image NLM methods are incapable of producing good results. Our proposed method consists of three major components: (1) a robust joint-view distance metric to measure the similarity of patches; (2) an adaptive procedure derived from statistical properties of the estimates to determine the optimal number of patches to be used; (3) a new NLM algorithm to denoise using only a set of similar patches. Experimental results show that the proposed method is robust to disparity estimation error, out-performs existing algorithms in multiview settings, and performs competitively in video settings. Index Terms-Non-local means, adaptive filtering, multiview denoising, patch-based denoisin

Genomic Analyses Reveal Mutational Signatures and Frequently Altered Genes in Esophageal Squamous Cell Carcinoma

Esophageal squamous cell carcinoma (ESCC) is one of the most common cancers worldwide and the fourth most lethal cancer in China. However, although genomic studies have identified some mutations associated with ESCC, we know little of the mutational processes responsible. To identify genome-wide mutational signatures, we performed either whole-genome sequencing (WGS) or whole-exome sequencing (WES) on 104 ESCC individuals and combined our data with those of 88 previously reported samples. An APOBEC-mediated mutational signature in 47% of 192 tumors suggests that APOBEC-catalyzed deamination provides a source of DNA damage in ESCC. Moreover, PIK3CA hotspot mutations (c.1624G>A [p.Glu542Lys] and c.1633G>A [p.Glu545Lys]) were enriched in APOBEC-signature tumors, and no smoking-associated signature was observed in ESCC. In the samples analyzed by WGS, we identified focal (<100 kb) amplifications of CBX4 and CBX8. In our combined cohort, we identified frequent inactivating mutations in AJUBA, ZNF750, and PTCH1 and the chromatin-remodeling genes CREBBP and BAP1, in addition to known mutations. Functional analyses suggest roles for several genes (CBX4, CBX8, AJUBA, and ZNF750) in ESCC. Notably, high activity of hedgehog signaling and the PI3K pathway in approximately 60% of 104 ESCC tumors indicates that therapies targeting these pathways might be particularly promising strategies for ESCC. Collectively, our data provide comprehensive insights into the mutational signatures of ESCC and identify markers for early diagnosis and potential therapeutic targets

Statistical and Adaptive Patch-based Image Denoising

With the explosion in the number of digital images taken every day, people are demanding more accurate and visually pleasing images. However, the captured images by modern cameras are inevitably degraded by noise. Besides deteriorating image visual quality, noise also degrades the performance of high-level vision tasks such as object recognition and tracking. Therefore, image denoising is a critical preprocessing step. This thesis presents novel contributions to the field of image denoising.Image denoising is a highly ill-posed inverse problem. To alleviate the ill-posedness, an effective prior plays an important role and is a key factor for successful image denoising. With abundance of images available online, we propose to obtain priors from external image databases. In this thesis, we perform statistical analyses and rigorous derivations on how to obtain effective priors by utilizing external databases. For three denoising applications under different external settings, we show how we can explore effective priors and accordingly we present adaptive patch-based image denoising algorithms. In specific, we propose three adaptive algorithms: (1) adaptive non-local means for multiview image denoising; (2) adaptive image denoising by targeted databases; (3) adaptive image denoising by mixture adaption.In (1), we present how to improve the non-local prior by finding more relevant patches in the multiview image denoising setting. We propose a method that uses a robust joint-view distance metric to measure the similarity of patches and derive an adaptive procedure to determine the optimal number of patches for final non-local means denoising. In (2), we propose to switch from generic database to targeted database, i.e., for specific objects to be denoised, only targeted databases with relevant images should be used. We explore both the group sparsity prior and the localized Bayesian prior, and show how a near optimal and adaptive denoising filter can be designed so that the targeted database can be maximally utilized. In (3), we propose an adaptive learning procedure called Expectation-Maximization (EM) adaptation. The adaptive process takes a generic prior learned from a generic database and transfers it to the image of interest to create a specific prior. This adapted prior better captures the distribution of the image of interest and is consistently better than the un-adapted one. For all the three denoising applications, we conduct various denoising experiments. Our proposed adaptive algorithms have some superior denoising performance than some state-of-the-art algorithms

Computational complexity reduction in the spatial scalable video coding encoder

Scalable Video Coding (SVC) has been approved as the extension of the H.264/AVC video coding standard. This extension allows multiple frame rates, multiple resolutions, or multiple SNR rates of an image sequence to be contained in a single bit stream. The support of multiple resolutions within a single compressed bit stream is referred to as spatial SVC. It has achieved significant improvements in the enhancement layer coding efficiency due to the introduction of new modes and additional inter-layer prediction tools. However, the computational complexity of spatial SVC encoder is rather high, which inhibits it from practical use. Thus how to reduce the computational complexity but at the same time preserving the coding efficiency is quite crucial. In this thesis, three different methods to reduce the coding time are investigated and proposed: 1) reduce the use of inter-layer residual prediction tool; 2) reduce part of the rate-distortion cost calculations for each inter mode; 3) use a fast mode decision algorithm to accelerate the mode selection process. The three methods can be embedded into SVC's reference software-JSVM and used individually or be combined together. The simulation results demonstrate that the coding efficiency can be preserved well while consistent time saving over a large bit rate range can be achieved, especially when three methods are combined, over 50% time saving can be obtained

Fast External Denoising Using Pre-Learned Transformations

We introduce a new external denoising algorithm that utilizes pre-learned transformations to accelerate filter calculations during runtime. The proposed fast external denoising (FED) algorithm shares characteristics of the powerful Targeted Image Denoising (TID) and Expected Patch Log-Likelihood (EPLL) algorithms. By moving computationally demanding steps to an offline learning stage, the proposed approach aims to find a balance between processing speed and obtaining high quality denoising estimates. We evaluate FED on three datasets with targeted databases (text, face and license plates) and also on a set of generic images without a targeted database. We show that, like TID, the proposed approach is extremely effective when the transformations are learned using a targeted database. We also demonstrate that FED converges to competitive solutions faster than EPLL and is orders of magnitude faster than TID while providing comparable denoising performance