Representation Learning: A Review and New Perspectives

The success of machine learning algorithms generally depends on data representation, and we hypothesize that this is because different representations can entangle and hide more or less the different explanatory factors of variation behind the data. Although specific domain knowledge can be used to help design representations, learning with generic priors can also be used, and the quest for AI is motivating the design of more powerful representation-learning algorithms implementing such priors. This paper reviews recent work in the area of unsupervised feature learning and deep learning, covering advances in probabilistic models, auto-encoders, manifold learning, and deep networks. This motivates longer-term unanswered questions about the appropriate objectives for learning good representations, for computing representations (i.e., inference), and the geometrical connections between representation learning, density estimation and manifold learning

Bitwise Source Separation on Hashed Spectra: An Efficient Posterior Estimation Scheme Using Partial Rank Order Metrics

This paper proposes an efficient bitwise solution to the single-channel source separation task. Most dictionary-based source separation algorithms rely on iterative update rules during the run time, which becomes computationally costly especially when we employ an overcomplete dictionary and sparse encoding that tend to give better separation results. To avoid such cost we propose a bitwise scheme on hashed spectra that leads to an efficient posterior probability calculation. For each source, the algorithm uses a partial rank order metric to extract robust features that form a binarized dictionary of hashed spectra. Then, for a mixture spectrum, its hash code is compared with each source's hashed dictionary in one pass. This simple voting-based dictionary search allows a fast and iteration-free estimation of ratio masking at each bin of a signal spectrogram. We verify that the proposed BitWise Source Separation (BWSS) algorithm produces sensible source separation results for the single-channel speech denoising task, with 6-8 dB mean SDR. To our knowledge, this is the first dictionary based algorithm for this task that is completely iteration-free in both training and testing

Learning generative texture models with extended Fields-of-Experts

We evaluate the ability of the popular Field-of-Experts (FoE) to model structure in images. As a test case we focus on modeling synthetic and natural textures. We find that even for modeling single textures, the FoE provides insufficient flexibility to learn good generative models – it does not perform any better than the much simpler Gaussian FoE. We propose an extended version of the FoE (allowing for bimodal potentials) and demonstrate that this novel formulation, when trained with a better approximation of the likelihood gradient, gives rise to a more powerful generative model of specific visual structure that produces significantly better results for the texture task

Tomographic inversion using ℓ1\ell_1-norm regularization of wavelet coefficients

We propose the use of $\ell_1$ regularization in a wavelet basis for the solution of linearized seismic tomography problems $Am=d$, allowing for the possibility of sharp discontinuities superimposed on a smoothly varying background. An iterative method is used to find a sparse solution $m$ that contains no more fine-scale structure than is necessary to fit the data $d$ to within its assigned errors.Comment: 19 pages, 14 figures. Submitted to GJI July 2006. This preprint does not use GJI style files (which gives wrong received/accepted dates). Corrected typ

An Efficient Method for Traffic Image Denoising

AbstractIn this paper, a novel method for traffic image denoising based on the low-rank decomposition is proposed. Firstly, the low-rank decomposition is carried out. Under the sparse and low-rank constraints of low-rank decomposition, the foreground images with complanate background and moving vehicles and the background images with similar road scene are obtained. Then the foreground image is segmented into blocks of a certain size. The variance of each block is calculated, among that the minimum is considered the estimate of the noise power. KSVD algorithm is performed for the foreground image denoising. Furthermore, the noisy pixel discrimination algorithm is performed to distinguish the noisy pixels from the noiseless pixels and the eight- neighborhood weight interpolation algorithm is performed to reconstruct the noisy pixels, where the weighted coefficients are inversely proportional to the Euclidean distances between the pixels. And PCA recovery combined with noisy pixel discrimination and eight-neighborhood weight interpolation is adopted for the background image denoising. Finally, our proposed method is conducted based on the traffic videos obtained under the same view and angle. Moreover, our proposed method is compared with several state-of-the-art denoising methods including BM3D, KSVD and PCA recovery. The experiment results illustrate that our proposed method can more effectively remove the noise, preserve the useful information and achieve a better performance in terms of both PSNR index and visual qualities