Role of homeostasis in learning sparse representations

Neurons in the input layer of primary visual cortex in primates develop edge-like receptive fields. One approach to understanding the emergence of this response is to state that neural activity has to efficiently represent sensory data with respect to the statistics of natural scenes. Furthermore, it is believed that such an efficient coding is achieved using a competition across neurons so as to generate a sparse representation, that is, where a relatively small number of neurons are simultaneously active. Indeed, different models of sparse coding, coupled with Hebbian learning and homeostasis, have been proposed that successfully match the observed emergent response. However, the specific role of homeostasis in learning such sparse representations is still largely unknown. By quantitatively assessing the efficiency of the neural representation during learning, we derive a cooperative homeostasis mechanism that optimally tunes the competition between neurons within the sparse coding algorithm. We apply this homeostasis while learning small patches taken from natural images and compare its efficiency with state-of-the-art algorithms. Results show that while different sparse coding algorithms give similar coding results, the homeostasis provides an optimal balance for the representation of natural images within the population of neurons. Competition in sparse coding is optimized when it is fair. By contributing to optimizing statistical competition across neurons, homeostasis is crucial in providing a more efficient solution to the emergence of independent components

An improved decoding scheme for Matching Pursuit Streams

This work presents an improved coefficient decoding method for Matching Pursuit streams. It builds on the adaptive a posteriori quantization of coefficients, and implements an interpolation scheme that enhances the inverse quantization performance at the decoder. A class of interpolation functions is introduced, that capture the behavior of coefficients after conditional scalar quantization. The accuracy of the interpolation scheme is verified experimentally, and the novel decoding algorithm is further evaluated in image coding applications. It can be seen that the proposed method improves the rate-distortion performance by up to 0.5 dB, only by changing the reconstruction strategy at the decoder

Sparsity and `Something Else': An Approach to Encrypted Image Folding

A property of sparse representations in relation to their capacity for information storage is discussed. It is shown that this feature can be used for an application that we term Encrypted Image Folding. The proposed procedure is realizable through any suitable transformation. In particular, in this paper we illustrate the approach by recourse to the Discrete Cosine Transform and a combination of redundant Cosine and Dirac dictionaries. The main advantage of the proposed technique is that both storage and encryption can be achieved simultaneously using simple processing steps.Comment: Revised manuscript- Software for implementing the Encrypted Image Folding proposed in this paper is available on http://www.nonlinear-approx.info