Efficient Sparse Coding in Early Sensory Processing: Lessons from Signal Recovery

Sensory representations are not only sparse, but often overcomplete: coding units significantly outnumber the input units. For models of neural coding this overcompleteness poses a computational challenge for shaping the signal processing channels as well as for using the large and sparse representations in an efficient way. We argue that higher level overcompleteness becomes computationally tractable by imposing sparsity on synaptic activity and we also show that such structural sparsity can be facilitated by statistics based decomposition of the stimuli into typical and atypical parts prior to sparse coding. Typical parts represent large-scale correlations, thus they can be significantly compressed. Atypical parts, on the other hand, represent local features and are the subjects of actual sparse coding. When applied on natural images, our decomposition based sparse coding model can efficiently form overcomplete codes and both center-surround and oriented filters are obtained similar to those observed in the retina and the primary visual cortex, respectively. Therefore we hypothesize that the proposed computational architecture can be seen as a coherent functional model of the first stages of sensory coding in early vision

A Chemocentric Approach to the Identification of Cancer Targets

A novel chemocentric approach to identifying cancer-relevant targets is introduced. Starting with a large chemical collection, the strategy uses the list of small molecule hits arising from a differential cytotoxicity screening on tumor HCT116 and normal MRC-5 cell lines to identify proteins associated with cancer emerging from a differential virtual target profiling of the most selective compounds detected in both cell lines. It is shown that this smart combination of differential in vitro and in silico screenings (DIVISS) is capable of detecting a list of proteins that are already well accepted cancer drug targets, while complementing it with additional proteins that, targeted selectively or in combination with others, could lead to synergistic benefits for cancer therapeutics. The complete list of 115 proteins identified as being hit uniquely by compounds showing selective antiproliferative effects for tumor cell lines is provided

A comparison of the amplitude spectra of the “atypical” output part of RPCA, the whitened input and the whitened ideal input.

<p>This plot demonstrates that the particular whitening filter as used in <a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1002372#pcbi.1002372-Olshausen1" target="_blank">[5]</a>, <a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1002372#pcbi.1002372-Rehn1" target="_blank">[40]</a> can be seen as a linear approximation of the filtering properties of RPCA when only the atypical output is considered. The thick (red) line is the amplitude spectrum of the RPCA output. The dashed (blue) line with square markers is the amplitude spectrum of the training images filtered with the whitening filter. The thin (green) line serves as a reference: this is the amplitude spectrum of whitened ideal input which has an amplitude spectrum proportional to 1/frequency. Due to the limited input size, there is a natural cutoff at higher frequencies. (Since the size of the images is 16×16, the largest frequency is .) The whitening filter: , where the cutoff frequency is . The variances of the plots are due the artifacts caused by the rectangular sampling lattice. For comparison purposes the plots are rescaled onto .</p

Reconstruction quality as a function of the number of nonzero coding units and .

<p>Reconstruction quality is measured by mean SNR: , where runs over the inputs. Since RPCA is an additive decomposition, the reconstruction error is given as . The total number of nonzero entries is given as the sum of the rank estimate of and the preserved number of nonzero units (k) in the sparse overcomplete representation of the atypical part () of the RPCA output. Since sparseness level is automatically set by SCE, the following arbitrary values for k were chosen. For and for , .</p

The pseudocode of the Subspace Pursuit method.

<p>The goal is to represent the input with minimal reconstruction error using basis only <a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1002372#pcbi.1002372-Dai1" target="_blank">[23]</a>. SP differs from other iterative greedy methods in the incremental refinement of the selected basis subset. First, a representation is generated with the help of the full basis set (using pseudoinverse computations). During iteration basis are selected based on the amplitude of the corresponding coordinates of the representation. The resulting residual (difference between the original input and the approximation obtained by projecting the representation onto the input space) is then again projected back to the representation space and another set of basis are chosen. The two selected subsets are then fused (<i>expansion</i>) and the resulting expanded set is used again to project the original input onto the representation space. Finally a new set of basis are selected by the amplitude of the corresponding coordinates of the projection (<i>shrinkage</i>). Iteration stops when the norm of the residual does not decrease anymore. Notation: denotes a sub-matrix of where index set contains the indices of the selected columns. The index set of the first sorted components of a vector is denoted by .</p

Distribution of the shape parameters for the model and for the experimental data.

<p>Receptive fields of simple cells in primary visual cortex, linearly approximated by spike triggered averaging. Data <a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1002372#pcbi.1002372-Ringach1" target="_blank">[8]</a> are available at <a href="http://web.mac.com/darioringach/lab/Data.html" target="_blank">http://web.mac.com/darioringach/lab/Data.html</a>. Our model filters show significant diversity in the fitted shapes similar to what has been found experimentally. While other models (e.g. <a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1002372#pcbi.1002372-Lcke1" target="_blank">[39]</a>, <a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1002372#pcbi.1002372-Rehn1" target="_blank">[40]</a>) are also able to partially match the filters to the observed RFs, a significant difference is that our model uses highly overcomplete representations. For other differences, see the main text.</p

RPCA on concatenated image sequences.

<p>Left: The first 10 spatio-temporal filters of the low rank signal,L (rank ) are shown. Each filter is shown as a sequence of 16 frames of size 8×8 pixels. It can be seen that there are spatio-temporally separable as well as non-separable filters. All filters correspond to low frequency temporal or spatial changes Right: 10 selected spatio-temporal filters of the corresponding overcomplete sparse codes that display different spatio-temporal localization and dynamics. While many filters are similar to the presented ones, more training would be needed to achieve similar locality for the majority of filters at this input dimensionality (8×8×16) and level of overcompleteness (16×).</p