Nonlinear Hebbian learning as a unifying principle in receptive field formation

The development of sensory receptive fields has been modeled in the past by a variety of models including normative models such as sparse coding or independent component analysis and bottom-up models such as spike-timing dependent plasticity or the Bienenstock-Cooper-Munro model of synaptic plasticity. Here we show that the above variety of approaches can all be unified into a single common principle, namely Nonlinear Hebbian Learning. When Nonlinear Hebbian Learning is applied to natural images, receptive field shapes were strongly constrained by the input statistics and preprocessing, but exhibited only modest variation across different choices of nonlinearities in neuron models or synaptic plasticity rules. Neither overcompleteness nor sparse network activity are necessary for the development of localized receptive fields. The analysis of alternative sensory modalities such as auditory models or V2 development lead to the same conclusions. In all examples, receptive fields can be predicted a priori by reformulating an abstract model as nonlinear Hebbian learning. Thus nonlinear Hebbian learning and natural statistics can account for many aspects of receptive field formation across models and sensory modalities

Bayesian separation of spectral sources under non-negativity and full additivity constraints

This paper addresses the problem of separating spectral sources which are linearly mixed with unknown proportions. The main difficulty of the problem is to ensure the full additivity (sum-to-one) of the mixing coefficients and non-negativity of sources and mixing coefficients. A Bayesian estimation approach based on Gamma priors was recently proposed to handle the non-negativity constraints in a linear mixture model. However, incorporating the full additivity constraint requires further developments. This paper studies a new hierarchical Bayesian model appropriate to the non-negativity and sum-to-one constraints associated to the regressors and regression coefficients of linear mixtures. The estimation of the unknown parameters of this model is performed using samples generated using an appropriate Gibbs sampler. The performance of the proposed algorithm is evaluated through simulation results conducted on synthetic mixture models. The proposed approach is also applied to the processing of multicomponent chemical mixtures resulting from Raman spectroscopy.Comment: v4: minor grammatical changes; Signal Processing, 200

Spectrum optimization in multi-user multi-carrier systems with iterative convex and nonconvex approximation methods

Several practical multi-user multi-carrier communication systems are characterized by a multi-carrier interference channel system model where the interference is treated as noise. For these systems, spectrum optimization is a promising means to mitigate interference. This however corresponds to a challenging nonconvex optimization problem. Existing iterative convex approximation (ICA) methods consist in solving a series of improving convex approximations and are typically implemented in a per-user iterative approach. However they do not take this typical iterative implementation into account in their design. This paper proposes a novel class of iterative approximation methods that focuses explicitly on the per-user iterative implementation, which allows to relax the problem significantly, dropping joint convexity and even convexity requirements for the approximations. A systematic design framework is proposed to construct instances of this novel class, where several new iterative approximation methods are developed with improved per-user convex and nonconvex approximations that are both tighter and simpler to solve (in closed-form). As a result, these novel methods display a much faster convergence speed and require a significantly lower computational cost. Furthermore, a majority of the proposed methods can tackle the issue of getting stuck in bad locally optimal solutions, and hence improve solution quality compared to existing ICA methods.Comment: 33 pages, 7 figures. This work has been submitted for possible publicatio