Expectation Propagation for Rectified Linear Poisson Regression

The Poisson likelihood with rectified linear function as non-linearity is a physically plausible model to discribe the stochastic arrival process of photons or other particles at a detector. At low emission rates the discrete nature of this process leads to measurement noise that behaves very differently from additive white Gaussian noise. To address the intractable inference problem for such models, we present a novel efficient and robust Expectation Propagation algorithm entirely based on analytically tractable computations operating re- liably in regimes where quadrature based implementations can fail. Full posterior inference therefore becomes an attractive alternative in areas generally dominated by methods of point estimation. Moreover, we discuss the rectified linear function in the context of other common non-linearities and identify situations where it can serve as a robust alternative

Applications of Approximate Learning and Inference for Probabilistic Models

We develop approximate inference and learning methods for facilitating the use of probabilistic modeling techniques motivated by applications in two different areas. First, we consider the ill-posed inverse problem of recovering an image from an underdetermined system of linear measurements corrupted by noise. Second, we consider the problem of inferring user preferences for items from counts, pairwise comparisons and user activity logs, instances of implicit feedback. Plausible models for images and the noise, incurred when recording them, render posterior inference intractable, while the scale of the inference problem makes sampling based approximations ineffective. Therefore, we develop deterministic approximate inference algorithms for two different augmentations of a typical sparse linear model: first, for the rectified-linear Poisson likelihood, and second, for tree-structured super-Gaussian mixture models. The rectified-linear Poisson likelihood is an alternative noise model, applicable in astronomical and biomedical imaging applications, that operate in intensity regimes in which quantum effects lead to observations that are best described by counts of particles arriving at a sensor, as well as in general Poisson regression problems arising in various fields. In this context we show, that the model-specific computations for Expectation Propagation can be robustly solved by a simple dynamic program. Next, we develop a scalable approximate inference algorithm for structured mixture models, that uses a discrete graphical model to represent dependencies between the latent mixture components of a collection of mixture models. Specifically, we use tree-structured mixtures of super-Gaussians to model the persistence across scales of large coefficients of the Wavelet transform of an image for improved reconstruction. In the second part on models of user preference, we consider two settings: the global static and the contextual dynamic setting. In the global static setting, we represent user-item preferences by a latent low-rank matrix. Instead of using numeric ratings we develop methods to infer this latent representation for two types of implicit feedback: aggregate counts of users interacting with a service and the binary outcomes of pairwise comparisons. We model count data using a latent Gaussian bilinear model with Poisson likelihoods. For this model, we show that the Variational Gaussian approximation can be further relaxed to be available in closed-form by adding additional constraints, leading to an efficient inference algorithm. In the second implicit feedback scenario, we infer the latent preference matrix from pairwise preference statements. We combine a low-rank bilinear model with non-parameteric item- feature regression and develop a novel approximate variational Expectation Maximization algorithm that mitigates the computational challenges due to latent couplings induced by the pairwise comparisons. Finally, in the contextual dynamic setting, we model sequences of user activity at the granularity of single interaction events instead of aggregate counts. Routinely gathered in the background at a large scale in many applications, such sequences can reveal temporal and contextual aspects of user behavior through recurrent patterns. To describe such data, we propose a generic collaborative sequence model based on recurrent neural networks, that combines ideas from collaborative filtering and language modeling

Interrogating the Role of Cocaine-Generated Silent Synapses in the Regulation of Cocaine-Associated Memory Dynamics

Drug addiction is an acquired behavioral state that develops progressively through repeated drug experience and is characterized by maladaptive and compulsive behavior associated with drug seeking and taking. Cravings and subsequent drug seeking are often precipitated by the reactivation of memories associated with drug use, which are formed between various external stimuli, or cues, and the rewarding and pleasurable experience of taking the drug. As such, drug addiction is often conceptualized as a pathological form of memory that drives maladaptive behavior. This has spurred intensive investigation into the neural substrates underlying drug-associated memories, with the ultimate goal of targeting these substrates to disrupt drug seeking behaviors. To explore the synaptic underpinnings of cocaine-associated memories, we studied AMPA receptor (AMPAR)-silent excitatory synapses, which are generated in the nucleus accumbens (NAc) by cocaine experience. These synapses functionally mature during withdrawal through the recruitment of AMPARs and contribute to subsequent cocaine seeking behavior, indicating these synapses contribute to the encoding of cocaine-associated memories and behaviors. In this dissertation, we have further investigated the role of cocaine-generated silent synapses in the encoding of cocaine-associated memories by examining their role in regulating the natural dynamics of cocaine-associated memories. Our results demonstrate that dynamic changes in the functional state of cocaine-generated synapses contributes to the natural destabilization and reconsolidation of cocaine-associated memories following memory retrieval, and that disrupting these synaptic dynamics impairs subsequent cocaine seeking behaviors. In addition, we also demonstrate that cocaine-generated synapses contribute to the recruitment and activation of neurons within the NAc associated with cocaine seeking behavior during withdrawal, suggesting they may contribute to the encoding of cocaine-associated memories at the circuit level. Collectively, these findings provide further support to the hypothesis that cocaine-generated synapses serve as discrete synaptic substrates underlying aspects of cocaine-associated memories and behaviors