Dual Averaging Method for Online Graph-structured Sparsity

Online learning algorithms update models via one sample per iteration, thus efficient to process large-scale datasets and useful to detect malicious events for social benefits, such as disease outbreak and traffic congestion on the fly. However, existing algorithms for graph-structured models focused on the offline setting and the least square loss, incapable for online setting, while methods designed for online setting cannot be directly applied to the problem of complex (usually non-convex) graph-structured sparsity model. To address these limitations, in this paper we propose a new algorithm for graph-structured sparsity constraint problems under online setting, which we call \textsc{GraphDA}. The key part in \textsc{GraphDA} is to project both averaging gradient (in dual space) and primal variables (in primal space) onto lower dimensional subspaces, thus capturing the graph-structured sparsity effectively. Furthermore, the objective functions assumed here are generally convex so as to handle different losses for online learning settings. To the best of our knowledge, \textsc{GraphDA} is the first online learning algorithm for graph-structure constrained optimization problems. To validate our method, we conduct extensive experiments on both benchmark graph and real-world graph datasets. Our experiment results show that, compared to other baseline methods, \textsc{GraphDA} not only improves classification performance, but also successfully captures graph-structured features more effectively, hence stronger interpretability.Comment: 11 pages, 14 figure

Fewer Flops at the Top: Accuracy, Diversity, and Regularization in Two-Class Collaborative Filtering

In most existing recommender systems, implicit or explicit interactions are treated as positive links and all unknown interactions are treated as negative links. The goal is to suggest new links that will be perceived as positive by users. However, as signed social networks and newer content services become common, it is important to distinguish between positive and negative preferences. Even in existing applications, the cost of a negative recommendation could be high when people are looking for new jobs, friends, or places to live. In this work, we develop novel probabilistic latent factor models to recommend positive links and compare them with existing methods on five different openly available datasets. Our models are able to produce better ranking lists and are effective in the task of ranking positive links at the top, with fewer negative links (flops). Moreover, we find that modeling signed social networks and user preferences this way has the advantage of increasing the diversity of recommendations. We also investigate the effect of regularization on the quality of recommendations, a matter that has not received enough attention in the literature. We find that regularization parameter heavily affects the quality of recommendations in terms of both accuracy and diversity

Simple Asymmetric Exclusion Model and Lattice Paths: Bijections and Involutions

We study the combinatorics of the change of basis of three representations of the stationary state algebra of the two parameter simple asymmetric exclusion process. Each of the representations considered correspond to a different set of weighted lattice paths which, when summed over, give the stationary state probability distribution. We show that all three sets of paths are combinatorially related via sequences of bijections and sign reversing involutions.Comment: 28 page

Recurrent Latent Variable Networks for Session-Based Recommendation

In this work, we attempt to ameliorate the impact of data sparsity in the context of session-based recommendation. Specifically, we seek to devise a machine learning mechanism capable of extracting subtle and complex underlying temporal dynamics in the observed session data, so as to inform the recommendation algorithm. To this end, we improve upon systems that utilize deep learning techniques with recurrently connected units; we do so by adopting concepts from the field of Bayesian statistics, namely variational inference. Our proposed approach consists in treating the network recurrent units as stochastic latent variables with a prior distribution imposed over them. On this basis, we proceed to infer corresponding posteriors; these can be used for prediction and recommendation generation, in a way that accounts for the uncertainty in the available sparse training data. To allow for our approach to easily scale to large real-world datasets, we perform inference under an approximate amortized variational inference (AVI) setup, whereby the learned posteriors are parameterized via (conventional) neural networks. We perform an extensive experimental evaluation of our approach using challenging benchmark datasets, and illustrate its superiority over existing state-of-the-art techniques

PlaNet - Photo Geolocation with Convolutional Neural Networks

Is it possible to build a system to determine the location where a photo was taken using just its pixels? In general, the problem seems exceptionally difficult: it is trivial to construct situations where no location can be inferred. Yet images often contain informative cues such as landmarks, weather patterns, vegetation, road markings, and architectural details, which in combination may allow one to determine an approximate location and occasionally an exact location. Websites such as GeoGuessr and View from your Window suggest that humans are relatively good at integrating these cues to geolocate images, especially en-masse. In computer vision, the photo geolocation problem is usually approached using image retrieval methods. In contrast, we pose the problem as one of classification by subdividing the surface of the earth into thousands of multi-scale geographic cells, and train a deep network using millions of geotagged images. While previous approaches only recognize landmarks or perform approximate matching using global image descriptors, our model is able to use and integrate multiple visible cues. We show that the resulting model, called PlaNet, outperforms previous approaches and even attains superhuman levels of accuracy in some cases. Moreover, we extend our model to photo albums by combining it with a long short-term memory (LSTM) architecture. By learning to exploit temporal coherence to geolocate uncertain photos, we demonstrate that this model achieves a 50% performance improvement over the single-image model

Sharing Social Network Data: Differentially Private Estimation of Exponential-Family Random Graph Models

Motivated by a real-life problem of sharing social network data that contain sensitive personal information, we propose a novel approach to release and analyze synthetic graphs in order to protect privacy of individual relationships captured by the social network while maintaining the validity of statistical results. A case study using a version of the Enron e-mail corpus dataset demonstrates the application and usefulness of the proposed techniques in solving the challenging problem of maintaining privacy \emph{and} supporting open access to network data to ensure reproducibility of existing studies and discovering new scientific insights that can be obtained by analyzing such data. We use a simple yet effective randomized response mechanism to generate synthetic networks under $\epsilon$-edge differential privacy, and then use likelihood based inference for missing data and Markov chain Monte Carlo techniques to fit exponential-family random graph models to the generated synthetic networks.Comment: Updated, 39 page

Deep Markov Random Field for Image Modeling

Markov Random Fields (MRFs), a formulation widely used in generative image modeling, have long been plagued by the lack of expressive power. This issue is primarily due to the fact that conventional MRFs formulations tend to use simplistic factors to capture local patterns. In this paper, we move beyond such limitations, and propose a novel MRF model that uses fully-connected neurons to express the complex interactions among pixels. Through theoretical analysis, we reveal an inherent connection between this model and recurrent neural networks, and thereon derive an approximated feed-forward network that couples multiple RNNs along opposite directions. This formulation combines the expressive power of deep neural networks and the cyclic dependency structure of MRF in a unified model, bringing the modeling capability to a new level. The feed-forward approximation also allows it to be efficiently learned from data. Experimental results on a variety of low-level vision tasks show notable improvement over state-of-the-arts.Comment: Accepted at ECCV 201

Differentially Private Model Selection with Penalized and Constrained Likelihood

In statistical disclosure control, the goal of data analysis is twofold: The released information must provide accurate and useful statistics about the underlying population of interest, while minimizing the potential for an individual record to be identified. In recent years, the notion of differential privacy has received much attention in theoretical computer science, machine learning, and statistics. It provides a rigorous and strong notion of protection for individuals' sensitive information. A fundamental question is how to incorporate differential privacy into traditional statistical inference procedures. In this paper we study model selection in multivariate linear regression under the constraint of differential privacy. We show that model selection procedures based on penalized least squares or likelihood can be made differentially private by a combination of regularization and randomization, and propose two algorithms to do so. We show that our private procedures are consistent under essentially the same conditions as the corresponding non-private procedures. We also find that under differential privacy, the procedure becomes more sensitive to the tuning parameters. We illustrate and evaluate our method using simulation studies and two real data examples

Discretization of variational regularization in Banach spaces

Consider a nonlinear ill-posed operator equation $F(u)=y$ where $F$ is defined on a Banach space $X$. In general, for solving this equation numerically, a finite dimensional approximation of $X$ and an approximation of $F$ are required. Moreover, in general the given data \yd of $y$ are noisy. In this paper we analyze finite dimensional variational regularization, which takes into account operator approximations and noisy data: We show (semi-)convergence of the regularized solution of the finite dimensional problems and establish convergence rates in terms of Bregman distances under appropriate sourcewise representation of a solution of the equation. The more involved case of regularization in nonseparable Banach spaces is discussed in detail. In particular we consider the space of finite total variation functions, the space of functions of finite bounded deformation, and the $L^\infty$--space

Aerosol mass and black carbon concentrations, a two year record at NCO-P (5079 m, Southern Himalayas)

Aerosol mass and the absorbing fraction are important variables, needed to constrain the role of atmospheric particles in the Earth radiation budget, both directly and indirectly through CCN activation. In particular, their monitoring in remote areas and mountain sites is essential for determining source regions, elucidating the mechanisms of long range transport of anthropogenic pollutants, and validating regional and global models. Since March 2006, aerosol mass and black carbon concentration have been monitored at the Nepal Climate Observatory-Pyramid, a permanent high-altitude research station located in the Khumbu valley at 5079 m a.s.l. below Mt. Everest. The first two-year averages of PM<sub>1</sub> and PM<sub>1−10</sub> mass were 1.94 μg m<sup>−3</sup> and 1.88 μg m<sup>−3</sup>, with standard deviations of 3.90 μg m<sup>−3</sup> and 4.45 μg m<sup>−3</sup>, respectively, while the black carbon concentration average is 160.5 ng m<sup>−3</sup>, with a standard deviation of 296.1 ng m<sup>−3</sup>. Both aerosol mass and black carbon show well defined annual cycles, with a maximum during the pre-monsoon season and a minimum during the monsoon. They also display a typical diurnal cycle during all the seasons, with the lowest particle concentration recorded during the night, and a considerable increase during the afternoon, revealing the major role played by thermal winds in influencing the behaviour of atmospheric compounds over the high Himalayas. The aerosol concentration is subject to high variability: in fact, as well as frequent "background conditions" (55% of the time) when BC concentrations are mainly below 100 ng m<sup>−3</sup>, concentrations up to 5 μg m<sup>−3</sup> are reached during some episodes (a few days every year) in the pre-monsoon seasons. The variability of PM and BC is the result of both short-term changes due to thermal wind development in the valley, and long-range transport/synoptic circulation. At NCO-P, higher concentrations of PM<sub>1</sub> and BC are mostly associated with regional circulation and westerly air masses from the Middle East, while the strongest contributions of mineral dust arrive from the Middle East and regional circulation, with a special contribution from North Africa and South-West Arabian Peninsula in post-monsoon and winter season