6,857 research outputs found
Efficient Transductive Online Learning via Randomized Rounding
Most traditional online learning algorithms are based on variants of mirror
descent or follow-the-leader. In this paper, we present an online algorithm
based on a completely different approach, tailored for transductive settings,
which combines "random playout" and randomized rounding of loss subgradients.
As an application of our approach, we present the first computationally
efficient online algorithm for collaborative filtering with trace-norm
constrained matrices. As a second application, we solve an open question
linking batch learning and transductive online learningComment: To appear in a Festschrift in honor of V.N. Vapnik. Preliminary
version presented in NIPS 201
Recommended from our members
Combining numeric and symbolic learning techniques
Incremental learning from examples in a noisy domain is a difficult problem in Machine Learning. In this paper we divide the task into two subproblems and present a combination of numeric and symbolic approaches that yields robust learning of boolean characterizations. Our method has been implemented in a computer program, and we plot its empirical learning performance in the presence of varying amounts of noise
Privacy Risk in Machine Learning: Analyzing the Connection to Overfitting
Machine learning algorithms, when applied to sensitive data, pose a distinct
threat to privacy. A growing body of prior work demonstrates that models
produced by these algorithms may leak specific private information in the
training data to an attacker, either through the models' structure or their
observable behavior. However, the underlying cause of this privacy risk is not
well understood beyond a handful of anecdotal accounts that suggest overfitting
and influence might play a role.
This paper examines the effect that overfitting and influence have on the
ability of an attacker to learn information about the training data from
machine learning models, either through training set membership inference or
attribute inference attacks. Using both formal and empirical analyses, we
illustrate a clear relationship between these factors and the privacy risk that
arises in several popular machine learning algorithms. We find that overfitting
is sufficient to allow an attacker to perform membership inference and, when
the target attribute meets certain conditions about its influence, attribute
inference attacks. Interestingly, our formal analysis also shows that
overfitting is not necessary for these attacks and begins to shed light on what
other factors may be in play. Finally, we explore the connection between
membership inference and attribute inference, showing that there are deep
connections between the two that lead to effective new attacks
On Unconstrained Quasi-Submodular Function Optimization
With the extensive application of submodularity, its generalizations are
constantly being proposed. However, most of them are tailored for special
problems. In this paper, we focus on quasi-submodularity, a universal
generalization, which satisfies weaker properties than submodularity but still
enjoys favorable performance in optimization. Similar to the diminishing return
property of submodularity, we first define a corresponding property called the
{\em single sub-crossing}, then we propose two algorithms for unconstrained
quasi-submodular function minimization and maximization, respectively. The
proposed algorithms return the reduced lattices in iterations,
and guarantee the objective function values are strictly monotonically
increased or decreased after each iteration. Moreover, any local and global
optima are definitely contained in the reduced lattices. Experimental results
verify the effectiveness and efficiency of the proposed algorithms on lattice
reduction.Comment: 11 page
- …