3,127 research outputs found
Sparse CCA: Adaptive Estimation and Computational Barriers
Canonical correlation analysis is a classical technique for exploring the
relationship between two sets of variables. It has important applications in
analyzing high dimensional datasets originated from genomics, imaging and other
fields. This paper considers adaptive minimax and computationally tractable
estimation of leading sparse canonical coefficient vectors in high dimensions.
First, we establish separate minimax estimation rates for canonical coefficient
vectors of each set of random variables under no structural assumption on
marginal covariance matrices. Second, we propose a computationally feasible
estimator to attain the optimal rates adaptively under an additional sample
size condition. Finally, we show that a sample size condition of this kind is
needed for any randomized polynomial-time estimator to be consistent, assuming
hardness of certain instances of the Planted Clique detection problem. The
result is faithful to the Gaussian models used in the paper. As a byproduct, we
obtain the first computational lower bounds for sparse PCA under the Gaussian
single spiked covariance model
Pareto-Path Multi-Task Multiple Kernel Learning
A traditional and intuitively appealing Multi-Task Multiple Kernel Learning
(MT-MKL) method is to optimize the sum (thus, the average) of objective
functions with (partially) shared kernel function, which allows information
sharing amongst tasks. We point out that the obtained solution corresponds to a
single point on the Pareto Front (PF) of a Multi-Objective Optimization (MOO)
problem, which considers the concurrent optimization of all task objectives
involved in the Multi-Task Learning (MTL) problem. Motivated by this last
observation and arguing that the former approach is heuristic, we propose a
novel Support Vector Machine (SVM) MT-MKL framework, that considers an
implicitly-defined set of conic combinations of task objectives. We show that
solving our framework produces solutions along a path on the aforementioned PF
and that it subsumes the optimization of the average of objective functions as
a special case. Using algorithms we derived, we demonstrate through a series of
experimental results that the framework is capable of achieving better
classification performance, when compared to other similar MTL approaches.Comment: Accepted by IEEE Transactions on Neural Networks and Learning System
Convex Banding of the Covariance Matrix
We introduce a new sparse estimator of the covariance matrix for
high-dimensional models in which the variables have a known ordering. Our
estimator, which is the solution to a convex optimization problem, is
equivalently expressed as an estimator which tapers the sample covariance
matrix by a Toeplitz, sparsely-banded, data-adaptive matrix. As a result of
this adaptivity, the convex banding estimator enjoys theoretical optimality
properties not attained by previous banding or tapered estimators. In
particular, our convex banding estimator is minimax rate adaptive in Frobenius
and operator norms, up to log factors, over commonly-studied classes of
covariance matrices, and over more general classes. Furthermore, it correctly
recovers the bandwidth when the true covariance is exactly banded. Our convex
formulation admits a simple and efficient algorithm. Empirical studies
demonstrate its practical effectiveness and illustrate that our exactly-banded
estimator works well even when the true covariance matrix is only close to a
banded matrix, confirming our theoretical results. Our method compares
favorably with all existing methods, in terms of accuracy and speed. We
illustrate the practical merits of the convex banding estimator by showing that
it can be used to improve the performance of discriminant analysis for
classifying sound recordings
A Survey of Monte Carlo Tree Search Methods
Monte Carlo tree search (MCTS) is a recently proposed search method that combines the precision of tree search with the generality of random sampling. It has received considerable interest due to its spectacular success in the difficult problem of computer Go, but has also proved beneficial in a range of other domains. This paper is a survey of the literature to date, intended to provide a snapshot of the state of the art after the first five years of MCTS research. We outline the core algorithm's derivation, impart some structure on the many variations and enhancements that have been proposed, and summarize the results from the key game and nongame domains to which MCTS methods have been applied. A number of open research questions indicate that the field is ripe for future work
Shaping Social Activity by Incentivizing Users
Events in an online social network can be categorized roughly into endogenous
events, where users just respond to the actions of their neighbors within the
network, or exogenous events, where users take actions due to drives external
to the network. How much external drive should be provided to each user, such
that the network activity can be steered towards a target state? In this paper,
we model social events using multivariate Hawkes processes, which can capture
both endogenous and exogenous event intensities, and derive a time dependent
linear relation between the intensity of exogenous events and the overall
network activity. Exploiting this connection, we develop a convex optimization
framework for determining the required level of external drive in order for the
network to reach a desired activity level. We experimented with event data
gathered from Twitter, and show that our method can steer the activity of the
network more accurately than alternatives
- …