89,542 research outputs found
Scalable Semidefinite Relaxation for Maximum A Posterior Estimation
Maximum a posteriori (MAP) inference over discrete Markov random fields is a
fundamental task spanning a wide spectrum of real-world applications, which is
known to be NP-hard for general graphs. In this paper, we propose a novel
semidefinite relaxation formulation (referred to as SDR) to estimate the MAP
assignment. Algorithmically, we develop an accelerated variant of the
alternating direction method of multipliers (referred to as SDPAD-LR) that can
effectively exploit the special structure of the new relaxation. Encouragingly,
the proposed procedure allows solving SDR for large-scale problems, e.g.,
problems on a grid graph comprising hundreds of thousands of variables with
multiple states per node. Compared with prior SDP solvers, SDPAD-LR is capable
of attaining comparable accuracy while exhibiting remarkably improved
scalability, in contrast to the commonly held belief that semidefinite
relaxation can only been applied on small-scale MRF problems. We have evaluated
the performance of SDR on various benchmark datasets including OPENGM2 and PIC
in terms of both the quality of the solutions and computation time.
Experimental results demonstrate that for a broad class of problems, SDPAD-LR
outperforms state-of-the-art algorithms in producing better MAP assignment in
an efficient manner.Comment: accepted to International Conference on Machine Learning (ICML 2014
Locally Differentially Private Gradient Tracking for Distributed Online Learning over Directed Graphs
Distributed online learning has been proven extremely effective in solving
large-scale machine learning problems over streaming data. However, information
sharing between learners in distributed learning also raises concerns about the
potential leakage of individual learners' sensitive data. To mitigate this
risk, differential privacy, which is widely regarded as the "gold standard" for
privacy protection, has been widely employed in many existing results on
distributed online learning. However, these results often face a fundamental
tradeoff between learning accuracy and privacy. In this paper, we propose a
locally differentially private gradient tracking based distributed online
learning algorithm that successfully circumvents this tradeoff. We prove that
the proposed algorithm converges in mean square to the exact optimal solution
while ensuring rigorous local differential privacy, with the cumulative privacy
budget guaranteed to be finite even when the number of iterations tends to
infinity. The algorithm is applicable even when the communication graph among
learners is directed. To the best of our knowledge, this is the first result
that simultaneously ensures learning accuracy and rigorous local differential
privacy in distributed online learning over directed graphs. We evaluate our
algorithm's performance by using multiple benchmark machine-learning
applications, including logistic regression of the "Mushrooms" dataset and
CNN-based image classification of the "MNIST" and "CIFAR-10" datasets,
respectively. The experimental results confirm that the proposed algorithm
outperforms existing counterparts in both training and testing accuracies.Comment: 21 pages, 4 figure
- …