10,038 research outputs found
On Quasi-Newton Forward--Backward Splitting: Proximal Calculus and Convergence
We introduce a framework for quasi-Newton forward--backward splitting
algorithms (proximal quasi-Newton methods) with a metric induced by diagonal
rank- symmetric positive definite matrices. This special type of
metric allows for a highly efficient evaluation of the proximal mapping. The
key to this efficiency is a general proximal calculus in the new metric. By
using duality, formulas are derived that relate the proximal mapping in a
rank- modified metric to the original metric. We also describe efficient
implementations of the proximity calculation for a large class of functions;
the implementations exploit the piece-wise linear nature of the dual problem.
Then, we apply these results to acceleration of composite convex minimization
problems, which leads to elegant quasi-Newton methods for which we prove
convergence. The algorithm is tested on several numerical examples and compared
to a comprehensive list of alternatives in the literature. Our quasi-Newton
splitting algorithm with the prescribed metric compares favorably against
state-of-the-art. The algorithm has extensive applications including signal
processing, sparse recovery, machine learning and classification to name a few.Comment: arXiv admin note: text overlap with arXiv:1206.115
From Common to Special: When Multi-Attribute Learning Meets Personalized Opinions
Visual attributes, which refer to human-labeled semantic annotations, have
gained increasing popularity in a wide range of real world applications.
Generally, the existing attribute learning methods fall into two categories:
one focuses on learning user-specific labels separately for different
attributes, while the other one focuses on learning crowd-sourced global labels
jointly for multiple attributes. However, both categories ignore the joint
effect of the two mentioned factors: the personal diversity with respect to the
global consensus; and the intrinsic correlation among multiple attributes. To
overcome this challenge, we propose a novel model to learn user-specific
predictors across multiple attributes. In our proposed model, the diversity of
personalized opinions and the intrinsic relationship among multiple attributes
are unified in a common-to-special manner. To this end, we adopt a
three-component decomposition. Specifically, our model integrates a common
cognition factor, an attribute-specific bias factor and a user-specific bias
factor. Meanwhile Lasso and group Lasso penalties are adopted to leverage
efficient feature selection. Furthermore, theoretical analysis is conducted to
show that our proposed method could reach reasonable performance. Eventually,
the empirical study carried out in this paper demonstrates the effectiveness of
our proposed method
- …