21 research outputs found
Self-Paced Multi-Task Learning
In this paper, we propose a novel multi-task learning (MTL) framework, called
Self-Paced Multi-Task Learning (SPMTL). Different from previous works treating
all tasks and instances equally when training, SPMTL attempts to jointly learn
the tasks by taking into consideration the complexities of both tasks and
instances. This is inspired by the cognitive process of human brain that often
learns from the easy to the hard. We construct a compact SPMTL formulation by
proposing a new task-oriented regularizer that can jointly prioritize the tasks
and the instances. Thus it can be interpreted as a self-paced learner for MTL.
A simple yet effective algorithm is designed for optimizing the proposed
objective function. An error bound for a simplified formulation is also
analyzed theoretically. Experimental results on toy and real-world datasets
demonstrate the effectiveness of the proposed approach, compared to the
state-of-the-art methods
Self-Paced Learning: an Implicit Regularization Perspective
Self-paced learning (SPL) mimics the cognitive mechanism of humans and
animals that gradually learns from easy to hard samples. One key issue in SPL
is to obtain better weighting strategy that is determined by minimizer
function. Existing methods usually pursue this by artificially designing the
explicit form of SPL regularizer. In this paper, we focus on the minimizer
function, and study a group of new regularizer, named self-paced implicit
regularizer that is deduced from robust loss function. Based on the convex
conjugacy theory, the minimizer function for self-paced implicit regularizer
can be directly learned from the latent loss function, while the analytic form
of the regularizer can be even known. A general framework (named SPL-IR) for
SPL is developed accordingly. We demonstrate that the learning procedure of
SPL-IR is associated with latent robust loss functions, thus can provide some
theoretical inspirations for its working mechanism. We further analyze the
relation between SPL-IR and half-quadratic optimization. Finally, we implement
SPL-IR to both supervised and unsupervised tasks, and experimental results
corroborate our ideas and demonstrate the correctness and effectiveness of
implicit regularizers.Comment: 12 pages, 3 figure