185,440 research outputs found
Learning to Efficiently Rank
Web search engines allow users to find information on almost any topic imaginable. To be successful, a search engine must return relevant information to the user in a short amount of time. However, efficiency (speed) and effectiveness (relevance) are competing forces that often counteract each other. It is often the case that methods developed for improving effectiveness incur moderate-to-large computational costs, thus sustained effectiveness gains typically have to be counter-balanced by buying more/faster hardware, implementing caching strategies if possible, or spending additional effort in low-level optimizations.Â
This thesis describes the "Learning to Efficiently Rank" framework for building highly effective ranking models for Web-scale data, without sacrificing run-time efficiency for returning results. It introduces new classes of ranking models that have the capability of being simultaneously fast and effective, and discusses the issue of how to optimize the models for speed and effectiveness. More specifically, a series of concrete instantiations of the general "Learning to Efficiently Rank" framework are illustrated in detail. First, given a desired tradeoff between effectiveness/efficiency, efficient linear models, which have a mechanism to directly optimize the tradeoff metric and achieve an optimal balance between effectiveness/efficiency, are introduced. Second, temporally constrained models for returning the most effective ranked results possible under a time constraint are described. Third, a cascade ranking model for efficient top-K retrieval over Web-scale documents is proposed, where the ranking effectiveness and efficiency are simultaneously optimized. Finally, a constrained cascade for returning results within time constraints by simultaneously reducing document set size and unnecessary features is discussed in detail
Model Spider: Learning to Rank Pre-Trained Models Efficiently
Figuring out which Pre-Trained Model (PTM) from a model zoo fits the target
task is essential to take advantage of plentiful model resources. With the
availability of numerous heterogeneous PTMs from diverse fields, efficiently
selecting the most suitable PTM is challenging due to the time-consuming costs
of carrying out forward or backward passes over all PTMs. In this paper, we
propose Model Spider, which tokenizes both PTMs and tasks by summarizing their
characteristics into vectors to enable efficient PTM selection. By leveraging
the approximated performance of PTMs on a separate set of training tasks, Model
Spider learns to construct tokens and measure the fitness score between a
model-task pair via their tokens. The ability to rank relevant PTMs higher than
others generalizes to new tasks. With the top-ranked PTM candidates, we further
learn to enrich task tokens with their PTM-specific semantics to re-rank the
PTMs for better selection. Model Spider balances efficiency and selection
ability, making PTM selection like a spider preying on a web. Model Spider
demonstrates promising performance in various configurations of model zoos
Fast Low-Rank Matrix Learning with Nonconvex Regularization
Low-rank modeling has a lot of important applications in machine learning,
computer vision and social network analysis. While the matrix rank is often
approximated by the convex nuclear norm, the use of nonconvex low-rank
regularizers has demonstrated better recovery performance. However, the
resultant optimization problem is much more challenging. A very recent
state-of-the-art is based on the proximal gradient algorithm. However, it
requires an expensive full SVD in each proximal step. In this paper, we show
that for many commonly-used nonconvex low-rank regularizers, a cutoff can be
derived to automatically threshold the singular values obtained from the
proximal operator. This allows the use of power method to approximate the SVD
efficiently. Besides, the proximal operator can be reduced to that of a much
smaller matrix projected onto this leading subspace. Convergence, with a rate
of O(1/T) where T is the number of iterations, can be guaranteed. Extensive
experiments are performed on matrix completion and robust principal component
analysis. The proposed method achieves significant speedup over the
state-of-the-art. Moreover, the matrix solution obtained is more accurate and
has a lower rank than that of the traditional nuclear norm regularizer.Comment: Long version of conference paper appeared ICDM 201
- …