3,549 research outputs found
LambdaOpt: Learn to Regularize Recommender Models in Finer Levels
Recommendation models mainly deal with categorical variables, such as
user/item ID and attributes. Besides the high-cardinality issue, the
interactions among such categorical variables are usually long-tailed, with the
head made up of highly frequent values and a long tail of rare ones. This
phenomenon results in the data sparsity issue, making it essential to
regularize the models to ensure generalization. The common practice is to
employ grid search to manually tune regularization hyperparameters based on the
validation data. However, it requires non-trivial efforts and large computation
resources to search the whole candidate space; even so, it may not lead to the
optimal choice, for which different parameters should have different
regularization strengths. In this paper, we propose a hyperparameter
optimization method, LambdaOpt, which automatically and adaptively enforces
regularization during training. Specifically, it updates the regularization
coefficients based on the performance of validation data. With LambdaOpt, the
notorious tuning of regularization hyperparameters can be avoided; more
importantly, it allows fine-grained regularization (i.e. each parameter can
have an individualized regularization coefficient), leading to better
generalized models. We show how to employ LambdaOpt on matrix factorization, a
classical model that is representative of a large family of recommender models.
Extensive experiments on two public benchmarks demonstrate the superiority of
our method in boosting the performance of top-K recommendation.Comment: Accepted by KDD 201
On the adequacy of untuned warmup for adaptive optimization
Adaptive optimization algorithms such as Adam are widely used in deep
learning. The stability of such algorithms is often improved with a warmup
schedule for the learning rate. Motivated by the difficulty of choosing and
tuning warmup schedules, recent work proposes automatic variance rectification
of Adam's adaptive learning rate, claiming that this rectified approach
("RAdam") surpasses the vanilla Adam algorithm and reduces the need for
expensive tuning of Adam with warmup. In this work, we refute this analysis and
provide an alternative explanation for the necessity of warmup based on the
magnitude of the update term, which is of greater relevance to training
stability. We then provide some "rule-of-thumb" warmup schedules, and we
demonstrate that simple untuned warmup of Adam performs more-or-less
identically to RAdam in typical practical settings. We conclude by suggesting
that practitioners stick to linear warmup with Adam, with a sensible default
being linear warmup over training iterations.Comment: AAAI 202
Real Time Animation of Virtual Humans: A Trade-off Between Naturalness and Control
Virtual humans are employed in many interactive applications using 3D virtual environments, including (serious) games. The motion of such virtual humans should look realistic (or ‘natural’) and allow interaction with the surroundings and other (virtual) humans. Current animation techniques differ in the trade-off they offer between motion naturalness and the control that can be exerted over the motion. We show mechanisms to parametrize, combine (on different body parts) and concatenate motions generated by different animation techniques. We discuss several aspects of motion naturalness and show how it can be evaluated. We conclude by showing the promise of combinations of different animation paradigms to enhance both naturalness and control
A Hit-and-Run approach for generating scale invariant Small World networks
Hit-and-Run is a well-known class of Markov chain algorithms for sampling from essentially arbitrary distributions over bounded regions of the Euclidean space. We present a class of Small World network models constructed using Hit-and-Run in a Euclidean ball. We prove that there is a unique scale invariant model in this class that admits efficient search by a decentralized algorithm. This research links two seemingly unrelated areas: Markov chain sampling techniques and scale invariant Small World networks, and may have interesting implications for stochastic search methods for continuous optimization. © 2008 Wiley Periodicals, Inc. NETWORKS, 2009Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/61434/1/20262_ftp.pd
- …