9,793 research outputs found
Self-organizing search lists using probabilistic back-pointers
A class of algorithms is given for maintaining self-organizing sequential search lists, where the only permutation applied is to move the accessed record of each search some distance towards the front of the list. During searches, these algorithms retain a back-pointer to a previously probed record in order to determine the destination of the accessed record's eventual move. The back-pointer does not traverse the list, but rather it is advanced occationally to point to the record just probed by the search algorithm. This avoids the cost of a second traversal through a significant portion of the list, which may be a significant savings when each record access may require a new page to be brought into primary memory. Probabilistic functions for deciding when to advance the pointer are presented and analyzed. These functions demonstrate average case complexities of measures such as asymptotic cost and convergence similar to some of the more common list update algorithms in the literature. In cases where the accessed record is moved forward a distance proportional to the distance to the front of the list, the use of these functions may save up to 50% of the time required for permuting the list
Review of Metaheuristics and Generalized Evolutionary Walk Algorithm
Metaheuristic algorithms are often nature-inspired, and they are becoming
very powerful in solving global optimization problems. More than a dozen of
major metaheuristic algorithms have been developed over the last three decades,
and there exist even more variants and hybrid of metaheuristics. This paper
intends to provide an overview of nature-inspired metaheuristic algorithms,
from a brief history to their applications. We try to analyze the main
components of these algorithms and how and why they works. Then, we intend to
provide a unified view of metaheuristics by proposing a generalized
evolutionary walk algorithm (GEWA). Finally, we discuss some of the important
open questions.Comment: 14 page
Model Accuracy and Runtime Tradeoff in Distributed Deep Learning:A Systematic Study
This paper presents Rudra, a parameter server based distributed computing
framework tuned for training large-scale deep neural networks. Using variants
of the asynchronous stochastic gradient descent algorithm we study the impact
of synchronization protocol, stale gradient updates, minibatch size, learning
rates, and number of learners on runtime performance and model accuracy. We
introduce a new learning rate modulation strategy to counter the effect of
stale gradients and propose a new synchronization protocol that can effectively
bound the staleness in gradients, improve runtime performance and achieve good
model accuracy. Our empirical investigation reveals a principled approach for
distributed training of neural networks: the mini-batch size per learner should
be reduced as more learners are added to the system to preserve the model
accuracy. We validate this approach using commonly-used image classification
benchmarks: CIFAR10 and ImageNet.Comment: Accepted by The IEEE International Conference on Data Mining 2016
(ICDM 2016
Learning to Respond: The Use of Heuristics in Dynamic Games
While many learning models have been proposed in the game theoretic literature to track individuals’ behavior, surprisingly little research has focused on how well these models describe human adaptation in changing dynamic environments. Analysis of human behavior demonstrates that people are often remarkably responsive to changes in their environment, on time scales ranging from millennia (evolution) to milliseconds (reflex). The goal of this paper is to evaluate several prominent learning models in light of a laboratory experiment on responsiveness in a lowinformation dynamic game subject to changes in its underlying structure. While history-dependent reinforcement learning models track convergence of play well in repeated games, it is shown that they are ill suited to these environments, in which sastisficing models accurately predict behavior. A further objective is to determine which heuristics, or “rules of thumb,” when incorporated into learning models, are responsible for accurately capturing responsiveness. Reference points and a particular type of experimentation are found to be important in both describing and predicting play.learning, limited information, dynamic games
- …