A Survey of Monte Carlo Tree Search Methods

Monte Carlo tree search (MCTS) is a recently proposed search method that combines the precision of tree search with the generality of random sampling. It has received considerable interest due to its spectacular success in the difficult problem of computer Go, but has also proved beneficial in a range of other domains. This paper is a survey of the literature to date, intended to provide a snapshot of the state of the art after the first five years of MCTS research. We outline the core algorithm's derivation, impart some structure on the many variations and enhancements that have been proposed, and summarize the results from the key game and nongame domains to which MCTS methods have been applied. A number of open research questions indicate that the field is ripe for future work

THE TOOLS AND MONTE CARLO WORKING GROUP Summary Report from the Les Houches 2009 Workshop on TeV Colliders

This is the summary and introduction to the proceedings contributions for the Les Houches 2009 "Tools and Monte Carlo" working group.Comment: 144 Pages. Workshop site http://wwwlapp.in2p3.fr/conferences/LesHouches/Houches2009/ . Conveners were Butterworth, Maltoni, Moortgat, Richardson, Schumann and Skand

The effect of pruning treatments on the vibration properties and wind-induced bending moments of Senegal mahogany (\u3cem\u3eKhaya senegalensis\u3c/em\u3e) and rain tree (\u3cem\u3eSamanea saman\u3c/em\u3e) in Singapore

During pruning, arborists often intend to increase a tree’s resistance to wind loading by selectively removing branches, but there are few studies examining the efficacy of these interventions, especially for large, open-grown trees. In this study, the mass and vibration properties of Senegal mahogany (Khaya senegalensis) and rain tree (Samanea saman) were measured before and after the crowns of trees were incrementally raised or reduced between 0 and 80%. In addition, the wind-induced vibration and bending moments of Senegal mahoganies were monitored before and after the same pruning treatments. For both species, total mass and leaf mass both decreased faster on reduced than raised trees. The frequency and damping ratio of trees varied with the severity of pruning for reduced, but not raised, trees. The frequency of reduced trees generally increased with pruning severity. In contrast, damping ratio of reduced trees generally decreased with the severity of pruning, except for the unique increase in damping ratio on Senegal mahoganies reduced by 10 to 20%. Post-pruning vibration properties were significantly related to the post-pruning morphometric attributes of reduced, but not raised, trees. For reduced trees, most of the examined tree and branch attributes better explained variability in post-pruning frequency than damping ratio. At each pruning severity, Fourier energy spectra showed that raised trees continued to vibrate primarily at their fundamental mode. As the severity of pruning increased, however, reduced trees vibrated progressively less than raised trees at all analyzed frequencies. Similarly, the average 30-minute maximum bending moment associated with a given 30-minute maximum wind speed decreased more for reduced than raised trees at low pruning severities. For those seeking to decrease the likelihood of tree failure, the results suggest that arborists should reduce trees to change their vibration properties and wind loads, but trees should be reduced by small amounts to avoid the undesirable decrease in damping ratio. Although the observed changes on reduced trees contributed favorably to risk mitigation, there are many adverse biological consequences of some pruning methods, especially topping, that shorten tree parts without considering the anatomy of trees or remove an excessive amount of branches and leaves, and arborists should use good judgment when pruning trees to reduce their size without unnecessarily disturbing tree growth and development. Moreover, these mechanical properties will inevitably change as trees grow after pruning, and more work is needed to understand both the long-term biological and mechanical consequences of pruning treatments

Effective techniques for handling incomplete data using decision trees

Decision Trees (DTs) have been recognized as one of the most successful formalisms for knowledge representation and reasoning and are currently applied to a variety of data mining or knowledge discovery applications, particularly for classification problems. There are several efficient methods to learn a DT from data. However, these methods are often limited to the assumption that data are complete. In this thesis, some contributions to the field of machine learning and statistics that solve the problem of extracting DTs for learning and classification tasks from incomplete databases are presented. The methodology underlying the thesis blends together well-established statistical theories with the most advanced techniques for machine learning and automated reasoning with uncertainty. The first contribution is the extensive simulations which study the impact of missing data on predictive accuracy of existing DTs which can cope with missing values, when missing values are in both the training and test sets or when they are in either of the two sets. All simulations are performed under missing completely at random, missing at random and informatively missing mechanisms and for different missing data patterns and proportions. The proposal of a simple, novel, yet effective proposed procedure for training and testing using decision trees in the presence of missing data is the next contribution. Original and simple splitting criteria for attribute selection in tree building are put forward. The proposed technique is evaluated and validated in empirical tests over many real world application domains. In this work, the proposed algorithm maintains (sometimes exceeds) the outstanding accuracy of multiple imputation, especially on datasets containing mixed attributes and purely nominal attributes. Also, the proposed algorithm greatly improves in accuracy for IM data. Another major advantage of this method over multiple imputation is the important saving in computational resources due to it simplicity. The next contribution is the proposal of three versions of simple probabilistic techniques that could be used for classifying incomplete vectors using decision trees based on complete data. The proposed procedure is superficially similar to that of fractional cases but more effective. The experimental results demonstrate that these approaches can achieve comparative quality to sophisticated algorithms like multiple imputation and therefore are applicable to all kinds of datasets. Finally, novel uses of two proposed ensemble procedures for handling incomplete training and test data are proposed and discussed. The algorithms combine the two best approaches either with resampling (REMIMIA) or without resampling (EMIMIA) of the training data before growing the decision trees. Experiments are used to evaluate and validate the success of the proposed ensemble methods with respect to individual missing data techniques in the form of empirical tests. EMIMIA attains the highest overall level of prediction accuracy

Temporal Difference Learning in Complex Domains

PhDThis thesis adapts and improves on the methods of TD(k) (Sutton 1988) that were successfully used for backgammon (Tesauro 1994) and applies them to other complex games that are less amenable to simple pattem-matching approaches. The games investigated are chess and shogi, both of which (unlike backgammon) require significant amounts of computational effort to be expended on search in order to achieve expert play. The improved methods are also tested in a non-game domain. In the chess domain, the adapted TD(k) method is shown to successfully learn the relative values of the pieces, and matches using these learnt piece values indicate that they perform at least as well as piece values widely quoted in elementary chess books. The adapted TD(X) method is also shown to work well in shogi, considered by many researchers to be the next challenge for computer game-playing, and for which there is no standardised set of piece values. An original method to automatically set and adjust the major control parameters used by TD(k) is presented. The main performance advantage comes from the learning rate adjustment, which is based on a new concept called temporal coherence. Experiments in both chess and a random-walk domain show that the temporal coherence algorithm produces both faster learning and more stable values than both human-chosen parameters and an earlier method for learning rate adjustment. The methods presented in this thesis allow programs to learn with as little input of external knowledge as possible, exploring the domain on their own rather than by being taught. Further experiments show that the method is capable of handling many hundreds of weights, and that it is not necessary to perform deep searches during the leaming phase in order to learn effective weight

Temoral Difference Learning in Complex Domains

Submitted to the University of London for the Degree of Doctor of Philosophy in Computer Scienc