A Survey of Monte Carlo Tree Search Methods

Monte Carlo tree search (MCTS) is a recently proposed search method that combines the precision of tree search with the generality of random sampling. It has received considerable interest due to its spectacular success in the difficult problem of computer Go, but has also proved beneficial in a range of other domains. This paper is a survey of the literature to date, intended to provide a snapshot of the state of the art after the first five years of MCTS research. We outline the core algorithm's derivation, impart some structure on the many variations and enhancements that have been proposed, and summarize the results from the key game and nongame domains to which MCTS methods have been applied. A number of open research questions indicate that the field is ripe for future work

Scenarios of Urban Growth in Kenya Using Regionalised Cellular Automata based on Multi temporal Landsat Satellite Data

The exponential increase of urban areas in Africa during the last decade has become a major concern in the context of local climatic change and the increasing amount of impervious surface. Major African cities such as Nairobi and Nakuru have undergone rapid urban growth in comparison to the rest of the world. In this research we investigated the land-use changes and used the results in urban growth modelling which integrates cellular automata (CA), remote sensing (RS) and geographic information systems (GIS) in order to simulate urban growth up to the year 2030. We used multi-temporal Landsat imageries for the years 1986, 2000 and 2010 to map urban land-use changes in Nairobi and Nakuru. The use of multi-sensor imageries was also explored incorporating World view 2, and Advanced Land Observing Satellite (ALOS) Phased Array type L-band Synthetic Aperture Radar (PALSAR) data for urban land-use mapping in Nakuru. We conducted supervised classification using support vector machine (SVM) which performed better than maximum likelihood classification. Land-use change estimates were obtained indicating increased urban growth into the year 2010. We used the land-use change analysis information to model urban growth in Nairobi and Nakuru. Our urban growth model (UGM) utilised various datasets in modelling urban growth namely urban land-use extracted from land-use maps, road network data, slope data and exclusion layer defining areas excluded from development. The Monte-Carlo technique was used in model calibration. The model was validated using Multiple Resolution Validation (MRV) technique. Prediction of urban land-use was done up to the year 2030 when Kenya plans to attain Vision 2030. Three scenarios were explored in the urban modelling process; unmanaged growth with no restriction on environmental areas, managed growth with moderate protection, and a managed growth with maximum protection on forest, agricultural areas, and urban green. Furthermore, we explored the spatial effects of varying UGM parameters using the city of Nairobi. The objective here was to investigate the contribution of each model parameter in simulating urban growth. The results obtained indicate that varying model coefficients leads to urban growth in different directions and magnitude. However, several model parameters were observed to be highly correlated namely; spread, breed and road. The lowest spatial effect was achieved by at least maintaining spread, breed and road while varying the other parameters. The highest spatial effect was observed by at least keeping slope constant while varying the other four parameters. Additionally, we used kappa statistics to compare the simulation maps. High values of Khisto indicated high similarity between the maps in terms of quantity and location thus indicating the lowest spatial effect obtained. Kenya plans to achieve Vision 2030 in the year 2030 and information on spatial effects of our UGM can help in identifying different scenarios of future urban growth. It is thus possible to discover areas that are likely to experience; spontaneous growth, edge growth, road influenced growth or new spreading centres growth. Policy makers can see the influence of establishing new infrastructure such as housing and road in new areas compared to existing settlements. Moreover, the outcome of this research indicates that Nairobi and Nakuru are experiencing fast urban sprawl with urban land-use consuming the available land. The results obtained illustrate the possibility of urban growth modelling in addressing regional planning issues. This can help in comprehensive land-use planning and an integrated management of resources to ensure sustainability of land and to achieve social equity, economic efficiency and environmental sustainability. Hence, cellular automata are a worthwhile approach for regional modelling of African cities such as Nairobi and Nakuru. This provides opportunities for other cities in Africa to be studied using UGM and its adaptability noted accordingly.Das exponentielle Wachstum afrikanischer Städte im letzten Jahrzehnt ist mit Blick auf die lokalen klimatischen Veränderungen und der zunehmenden Menge an versiegelten Oberflächen von besonderer Tragweite. Im Vergleich zu anderen Metropolen erfuhren afrikanische Städte wie Nairobi und Nakuru ein extensives Wachstum der urbanen Flächen. Die vorliegende Arbeit setzt sich mit dem urbanen Landnutzungswandel auseinander und modelliert die Siedlungsflächenausdehnung für das Jahr 2030 mit Hilfe eines Zellulären Automaten (CA), Fernerkundungsdaten (RS) sowie Geographischen Informationssystemen (GIS). Zur Kartierung der Siedlungsflächenausdehnung von Nairobi und Nakuru wurden multitemporale Landsat-Daten der Jahre 1986, 2000 und 2010 verwendet. Zusätzlich wurden multisensorale Daten von World View 2 und ALOS PALSAR für Nakuru eingesetzt. Die Landnutzungsklassifikation erfolgte mit support vector machines (SVM). Dieses Verfahren zeigte bessere Ergebnisse als eine Maximum-Likelihood-Klassifikation. Auf Basis der klassifizierten Satellitendaten erfolgte die Landnutzungsmodellierung für Nairobi und Nakuru. Hierzu wurde die von Goetzke (2011) modifizierte Version von Clarke’s Urban Growth Model (Clarke, Hoppen, & Gaydos, 1997) benutzt. Neben den Landnutzungskarten fungieren Informationen zum Verkehrsnetz, zur Hangneigung und zu Ausschlussflächen als Hauptinputdaten. Die Kalibration erfolgte mit Hilfe von Monte Carlo Iterationen. Zur Validation des Modells wurde eine Multiple Resolution Validation (MRV) durchgeführt. Die Siedlungsflächenausdehnung wurde für das Jahr 2030 simuliert. Zu diesem Zeitpunkt plant das Land Kenia die Umsetzung des Vision 2030 Programmes. Es wurden insgesamt drei Szenarien mit dem Wachstumsmodell gerechnet: (1) Wachstum ohne Planungszwänge, so dass auch Siedlungsflächen in Naturschutzgebieten entstehen dürfen. (2) Siedlungsflächenausdehnung unter moderaten Planungsbedingungen. (3) Wachstum mit sehr restriktiven Planungsbedingungen, unter Einschluss des Schutzes von Wald-, Grün- und- Agrarflächen. Des Weiteren wurde eine Sensitivitätsanalyse der modelleigenen Wachstumsparameter am Beispiel von Nairobi durchgeführt. Es konnte gezeigt werden, welchen Einfluss die Parameter auf die Intensität und das Muster der modellierten Siedlungsflächenausdehnung ausüben. Dabei zeigten die Wachstumskoeffizienten „spread“, „breed“ und „road“ eine signifikante Korrelation. Zur weiteren Analyse der erzielten Modellierungsergebnisse und zum Vergleich der räumlichen Muster wurden Kappa-Statistiken herangezogen. Die Arbeit sieht sich als Beitrag zum Vision 2030 Diskurs der kenianischen Regierung. Die simulierten Szenarien der Siedlungsflächenausdehnung von Nairobi und Nakuru identifizieren die für eine Urbanisierung wahrscheinlich in Frage kommenden Regionen. Die Studie zeigt zudem, dass sich die Siedlungsflächenausdehnung von Nairobi und Nakuru schnell und mit hohen Wachstumsraten vollzieht. Der Einsatz von CA Modellen ist ein wertvoller Ansatz zur regionalen Modellierung nicht nur von kenianischen sondern auch von afrikanischen Städten. Die Arbeit kann somit Entscheidungsträger aus Politik und Verwaltung unterstützen, indem sie die räumlichen Auswirkungen des zukünftigen Ausbaus der Infrastruktur und von Wohnflächen aufzeigt. Eine umfassende Planung von Landnutzungswandel und ein integriertes Management sind essentiell auf dem Weg zu einem bewussteren Umgang mit der Ressource Land sowie zu einer sozialen Gleichheit, wirtschaftlichen Effizienz und einer ökologischen Nachhaltigkeit

Evolutionary games on graphs

Game theory is one of the key paradigms behind many scientific disciplines from biology to behavioral sciences to economics. In its evolutionary form and especially when the interacting agents are linked in a specific social network the underlying solution concepts and methods are very similar to those applied in non-equilibrium statistical physics. This review gives a tutorial-type overview of the field for physicists. The first three sections introduce the necessary background in classical and evolutionary game theory from the basic definitions to the most important results. The fourth section surveys the topological complications implied by non-mean-field-type social network structures in general. The last three sections discuss in detail the dynamic behavior of three prominent classes of models: the Prisoner's Dilemma, the Rock-Scissors-Paper game, and Competing Associations. The major theme of the review is in what sense and how the graph structure of interactions can modify and enrich the picture of long term behavioral patterns emerging in evolutionary games.Comment: Review, final version, 133 pages, 65 figure

Bayesian Nonparametric Methods for Partially-Observable Reinforcement Learning

Making intelligent decisions from incomplete information is critical in many applications: for example, robots must choose actions based on imperfect sensors, and speech-based interfaces must infer a user’s needs from noisy microphone inputs. What makes these tasks hard is that often we do not have a natural representation with which to model the domain and use for choosing actions; we must learn about the domain’s properties while simultaneously performing the task. Learning a representation also involves trade-offs between modeling the data that we have seen previously and being able to make predictions about new data. This article explores learning representations of stochastic systems using Bayesian nonparametric statistics. Bayesian nonparametric methods allow the sophistication of a representation to scale gracefully with the complexity in the data. Our main contribution is a careful empirical evaluation of how representations learned using Bayesian nonparametric methods compare to other standard learning approaches, especially in support of planning and control. We show that the Bayesian aspects of the methods result in achieving state-of-the-art performance in decision making with relatively few samples, while the nonparametric aspects often result in fewer computations. These results hold across a variety of different techniques for choosing actions given a representation

A survey of statistical network models

Networks are ubiquitous in science and have become a focal point for discussion in everyday life. Formal statistical models for the analysis of network data have emerged as a major topic of interest in diverse areas of study, and most of these involve a form of graphical representation. Probability models on graphs date back to 1959. Along with empirical studies in social psychology and sociology from the 1960s, these early works generated an active network community and a substantial literature in the 1970s. This effort moved into the statistical literature in the late 1970s and 1980s, and the past decade has seen a burgeoning network literature in statistical physics and computer science. The growth of the World Wide Web and the emergence of online networking communities such as Facebook, MySpace, and LinkedIn, and a host of more specialized professional network communities has intensified interest in the study of networks and network data. Our goal in this review is to provide the reader with an entry point to this burgeoning literature. We begin with an overview of the historical development of statistical network modeling and then we introduce a number of examples that have been studied in the network literature. Our subsequent discussion focuses on a number of prominent static and dynamic network models and their interconnections. We emphasize formal model descriptions, and pay special attention to the interpretation of parameters and their estimation. We end with a description of some open problems and challenges for machine learning and statistics.Comment: 96 pages, 14 figures, 333 reference

Intelligent Simulation Modeling of a Flexible Manufacturing System with Automated Guided Vehicles

Although simulation is a very flexible and cost effective problem solving technique, it has been traditionally limited to building models which are merely descriptive of the system under study. Relatively new approaches combine improvement heuristics and artificial intelligence with simulation to provide prescriptive power in simulation modeling. This study demonstrates the synergy obtained by bringing together the "learning automata theory" and simulation analysis. Intelligent objects are embedded in the simulation model of a Flexible Manufacturing System (FMS), in which Automated Guided Vehicles (AGVs) serve as the material handling system between four unique workcenters. The objective of the study is to find satisfactory AGV routing patterns along available paths to minimize the mean time spent by different kinds of parts in the system. System parameters such as different part routing and processing time requirements, arrivals distribution, number of palettes, available paths between workcenters, number and speed of AGVs can be defined by the user. The network of learning automata acts as the decision maker driving the simulation, and the FMS model acts as the training environment for the automata network; providing realistic, yet cost-effective and risk-free feedback. Object oriented design and implementation of the simulation model with a process oriented world view, graphical animation and visually interactive simulation (using GUI objects such as windows, menus, dialog boxes; mouse sensitive dynamic automaton trace charts and dynamic graphical statistical monitoring) are other issues dealt with in the study