Gaining Insight into Determinants of Physical Activity using Bayesian Network Learning

Contains fulltext : 228326pre.pdf (preprint version ) (Open Access) Contains fulltext : 228326pub.pdf (publisher's version ) (Open Access)BNAIC/BeneLearn 202

Can bounded and self-interested agents be teammates? Application to planning in ad hoc teams

Planning for ad hoc teamwork is challenging because it involves agents collaborating without any prior coordination or communication. The focus is on principled methods for a single agent to cooperate with others. This motivates investigating the ad hoc teamwork problem in the context of self-interested decision-making frameworks. Agents engaged in individual decision making in multiagent settings face the task of having to reason about other agents’ actions, which may in turn involve reasoning about others. An established approximation that operationalizes this approach is to bound the infinite nesting from below by introducing level 0 models. For the purposes of this study, individual, self-interested decision making in multiagent settings is modeled using interactive dynamic influence diagrams (I-DID). These are graphical models with the benefit that they naturally offer a factored representation of the problem, allowing agents to ascribe dynamic models to others and reason about them. We demonstrate that an implication of bounded, finitely-nested reasoning by a self-interested agent is that we may not obtain optimal team solutions in cooperative settings, if it is part of a team. We address this limitation by including models at level 0 whose solutions involve reinforcement learning. We show how the learning is integrated into planning in the context of I-DIDs. This facilitates optimal teammate behavior, and we demonstrate its applicability to ad hoc teamwork on several problem domains and configurations

Dynamic land use/cover change modelling

Landnutzungswandel ist eine komplexe Angelegenheit, die durch zahlreiche biophysikalische, sozioökonomische und wirtschaftliche Faktoren verursacht wird. Eine offensichtliche Art des Landnutzungswandels, die in den suburbanen Gebieten einer Metropole stattfindet, ist die Zersiedelung. Es gibt viele Modellierungstechniken, um dieses Phänomen zu studieren. Diese wurden seit den 1960iger Jahren entwickelt und finden weite Verbreitung. Einige dieser Modelle leiden unter dem Vernachlässigen signifikanter Variablen. Traditionelle Methoden wie etwa zellulare Automaten, Markow-Ketten-Modelle, zellulare Automaten-Markow-Modelle und logistische Regressionsmodelle, weisen inhärente Schwächen auf in Bezug auf menschliche Aktivitäten in der Umwelt. Das liegt daran, dass der Mensch der Hauptakteur in der Transformation der Umwelt ist und die suburbanen Gebiete durch Niederlassungspräferenzen und Lebensstil prägt. Das Hauptziel dieser Dissertation ist es, einige dieser traditionellen Techniken zu untersuchen, um ihre Vor- und Nachteile zu identifizieren. Diese Modelle werden miteinander verglichen, um ihre Funktionalität zu hinterfragen. Obwohl die Methodologie zur Evaluierung agentenbasierter Modelle unzureichend ist, wurde hier versucht, ein selbst-kalibriertes agentenbasiertes Modell für den Großraum Teheran zu erstellen. Einige Variablen, die in der Wirklichkeit die Zersiedelung im Studiengebiet kontrollieren, wurden durch Expertenwissen und ähnliche Studien extrahiert. Drei Hauptagenten, die mit der Ausbreitung von Städten zu tun haben, wurden definiert: Entwickler, Bewohner, Behörden. Jeder einzelne Agent beeinflusst Variablen; d.h. die Entscheidungen eines Agenten werden von einer Reihe realer Variablen beeinflusst. Das Verhalten der einzelnen Agenten wurde in einer GIS Umgebung kodiert und anschließend zusammengeführt, um einen Prototyp zur Simulation der Landnutzungsänderung zu erzeugen. Dieser Geosimulations-Prototyp ist in der Lage, die Quantität und die Lage von Landnutzungsänderungen insbesondere in der Umgebung von Teheran zu simulieren. Dieses agentenbasierte Modell zieht Nutzen aus der Stärke traditioneller Techniken wie etwa zellularen Automaten zur Änderungsallokation, Markow-Modellen zur Schätzung der Quantität der Änderung und einer Gewichtung der individuellen Faktoren. Eine detaillierte Diskussion der Implementierung der unterschiedlichen Methoden sowie eine Stärken-Schwächen-Analyse werden präsentiert und die Ergebnisse mit der tatsächlichen Situation verglichen, um die Modelle zu verifizieren. In dieser Arbeit wurden GIS Funktionen verwendet und zusätzliche Funktionen in Python programmiert. Diese Untersuchungen sollen Stadtplaner und Entscheidungsträger unterstützen, Städte und deren Ausbreitung zu simulieren.Land use/ cover change is a complex matter, which is caused by numerous biophysical, socio-economical and economic factors. An obvious form of land use change in the suburbs of the metropolis is defined as urban sprawl. There are a number of techniques to model this issue in order to investigate this topic. These models have been developed since the 1960s and are increasing in terms of quantity and popularity. Some of these models suffer from a lack of consideration of some significant variables. The traditional methods (e.g. Cellular Automata, the Markov Chain Model, the CA-Markov Model, and the Logistic Regression Model) have some inherent weaknesses in consideration of human activity in the environment. The particular significance of this problem is the fact that humans are the main actors in the transformation of the environment, and impact upon the suburbs due to their settlement preferences and lifestyle choices. The main aim of this thesis was to examine some of those traditional techniques in order to discover their considerable advantages and disadvantages. These models were compared against each other to challenge their functionality. Whereas there is a lack of methodology in evaluation of agent-based models, it was presumed to create a self-calibrated agent based model, by focussing on the Tehran metropolitan area. Some variables in reality control urban sprawl in the study area, which were extracted through the expert knowledge and similar studies. Three main agents, which deal with urban expansion, were defined: developers, residents, government. Each particular agent affects some variables, i.e. the agents‟ decisions are being influenced by a set of real variables. Agents‟ behaviours were coded in a GIS environment and, thereafter, the predefined agents were combined through a function to create a prototype for simulation of land change. This designed geosimulation prototype can simulate the quantity and location of changes specifically in the vicinity of the metropolis of Tehran. This customised agent-based model benefits from the strengths of traditional techniques; for instance, a Cellular Automata structure for change allocation, a Markov model for change quantity estimation and a weighting system to differentiate between the weights of the driving factors. A detailed discussion of each methodology implementation, and their weakness and strengths, is then presented, specifically comparing results with the reality to verify the models. In this research, we used only the GIS functionalities within GIS environments and the required functions were coded in the Python engine. This investigation will help urban planners and urban decision-makers to simulate cities and their movements over time

Reinforcement Learning

Brains rule the world, and brain-like computation is increasingly used in computers and electronic devices. Brain-like computation is about processing and interpreting data or directly putting forward and performing actions. Learning is a very important aspect. This book is on reinforcement learning which involves performing actions to achieve a goal. The first 11 chapters of this book describe and extend the scope of reinforcement learning. The remaining 11 chapters show that there is already wide usage in numerous fields. Reinforcement learning can tackle control tasks that are too complex for traditional, hand-designed, non-learning controllers. As learning computers can deal with technical complexities, the tasks of human operators remain to specify goals on increasingly higher levels. This book shows that reinforcement learning is a very dynamic area in terms of theory and applications and it shall stimulate and encourage new research in this field

On the connection of probabilistic model checking, planning, and learning for system verification

This thesis presents approaches using techniques from the model checking, planning, and learning community to make systems more reliable and perspicuous. First, two heuristic search and dynamic programming algorithms are adapted to be able to check extremal reachability probabilities, expected accumulated rewards, and their bounded versions, on general Markov decision processes (MDPs). Thereby, the problem space originally solvable by these algorithms is enlarged considerably. Correctness and optimality proofs for the adapted algorithms are given, and in a comprehensive case study on established benchmarks it is shown that the implementation, called Modysh, is competitive with state-of-the-art model checkers and even outperforms them on very large state spaces. Second, Deep Statistical Model Checking (DSMC) is introduced, usable for quality assessment and learning pipeline analysis of systems incorporating trained decision-making agents, like neural networks (NNs). The idea of DSMC is to use statistical model checking to assess NNs resolving nondeterminism in systems modeled as MDPs. The versatility of DSMC is exemplified in a number of case studies on Racetrack, an MDP benchmark designed for this purpose, flexibly modeling the autonomous driving challenge. In a comprehensive scalability study it is demonstrated that DSMC is a lightweight technique tackling the complexity of NN analysis in combination with the state space explosion problem.Diese Arbeit präsentiert Ansätze, die Techniken aus dem Model Checking, Planning und Learning Bereich verwenden, um Systeme verlässlicher und klarer verständlich zu machen. Zuerst werden zwei Algorithmen für heuristische Suche und dynamisches Programmieren angepasst, um Extremwerte für Erreichbarkeitswahrscheinlichkeiten, Erwartungswerte für Kosten und beschränkte Varianten davon, auf generellen Markov Entscheidungsprozessen (MDPs) zu untersuchen. Damit wird der Problemraum, der ursprünglich mit diesen Algorithmen gelöst wurde, deutlich erweitert. Korrektheits- und Optimalitätsbeweise für die angepassten Algorithmen werden gegeben und in einer umfassenden Fallstudie wird gezeigt, dass die Implementierung, namens Modysh, konkurrenzfähig mit den modernsten Model Checkern ist und deren Leistung auf sehr großen Zustandsräumen sogar übertrifft. Als Zweites wird Deep Statistical Model Checking (DSMC) für die Qualitätsbewertung und Lernanalyse von Systemen mit integrierten trainierten Entscheidungsgenten, wie z.B. neuronalen Netzen (NN), eingeführt. Die Idee von DSMC ist es, statistisches Model Checking zur Bewertung von NNs zu nutzen, die Nichtdeterminismus in Systemen, die als MDPs modelliert sind, auflösen. Die Vielseitigkeit des Ansatzes wird in mehreren Fallbeispielen auf Racetrack gezeigt, einer MDP Benchmark, die zu diesem Zweck entwickelt wurde und die Herausforderung des autonomen Fahrens flexibel modelliert. In einer umfassenden Skalierbarkeitsstudie wird demonstriert, dass DSMC eine leichtgewichtige Technik ist, die die Komplexität der NN-Analyse in Kombination mit dem State Space Explosion Problem bewältigt

The computerization of archaeology: survey on AI techniques

This paper analyses the application of artificial intelligence techniques to various areas of archaeology and more specifically: a) The use of software tools as a creative stimulus for the organization of exhibitions; the use of humanoid robots and holographic displays as guides that interact and involve museum visitors; b) The analysis of methods for the classification of fragments found in archaeological excavations and for the reconstruction of ceramics, with the recomposition of the parts of text missing from historical documents and epigraphs; c) The cataloguing and study of human remains to understand the social and historical context of belonging with the demonstration of the effectiveness of the AI techniques used; d) The detection of particularly difficult terrestrial archaeological sites with the analysis of the architectures of the Artificial Neural Networks most suitable for solving the problems presented by the site; the design of a study for the exploration of marine archaeological sites, located at depths that cannot be reached by man, through the construction of a freely explorable 3D version

Multi-Agent Systems

A multi-agent system (MAS) is a system composed of multiple interacting intelligent agents. Multi-agent systems can be used to solve problems which are difficult or impossible for an individual agent or monolithic system to solve. Agent systems are open and extensible systems that allow for the deployment of autonomous and proactive software components. Multi-agent systems have been brought up and used in several application domains

Computational intelligence approaches to robotics, automation, and control [Volume guest editors]

No abstract available