1,355 research outputs found
Artificial intelligence in co-operative games with partial observability
This thesis investigates Artificial Intelligence in co-operative games that feature Partial Observability. Most video games feature a combination of both co-operation, as well as Partial Observability. Co-operative games are games that feature a team of at least two agents, that must achieve a shared goal of some kind. Partial Observability is the restriction of how much of an environment that an agent can observe. The research performed in this thesis examines the challenge of creating Artificial Intelligence for co-operative games that feature Partial Observability. The main contributions are that Monte-Carlo Tree Search outperforms Genetic Algorithm based agents in solving co-operative problems without communication, the creation of a co-operative Partial Observability competition promoting Artificial Intelligence research as well as an investigation of the effect of varying Partial Observability to Artificial Intelligence, and finally the creation of a high performing Monte-Carlo Tree Search agent for the game Hanabi that uses agent modelling to rationalise about other players
Deep Reinforcement Learning and Game Theoretic Monte Carlo Decision Process for Safe and Efficient Lane Change Maneuver and Speed Management
Predicting the states of the surrounding traffic is one of the major problems in automated driving. Maneuvers such as lane change, merge, and exit management could pose challenges in the absence of intervehicular communication and can benefit from driver behavior prediction. Predicting the motion of surrounding vehicles and trajectory planning need to be computationally efficient for real-time implementation. This dissertation presents a decision process model for real-time automated lane change and speed management in highway and urban traffic. In lane change and merge maneuvers, it is important to know how neighboring vehicles will act in the imminent future. Human driver models, probabilistic approaches, rule-base techniques, and machine learning approach have addressed this problem only partially as they do not focus on the behavioral features of the vehicles. The main goal of this research is to develop a fast algorithm that predicts the future states of the neighboring vehicles, runs a fast decision process, and learns the regretfulness and rewardfulness of the executed decisions. The presented algorithm is developed based on level-K game theory to model and predict the interaction between the vehicles. Using deep reinforcement learning, this algorithm encodes and memorizes the past experiences that are recurrently used to reduce the computations and speed up motion planning. Also, we use Monte Carlo Tree Search (MCTS) as an effective tool that is employed nowadays for fast planning in complex and dynamic game environments. This development leverages the computation power efficiently and showcases promising outcomes for maneuver planning and predicting the environment’s dynamics. In the absence of traffic connectivity that may be due to either passenger’s choice of privacy or the vehicle’s lack of technology, this development can be extended and employed in automated vehicles for real-world and practical applications
Long-term Informative Path Planning with Autonomous Soaring
The ability of UAVs to cover large areas efficiently is valuable for information gathering missions. For long-term information gathering, a UAV may extend its endurance by accessing energy sources present in the atmosphere. Thermals are a favourable source of wind energy and thermal soaring is adopted in this thesis to enable long-term information gathering. This thesis proposes energy-constrained path planning algorithms for a gliding UAV to maximise information gain given a mission time that greatly exceeds the UAV's endurance. This thesis is motivated by the problem of probabilistic target-search performed by an energy-constrained UAV, which is tasked to simultaneously search for a lost ground target and explore for thermals to regain energy. This problem is termed informative soaring (IFS) and combines informative path planning (IPP) with energy constraints. IFS is shown to be NP-hard by showing that it has a similar problem structure to the weight-constrained shortest path problem with replenishments. While an optimal solution may not exist in polynomial time, this thesis proposes path planning algorithms based on informed tree search to find high quality plans with low computational cost. This thesis addresses complex probabilistic belief maps and three primary contributions are presented: • First, IFS is formulated as a graph search problem by observing that any feasible long-term plan must alternate between 1) information gathering between thermals and 2) replenishing energy within thermals. This is a first step to reducing the large search state space. • The second contribution is observing that a complex belief map can be viewed as a collection of information clusters and using a divide and conquer approach, cluster tree search (CTS), to efficiently find high-quality plans in the large search state space. In CTS, near-greedy tree search is used to find locally optimal plans and two global planning versions are proposed to combine local plans into a full plan. Monte Carlo simulation studies show that CTS produces similar plans to variations of exhaustive search, but runs five to 20 times faster. The more computationally efficient version, CTSDP, uses dynamic programming (DP) to optimally combine local plans. CTSDP is executed in real time on board a UAV to demonstrate computational feasibility. • The third contribution is an extension of CTS to unknown drifting thermals. A thermal exploration map is created to detect new thermals that will eventually intercept clusters, and therefore be valuable to the mission. Time windows are computed for known thermals and an optimal cluster visit schedule is formed. A tree search algorithm called CTSDrift combines CTS and thermal exploration. Using 2400 Monte Carlo simulations, CTSDrift is evaluated against a Full Knowledge method that has full knowledge of the thermal field and a Greedy method. On average, CTSDrift outperforms Greedy in one-third of trials, and achieves similar performance to Full Knowledge when environmental conditions are favourable
Explaining Deep Q-Learning Experience Replay with SHapley Additive exPlanations
Reinforcement Learning (RL) has shown promise in optimizing complex control and decision-making processes but Deep Reinforcement Learning (DRL) lacks interpretability, limiting its adoption in regulated sectors like manufacturing, finance, and healthcare. Difficulties arise from DRL’s opaque decision-making, hindering efficiency and resource use, this issue is amplified with every advancement. While many seek to move from Experience Replay to A3C, the latter demands more resources. Despite efforts to improve Experience Replay selection strategies, there is a tendency to keep capacity high. This dissertation investigates training a Deep Convolutional Q-learning agent across 20 Atari games, in solving a control task, physics task, and simulating addition, while intentionally reducing Experience Replay capacity from 1×106 to 5×102 . It was found that over 40% in the reduction of Experience Replay size is allowed for 18 of 23 simulations tested, offering a practical path to resource-efficient DRL. To illuminate agent decisions and align them with game mechanics, a novel method is employed: visualizing Experience Replay via Deep SHAP Explainer. This approach fosters comprehension and transparent, interpretable explanations, though any capacity reduction must be cautious to avoid overfitting. This study demonstrates the feasibility of reducing Experience Replay and advocates for transparent, interpretable decision explanations using the Deep SHAP Explainer to promote enhancing resource efficiency in Experience Replay
Deep Reinforcement Learning and its application to games
The following project aims is to review the main concepts of Reinforcement Learning and
combine them with the tools of Deep Learning, studying in depth the application of these
methodologies, the Deep Reinforcement Learning algorithms, that are having such an
impact today being applied to numerous fields such as autonomous driving, robot control, gaming
and many more. In order to do this, first, in chapter 1, we will give a general overview of Deep
Reinforcement Learning as a introduction, as well as which is motivation to study this topic.
Then, in chapter 2, since it will be fundamental to achieve our goal, we give a brief review of
Deep Learning. We get into details with chapter 3, where we define Reinforcement Learning
mathematically, formalizing the concepts in order to build the classic solution algorithms in
chapter 4. As an application of these techniques, the implementation of the algorithms for the
game of Blackjack is presented in chapter 5. Finally, in chapter 6, we reach our initial objective by
building the algorithm that hides behind the Deep Q-Networks and we apply it to the Gridworld
games in chapter 7. A conclusions and improvements section for the project culminates the text.El siguiente proyecto tiene como objetivo revisar los principales conceptos del Aprendizaje
con Refuerzo y combinarlo con las herramientas del Aprendizaje Profundo, estudiando con
detalle la aplicación de estas metodologías, Aprendizaje con Refuerzo Profundo, que están
teniendo tanto impacto en la actualidad siendo aplicados a numerosos campos como la conducción
autónoma, el control de robots, juegos y muchos más. Para ello, en primer lugar, en el capítulo 1,
situaremos al Aprendizaje con Refuerzo Profundo a modo de introducción, motivando el estudio
de este campo. Acto seguido, en el capítulo 2, ya que será fundamental para lograr nuestro
objetivo, se realiza una breve revisión del Aprendizaje Profundo. Entraremos en detalles con
el capítulo 3, donde definiremos matemáticamente que se entiende Aprendizaje con Refuerzo,
formalizando los conceptos con el fin de construir los algoritmos de solución clásicos en el capítulo
4. Como aplicación de estas técnicas, en el capítulo 5 se presenta la implementación de los
algoritmos para el juego del Blackjack. Finalmente, en el capítulo 5, alcanzaremos nuestro
objetivo inicial construyendo el algoritmo detrás de las Deep Q-Networks y lo aplicamos a los
juegos Gridworld en capítulo 7. Una sección de conclusiones y mejoras para el proyecto culmina
el texto.Universidad de Sevilla. Grado en Matemáticas y Estadístic
Spatial representation for planning and executing robot behaviors in complex environments
Robots are already improving our well-being and productivity in
different applications such as industry, health-care and indoor
service applications. However, we are still far from developing (and
releasing) a fully functional robotic agent that can autonomously
survive in tasks that require human-level
cognitive capabilities. Robotic systems on the market, in fact, are
designed to address specific applications, and can only run
pre-defined behaviors to robustly repeat few tasks (e.g., assembling
objects parts, vacuum cleaning). They internal representation of the
world is usually constrained to the task they are performing, and
does not allows for generalization to other
scenarios. Unfortunately, such a paradigm only apply to a very
limited set of domains, where the environment can be assumed to be
static, and its dynamics can be handled before
deployment. Additionally, robots configured in this way will
eventually fail if their "handcrafted'' representation of the
environment does not match the external world.
Hence, to enable more sophisticated cognitive skills, we investigate
how to design robots to properly represent the environment and
behave accordingly. To this end, we formalize a representation of
the environment that enhances the robot spatial knowledge to
explicitly include a representation of its own actions. Spatial
knowledge constitutes the core of the robot understanding of the
environment, however it is not sufficient to represent what the
robot is capable to do in it. To overcome such a limitation, we
formalize SK4R, a spatial knowledge representation for robots which
enhances spatial knowledge with a novel and "functional"
point of view that explicitly models robot actions. To this end, we
exploit the concept of affordances, introduced to express
opportunities (actions) that objects offer to an agent. To encode
affordances within SK4R, we define the "affordance
semantics" of actions that is used to annotate an environment, and
to represent to which extent robot actions support goal-oriented
behaviors.
We demonstrate the benefits of a functional representation of the
environment in multiple robotic scenarios that traverse and
contribute different research topics relating to: robot knowledge
representations, social robotics, multi-robot systems and robot
learning and planning. We show how a domain-specific representation,
that explicitly encodes affordance semantics, provides the robot
with a more concrete understanding of the environment and of the
effects that its actions have on it. The goal of our work is to
design an agent that will no longer execute an action, because of
mere pre-defined routine, rather, it will execute an actions because
it "knows'' that the resulting state leads one step closer to
success in its task
- …