Exploring an Infinite Space with Finite Memory Scouts

Consider a small number of scouts exploring the infinite $d$-dimensional grid with the aim of hitting a hidden target point. Each scout is controlled by a probabilistic finite automaton that determines its movement (to a neighboring grid point) based on its current state. The scouts, that operate under a fully synchronous schedule, communicate with each other (in a way that affects their respective states) when they share the same grid point and operate independently otherwise. Our main research question is: How many scouts are required to guarantee that the target admits a finite mean hitting time? Recently, it was shown that $d + 1$ is an upper bound on the answer to this question for any dimension $d \geq 1$ and the main contribution of this paper comes in the form of proving that this bound is tight for $d \in \{ 1, 2 \}$.Comment: Added (forgotten) acknowledgement

Building a Nest by an Automaton

A robot modeled as a deterministic finite automaton has to build a structure from material available to it. The robot navigates in the infinite oriented grid Z x Z. Some cells of the grid are full (contain a brick) and others are empty. The subgraph of the grid induced by full cells, called the field, is initially connected. The (Manhattan) distance between the farthest cells of the field is called its span. The robot starts at a full cell. It can carry at most one brick at a time. At each step it can pick a brick from a full cell, move to an adjacent cell and drop a brick at an empty cell. The aim of the robot is to construct the most compact possible structure composed of all bricks, i.e., a nest. That is, the robot has to move all bricks in such a way that the span of the resulting field be the smallest. Our main result is the design of a deterministic finite automaton that accomplishes this task and subsequently stops, for every initially connected field, in time O(sz), where s is the span of the initial field and z is the number of bricks. We show that this complexity is optimal

Exploration of High-Dimensional Grids by Finite Automata

We consider the problem of finding a treasure at an unknown point of an n-dimensional infinite grid, n >= 3, by initially collocated finite automaton agents (scouts/robots). Recently, the problem has been well characterized for 2 dimensions for deterministic as well as randomized agents, both in synchronous and semi-synchronous models [S. Brandt et al., 2018; Y. Emek et al., 2015]. It has been conjectured that n+1 randomized agents are necessary to solve this problem in the n-dimensional grid [L. Cohen et al., 2017]. In this paper we disprove the conjecture in a strong sense: we show that three randomized synchronous agents suffice to explore an n-dimensional grid for any n. Our algorithm is optimal in terms of the number of the agents. Our key insight is that a constant number of finite automaton agents can, by their positions and movements, implement a stack, which can store the path being explored. We also show how to implement our algorithm using: four randomized semi-synchronous agents; four deterministic synchronous agents; or five deterministic semi-synchronous agents. We give a different algorithm that uses 4 deterministic semi-synchronous agents for the 3-dimensional grid. This is provably optimal, and surprisingly, matches the result for 2 dimensions. For n >= 4, the time complexity of the solutions mentioned above is exponential in distance D of the treasure from the starting point of the agents. We show that in the deterministic case, one additional agent brings the time down to a polynomial. Finally, we focus on algorithms that never venture much beyond the distance D. We describe an algorithm that uses O(sqrt{n}) semi-synchronous deterministic agents that never go beyond 2D, as well as show that any algorithm using 3 synchronous deterministic agents in 3 dimensions, if it exists, must travel beyond Omega(D^{3/2}) from the origin

Towards Informed Exploration for Deep Reinforcement Learning

In this thesis, we discuss various techniques for improving exploration for deep reinforcement learning. We begin with a brief review of reinforcement learning (RL) and the fundamental v.s. exploitation trade-off. Then we review how deep RL has improved upon classical and summarize six categories of the latest exploration methods for deep RL, in the order increasing usage of prior information. We then explore representative works in three categories discuss their strengths and weaknesses. The first category, represented by Soft Q-learning, uses regularization to encourage exploration. The second category, represented by count-based via hashing, maps states to hash codes for counting and assigns higher exploration to less-encountered states. The third category utilizes hierarchy and is represented by modular architecture for RL agents to play StarCraft II. Finally, we conclude that exploration by prior knowledge is a promising research direction and suggest topics of potentially impact

Multi‑Agent Foraging: state‑of‑the‑art and research challenges

International audienceThe foraging task is one of the canonical testbeds for cooperative robotics, in which a collection of robots has to search and transport objects to specific storage point(s). In this paper, we investigate the Multi-Agent Foraging (MAF) problem from several perspectives that we analyze in depth. First, we define the Foraging Problem according to literature definitions. Then we analyze previously proposed taxonomies, and propose a new foraging taxonomy characterized by four principal axes: Environment, Collective, Strategy and Simulation, summarize related foraging works and classify them through our new foraging taxonomy. Then, we discuss the real implementation of MAF and present a comparison between some related foraging works considering important features that show extensibility, reliability and scalability of MAF systems. Finally we present and discuss recent trends in this field, emphasizing the various challenges that could enhance the existing MAF solutions and make them realistic

Constant Space and Non-Constant Time in Distributed Computing

While the relationship of time and space is an established topic in traditional centralised com- plexity theory, this is not the case in distributed computing. We aim to remedy this by studying the time and space complexity of algorithms in a weak message-passing model of distributed com- puting. While a constant number of communication rounds implies a constant number of states visited during the execution, the other direction is not clear at all. We show that indeed, there exist non-trivial graph problems that are solvable by constant-space algorithms but that require a non-constant running time. Somewhat surprisingly, this holds even when restricted to the class of only cycle and path graphs. Our work provides us with a new complexity class for distributed computing and raises interesting questions about the existence of further combinations of time and space complexity

Algoritmo bioinspirado a redes de robots para la asistencia en operaciones de busqueda y rescate

ilustraciones, diagramas, fotografíasThis thesis proposes a bio-inspired algorithm for robot networks assisting in the operations of search and rescue scenarios. We consider ants as social animals to study and abstract beha- viors that can be useful in the framework of search and rescue using robots. We consider three main topics to address when using robots to assist rescuers. First, the exploration and mapping of the disaster zones. For this, we consider the mecha- nisms and interactions of ants to explore their environment, look for food, avoid predators, and explore better places to establish a nest. Then, we deploy robots to explore the en- vironment and discourage robots from entering regions other robots have explored using pheromones as markers for the robots. We also abstract the randomness ants use to explore and implement a Q-learning algorithm that allows robots to explore unvisited regions. Second, the navigation and victim detection. Once the environment has been explored, we vi use Reynolds rules to allow the navigation of robots to create cohesion, attraction to target goals, and repulsion to obstacles and inter-agent collisions. Then, we use a neural network to determine whether what robots are detecting is a victim. Lastly, we use a consensus-like approach to classify victims or no victims based on distributed information. Lastly, ants have been famous for carrying loads that surpass their size and payload capacity by cooperating. We consider quadrotors to carry loads cooperatively that can be medical supplies or victims in search and rescue (Texto tomado de la fuente)Esta tesis propone un algoritmo bioinspirado para redes de robots que asisten en las operaciones de escenarios de busqueda y rescate. Consideramos a las hormigas como animales sociales para estudiar y abstraer comportamientos que pueden ser utiles en el marco de la busqueda y rescate mediante robots. Consideramos tres temas principales para abordar cuando se utilizan robots para ayudar a los rescatistas. Primero, la exploracion y mapeo de las zonas de desastre. Para esto, consideramos los mecanismos e interacciones de las hormigas para explorar su entorno, buscar comida, evitar depredadores y explorar mejores lugares para establecer un nido. Luego, desplegamos robots para explorar el entorno y disuadimos a los robots de ingresar a regiones que otros robots han explorado usando feromonas como marcadores para los robots. Tambien abstraemos la aleatoriedad que usan las hormigas para explorar e implementar un algoritmo Q-learning que permite a los robots explorar regiones no visitadas. En segundo lugar, la navegacion y deteccion de vıctimas. Una vez que se ha explorado el entorno, usamos las reglas de Reynolds para permitir que la navegacion de los robots cree cohesion, atraccion hacia los objetivos y repulsion hacia los obstaculos y las colisiones entre agentes. Luego, usamos una red neuronal para determinar si lo que detectan los robots es una vıctima. Por ultimo, utilizamos un enfoque de consenso para clasificar a las vıctimas o no vıctimas en funcion de la informacion distribuida. Por ultimo, las hormigas han sido famosas por llevar cargas que superan su tamano y capacidad de carga al cooperar. Consideramos quadrotors para transportar cargas de manera cooperativa que pueden ser suministros medicos o vıctimas en busqueda y rescate.MaestríaMagister en Ingenieria - Automatizacion IndustrialRobotic