Strategy Synthesis for Autonomous Agents Using PRISM

We present probabilistic models for autonomous agent search and retrieve missions derived from Simulink models for an Unmanned Aerial Vehicle (UAV) and show how probabilistic model checking and the probabilistic model checker PRISM can be used for optimal controller generation. We introduce a sequence of scenarios relevant to UAVs and other autonomous agents such as underwater and ground vehicles. For each scenario we demonstrate how it can be modelled using the PRISM language, give model checking statistics and present the synthesised optimal controllers. We conclude with a discussion of the limitations when using probabilistic model checking and PRISM in this context and what steps can be taken to overcome them. In addition, we consider how the controllers can be returned to the UAV and adapted for use on larger search areas

Real-time tree search with pessimistic scenarios

Autonomous agents need to make decisions in a sequential manner, under partially observable environment, and in consideration of how other agents behave. In critical situations, such decisions need to be made in real time for example to avoid collisions and recover to safe conditions. We propose a technique of tree search where a deterministic and pessimistic scenario is used after a specified depth. Because there is no branching with the deterministic scenario, the proposed technique allows us to take into account the events that can occur far ahead in the future. The effectiveness of the proposed technique is demonstrated in Pommerman, a multi-agent environment used in a NeurIPS 2018 competition, where the agents that implement the proposed technique have won the first and third places.Comment: 14 pages, 3 figures, Published as IBM Research Report RT098

Multimodal Interactive Learning of Primitive Actions

We describe an ongoing project in learning to perform primitive actions from demonstrations using an interactive interface. In our previous work, we have used demonstrations captured from humans performing actions as training samples for a neural network-based trajectory model of actions to be performed by a computational agent in novel setups. We found that our original framework had some limitations that we hope to overcome by incorporating communication between the human and the computational agent, using the interaction between them to fine-tune the model learned by the machine. We propose a framework that uses multimodal human-computer interaction to teach action concepts to machines, making use of both live demonstration and communication through natural language, as two distinct teaching modalities, while requiring few training samples.Comment: Presented at AI-HRI AAAI-FSS, 2018 (arXiv:1809.06606

Optimal Search for a Moving Target with the Option to Wait

We investigate the problem in which an agent has to find an object that moves between two locations according to a discrete Markov process (see Pollock, 1970). At every period, the agent has three options: searching left, searching right, and waiting. We assume that waiting is costless whereas searching is costly. Waiting can be useful because it could induce a more favorable probability distribution over the two locations next period. We find an essentially unique (nearly) optimal strategy, and prove that it is characterized by two thresholds (as conjectured by Weber, 1986). We show, moreover, that it can never be optimal to search the location with the lower probability of containing the object. The latter result is far from obvious and is in clear contrast with the example in Ross (1983) for the model without waiting.We also analyze the case of multiple agents. This makes the problem a more strategic one, since now the agents not only compete against time but also against each other in finding the object. We find different kinds of subgame perfect equilibria, possibly containing strategies that are not optimal in the one agent case. We compare the various equilibria in terms of cost-effectiveness.Strategy;

Situational Grounding within Multimodal Simulations

In this paper, we argue that simulation platforms enable a novel type of embodied spatial reasoning, one facilitated by a formal model of object and event semantics that renders the continuous quantitative search space of an open-world, real-time environment tractable. We provide examples for how a semantically-informed AI system can exploit the precise, numerical information provided by a game engine to perform qualitative reasoning about objects and events, facilitate learning novel concepts from data, and communicate with a human to improve its models and demonstrate its understanding. We argue that simulation environments, and game engines in particular, bring together many different notions of "simulation" and many different technologies to provide a highly-effective platform for developing both AI systems and tools to experiment in both machine and human intelligence.Comment: AAAI-19 Workshop on Games and Simulations for Artificial Intelligenc

Exploiting Heterogeneous Robotic Systems in Cooperative Missions

In this paper we consider the problem of coordinating robotic systems with different kinematics, sensing and vision capabilities to achieve certain mission goals. An approach that makes use of a heterogeneous team of agents has several advantages when cost, integration of capabilities, or large search areas need to be considered. A heterogeneous team allows for the robots to become "specialized", accomplish sub-goals more effectively, and thus increase the overall mission efficiency. Two main scenarios are considered in this work. In the first case study we exploit mobility to implement a power control algorithm that increases the Signal to Interference plus Noise Ratio (SINR) among certain members of the network. We create realistic sensing fields and manipulation by using the geometric properties of the sensor field-of-view and the manipulability metric, respectively. The control strategy for each agent of the heterogeneous system is governed by an artificial physics law that considers the different kinematics of the agents and the environment, in a decentralized fashion. Through simulation results we show that the network is able to stay connected at all times and covers the environment well. The second scenario studied in this paper is the biologically-inspired coordination of heterogeneous physical robotic systems. A team of ground rovers, designed to emulate desert seed-harvester ants, explore an experimental area using behaviors fine-tuned in simulation by a genetic algorithm. Our robots coordinate with a base station and collect clusters of resources scattered within the experimental space. We demonstrate experimentally that through coordination with an aerial vehicle, our ant-like ground robots are able to collect resources two times faster than without the use of heterogeneous coordination

Knowledge-Enabled Robotic Agents for Shelf Replenishment in Cluttered Retail Environments

Autonomous robots in unstructured and dynamically changing retail environments have to master complex perception, knowledgeprocessing, and manipulation tasks. To enable them to act competently, we propose a framework based on three core components: (o) a knowledge-enabled perception system, capable of combining diverse information sources to cope with occlusions and stacked objects with a variety of textures and shapes, (o) knowledge processing methods produce strategies for tidying up supermarket racks, and (o) the necessary manipulation skills in confined spaces to arrange objects in semi-accessible rack shelves. We demonstrate our framework in an simulated environment as well as on a real shopping rack using a PR2 robot. Typical supermarket products are detected and rearranged in the retail rack, tidying up what was found to be misplaced items.Comment: published in the proceedings of AAMAS 2016 as an extended abstrac

Optimal Distributed Searching in the Plane with and without Uncertainty

We consider the problem of multiple agents or robots searching for a target in the plane. This is motivated by Search and Rescue operations (SAR) in the high seas which in the past were often performed with several vessels, and more recently by swarms of aerial drones and/or unmanned surface vessels. Coordinating such a search in an effective manner is a non trivial task. In this paper, we develop first an optimal strategy for searching with k robots starting from a common origin and moving at unit speed. We then apply the results from this model to more realistic scenarios such as differential search speeds, late arrival times to the search effort and low probability of detection under poor visibility conditions. We show that, surprisingly, the theoretical idealized model still governs the search with certain suitable minor adaptations

Search in the Universe of Big Networks and Data

Searching in the Internet for some object characterised by its attributes in the form of data, such as a hotel in a certain city whose price is less than something, is one of our most common activities when we access the Web. We discuss this problem in a general setting, and compute the average amount of time and the energy it takes to find an object in an infinitely large search space. We consider the use of N search agents which act concurrently. Both the case where the search agent knows which way it needs to go to find the object, and the case where the search agent is perfectly ignorant and may even head away from the object being sought. We show that under mild conditions regarding the randomness of the search and the use of a time-out, the search agent will always find the object despite the fact that the search space is infinite. We obtain a formula for the average search time and the average energy expended by N search agents acting concurrently and independently of each other. We see that the time-out itself can be used to minimise the search time and the amount of energy that is consumed to find an object. An approximate formula is derived for the number of search agents that can help us guarantee that an object is found in a given time, and we discuss how the competition between search agents and other agents that try to hide the data object, can be used by opposing parties to guarantee their own success.Comment: IEEE Network Magazine - Special Issue on Networking for Big Data, July-August 201

Generalized Regressive Motion: a Visual Cue to Collision

Brains and sensory systems evolved to guide motion. Central to this task is controlling the approach to stationary obstacles and detecting moving organisms. Looming has been proposed as the main monocular visual cue for detecting the approach of other animals and avoiding collisions with stationary obstacles. Elegant neural mechanisms for looming detection have been found in the brain of insects and vertebrates. However, looming has not been analyzed in the context of collisions between two moving animals. We propose an alternative strategy, Generalized Regressive Motion (GRM), which is consistent with recently observed behavior in fruit flies. Geometric analysis proves that GRM is a reliable cue to collision among conspecifics, whereas agent-based modeling suggests that GRM is a better cue than looming as a means to detect approach, prevent collisions and maintain mobility