12,219 research outputs found

    Cooperative Control and Fault Recovery for Network of Heterogeneous Autonomous Underwater Vehicles

    Get PDF
    The purpose of this thesis is to develop cooperative recovery control schemes for a team of heterogeneous autonomous underwater vehicles (AUV). The objective is to have the network of autonomous underwater vehicles follow a desired trajectory while agents maintain a desired formation. It is assumed that the model parameters associated with each vehicle is different although the order of the vehicles are the same. Three cooperative control schemes based on dynamic surface control (DSC) technique are developed. First, a DSC-based centralized scheme is presented in which there is a central controller that has access to information of all agents at the same time and designs the optimal solution for this cooperative problem. This scheme is used as a benchmark to evaluate the performance of other schemes developed in this thesis. Second, a DSC-based decentralized scheme is presented in which each agent designs its controller based on only its information and the information of its desired trajectory. In this scheme, there is no information exchange among the agents in the team. This scheme is also developed for the purpose of comparative studies. Third, two different semi-decentralized or distributed schemes for the network of heterogeneous autonomous underwater vehicles are proposed. These schemes are a synthesis of a consensus-based algorithm and the dynamic surface control technique with the difference that in one of them the desired trajectories of agents are used in the consensus algorithm while in the other the actual states of the agents are used. In the former scheme, the agents communicate their desired relative distances with the agents within their set of nearest neighbors and each agent determines its own control trajectory. In this semi-decentralized scheme, the velocity measurements of the virtual leader and all the followers are not required to reach the consensus formation. However, in the latter, agents communicate their relative distances and velocities with the agents within their set of nearest neighbors. In both semi-decentralized schemes only a subset of agents has access to information of a virtual leader. The comparative studies between these two semi-decentralized schemes are provided which show the superiority of the former semi-decentralized scheme over latter. Furthermore, to evaluate the efficiency of the proposed DSC-based semi-decentralized scheme with consensus algorithm using desired trajectories, a comparative study is performed between this scheme and three cooperative schemes of model-dependent coordinated tracking algorithm, namely the centralized, decentralized, and semi-decentralized schemes. Given that the dynamics of autonomous underwater vehicles are inevitably subjected to system faults, and in particular the actuator faults, to improve the performance of the network of agents, active fault-tolerant control strategies corresponding to the three developed schemes are also designed to recover the team from the loss-of-effectiveness in the actuators and to ensure that the closed-loop signals remain bounded and the team of heterogeneous autonomous underwater vehicles satisfy the overall design specifications and requirements. The results of this research can potentially be used in various marine applications such as underwater oil and gas pipeline inspection and repairing, monitoring oil and gas pipelines, detecting and preventing any oil and gas leakages. However, the applications of the proposed cooperative control and its fault-tolerant scheme are not limited to underwater formation path-tracking and can be applied to any other multi-vehicle systems that are characterized by Euler–Lagrange equations

    Decentralized Control of Partially Observable Markov Decision Processes using Belief Space Macro-actions

    Get PDF
    The focus of this paper is on solving multi-robot planning problems in continuous spaces with partial observability. Decentralized partially observable Markov decision processes (Dec-POMDPs) are general models for multi-robot coordination problems, but representing and solving Dec-POMDPs is often intractable for large problems. To allow for a high-level representation that is natural for multi-robot problems and scalable to large discrete and continuous problems, this paper extends the Dec-POMDP model to the decentralized partially observable semi-Markov decision process (Dec-POSMDP). The Dec-POSMDP formulation allows asynchronous decision-making by the robots, which is crucial in multi-robot domains. We also present an algorithm for solving this Dec-POSMDP which is much more scalable than previous methods since it can incorporate closed-loop belief space macro-actions in planning. These macro-actions are automatically constructed to produce robust solutions. The proposed method's performance is evaluated on a complex multi-robot package delivery problem under uncertainty, showing that our approach can naturally represent multi-robot problems and provide high-quality solutions for large-scale problems

    Learning for Multi-robot Cooperation in Partially Observable Stochastic Environments with Macro-actions

    Get PDF
    This paper presents a data-driven approach for multi-robot coordination in partially-observable domains based on Decentralized Partially Observable Markov Decision Processes (Dec-POMDPs) and macro-actions (MAs). Dec-POMDPs provide a general framework for cooperative sequential decision making under uncertainty and MAs allow temporally extended and asynchronous action execution. To date, most methods assume the underlying Dec-POMDP model is known a priori or a full simulator is available during planning time. Previous methods which aim to address these issues suffer from local optimality and sensitivity to initial conditions. Additionally, few hardware demonstrations involving a large team of heterogeneous robots and with long planning horizons exist. This work addresses these gaps by proposing an iterative sampling based Expectation-Maximization algorithm (iSEM) to learn polices using only trajectory data containing observations, MAs, and rewards. Our experiments show the algorithm is able to achieve better solution quality than the state-of-the-art learning-based methods. We implement two variants of multi-robot Search and Rescue (SAR) domains (with and without obstacles) on hardware to demonstrate the learned policies can effectively control a team of distributed robots to cooperate in a partially observable stochastic environment.Comment: Accepted to the 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2017

    Planning for Decentralized Control of Multiple Robots Under Uncertainty

    Full text link
    We describe a probabilistic framework for synthesizing control policies for general multi-robot systems, given environment and sensor models and a cost function. Decentralized, partially observable Markov decision processes (Dec-POMDPs) are a general model of decision processes where a team of agents must cooperate to optimize some objective (specified by a shared reward or cost function) in the presence of uncertainty, but where communication limitations mean that the agents cannot share their state, so execution must proceed in a decentralized fashion. While Dec-POMDPs are typically intractable to solve for real-world problems, recent research on the use of macro-actions in Dec-POMDPs has significantly increased the size of problem that can be practically solved as a Dec-POMDP. We describe this general model, and show how, in contrast to most existing methods that are specialized to a particular problem class, it can synthesize control policies that use whatever opportunities for coordination are present in the problem, while balancing off uncertainty in outcomes, sensor information, and information about other agents. We use three variations on a warehouse task to show that a single planner of this type can generate cooperative behavior using task allocation, direct communication, and signaling, as appropriate

    Coordinated Multi-Agent Imitation Learning

    Get PDF
    We study the problem of imitation learning from demonstrations of multiple coordinating agents. One key challenge in this setting is that learning a good model of coordination can be difficult, since coordination is often implicit in the demonstrations and must be inferred as a latent variable. We propose a joint approach that simultaneously learns a latent coordination model along with the individual policies. In particular, our method integrates unsupervised structure learning with conventional imitation learning. We illustrate the power of our approach on a difficult problem of learning multiple policies for fine-grained behavior modeling in team sports, where different players occupy different roles in the coordinated team strategy. We show that having a coordination model to infer the roles of players yields substantially improved imitation loss compared to conventional baselines.Comment: International Conference on Machine Learning 201
    • …
    corecore