Modeling and stochastic optimization of complete coverage under uncertainties in multi-robot base placements

© 2016 IEEE. Uncertainties in base placements of mobile, autonomous industrial robots can cause incomplete coverage in tasks such as grit-blasting and spray painting. Sensing and localization errors can cause such uncertainties in robot base placements. This paper addresses the problem of collaborative complete coverage under uncertainties through appropriate base placements of multiple mobile and autonomous industrial robots while aiming to optimize the performance of the robot team. A mathematical model for complete coverage under uncertainties is proposed and then solved using a stochastic multi-objective optimization algorithm. The approach aims to concurrently find an optimal number and sequence of base placements for each robot such that the robot team's objectives are optimized whilst uncertainties are accounted for. Several case studies based on a real-world application using a realworld object and a complex simulated object are provided to demonstrate the effectiveness of the approach for different conditions and scenarios, e.g. various levels of uncertainties, different numbers of robots, and robots with different capabilities

An Approach to Base Placement for Effective Collaboration of Multiple Autonomous Industrial Robots

There are many benefits for the deployment of multiple autonomous industrial robots to carry out a task, particularly if the robots act in a highly collaborative manner. Collaboration can be possible when each robot is able to autonomously explore the environment, localize itself, create a map of the environment and communicate with other robots. This paper presents an approach to the modeling of the collaboration problem of multiple robots determining optimal base positions and orientations in an environment by considering the team objectives and the information shared amongst the robots. It is assumed that the robots can communicate so as to share information on the environment, their operation status and their capabilities. The approach has been applied to a team of robots that are required to perform complete surface coverage tasks such as grit-blasting and spray painting in unstructured environments. Case studies of such applications are presented to demonstrate the effectiveness of the approach

A Two-Stage Approach to Collaborative Fiber Placement through Coordination of Multiple Autonomous Industrial Robots

© 2018, Springer Nature B.V. The use of multiple Autonomous Industrial Robots (AIRs) as opposed to a single AIR to perform fiber placement brings about many challenges which have not been addressed by researchers. These challenges include optimal division and allocation of the work and performing path planning in a coordinated manner while considering the requirements and constraints that are unique to the fiber placement task. To solve these challenges, a two-stage approach is proposed in this paper. The first stage considers multiple objectives to optimally allocate each AIR with surface areas, while the second stage aims to generate coordinated paths for the AIRs. Within each stage, mathematical models are developed with several unique objectives and constraints that are specific to the multi-AIR collaborative fiber placement. Several case studies are presented to validate the approach and the proposed mathematical models. Comparison studies with different number of AIRs and variations of the developed mathematical models are also presented

Multi-robot cooperation

Abstract. This bachelor’s thesis familiarizes with multi-robot cooperation. The main interest is in two robot manipulators. This thesis is a literature review. The operation of the robot and the phenomena that act on them while in operation are investigated from kinematics and command architecture point of view. This thesis is based on manuals from two KUKA robots from University of Oulu, so in the future the use and understanding of their cooperation would be easier. The results gave good understanding of robot software calculations for trajectories and geometrics and what other has to consider when controlling a multi-robot system. This is a good base for deeper theoretical research for robot system software and practical testing.Usean robotin yhteiskäyttö. Tiivistelmä. Tässä opinnäytetyössä perehdytään usean robotin yhteiskäyttöön, jossa mielenkiinnon kohteena on kahden robottikäden yhteistoiminta. Työ on kirjallisuuskatsaus. Robottien toimintaa ja niihin vaikuttavia asioita tarkastellaan niin kinematiikan, kuin ohjelmisto- ja käskyarkkitehtuurin kautta. Työn pohjana käytettiin yliopistolla olevien KUKA robottikäsien oppaita, jotta jatkossa niiden yhteiskäyttö olisi helpommin ymmärrettävissä. Työn tulokset avasivat sitä, miten robottien ohjelmisto ohjaa ja laskee tarvittavat liikeradat ja geometriat ja mitä kaikkea usean robotin ohjauksessa pitää ottaa huomioon. Tämä on hyvä pohja syvemmälle teoreettiselle robottiohjelmistolle tai käytännön testaamiselle

Combining task and motion planning for mobile manipulators

Aplicat embargament des de la data de defensa fins el dia 31/12/2019Premi Extraordinari de Doctorat, promoció 2018-2019. Àmbit d’Enginyeria IndustrialThis thesis addresses the combination of task and motion planning which deals with different types of robotic manipulation problems. Manipulation problems are referred to as mobile manipulation, collaborative multiple mobile robots tasks, and even higher dimensional tasks (like bi-manual robots or mobile manipulators). Task and motion planning problems needs to obtain a geometrically feasible manipulation plan through symbolic and geometric search space. The combination of task and motion planning levels has emerged as a challenging issue as the failure leads robots to dead-end tasks due to geometric constraints. In addition, task planning is combined with physics-based motion planning and information to cope with manipulation tasks in which interactions between robots and objects are required, or also a low-cost feasible plan in terms of power is looked for. Moreover, combining task and motion planning frameworks is enriched by introducing manipulation knowledge. It facilitates the planning process and aids to provide the way of executing symbolic actions. Combining task and motion planning can be considered under uncertain information and with human-interaction. Uncertainty can be viewed in the initial state of the robot world or the result of symbolic actions. To deal with such issues, contingent-based task and motion planning is proposed using a perception system and human knowledge. Also, robots can ask human for those tasks which are difficult or infeasible for the purpose of collaboration. An implementation framework to combine different types of task and motion planning is presented. All the required modules and tools are also illustrated. As some task planning algorithms are implemented in Prolog or C++ languages and our geometric reasoner is developed in C++, the flow of information between different languages is explained.Aquesta tesis es centra en les eines de planificació combinada a nivell de tasca i a nivell de moviments per abordar diferents problemes de manipulació robòtica. Els problemes considerats són de navegació de robots mòbil enmig de obstacles no fixes, tasques de manipulació cooperativa entre varis robots mòbils, i tasques de manipulació de dimensió més elevada com les portades a terme amb robots bi-braç o manipuladors mòbils. La planificació combinada de tasques i de moviments ha de cercar un pla de manipulació que sigui geomètricament realitzable, a través de d'un espai de cerca simbòlic i geomètric. La combinació dels nivells de planificació de tasca i de moviments ha sorgit com un repte ja que les fallades degudes a les restriccions geomètriques poden portar a tasques sense solució. Addicionalment, la planificació a nivell de tasca es combina amb informació de la física de l'entorn i amb mètodes de planificació basats en la física, per abordar tasques de manipulació en les que la interacció entre el robot i els objectes és necessària, o també si es busca un pla realitzable i amb un baix cost en termes de potència. A més, el marc proposat per al combinació de la planificació a nivell de tasca i a nivell de moviments es millora mitjançant l'ús de coneixement, que facilita el procés de planificació i ajuda a trobar la forma d'executar accions simbòliques. La combinació de nivells de planificació també es pot considerar en casos d'informació incompleta i en la interacció humà-robot. La incertesa es considera en l'estat inicial i en el resultat de les accions simbòliques. Per abordar aquest problema, es proposa la planificació basada en contingències usant un sistema de percepció i el coneixement de l'operari humà. Igualment, els robots poden demanar col·laboració a l'operari humà per a que realitzi aquelles accions que són difícils o no realitzables pel robot. Es presenta també un marc d'implementació per a la combinació de nivells de planificació usant diferents mètodes, incloent tots els mòduls i eines necessàries. Com que alguns algorismes estan implementats en Prolog i d'altres en C++, i el mòdul de raonament geomètric proposat està desenvolupat en C++, es detalla el flux d'informació entre diferents llenguatges.Award-winningPostprint (published version

FABRIC: A Framework for the Design and Evaluation of Collaborative Robots with Extended Human Adaptation

A limitation for collaborative robots (cobots) is their lack of ability to adapt to human partners, who typically exhibit an immense diversity of behaviors. We present an autonomous framework as a cobot's real-time decision-making mechanism to anticipate a variety of human characteristics and behaviors, including human errors, toward a personalized collaboration. Our framework handles such behaviors in two levels: 1) short-term human behaviors are adapted through our novel Anticipatory Partially Observable Markov Decision Process (A-POMDP) models, covering a human's changing intent (motivation), availability, and capability; 2) long-term changing human characteristics are adapted by our novel Adaptive Bayesian Policy Selection (ABPS) mechanism that selects a short-term decision model, e.g., an A-POMDP, according to an estimate of a human's workplace characteristics, such as her expertise and collaboration preferences. To design and evaluate our framework over a diversity of human behaviors, we propose a pipeline where we first train and rigorously test the framework in simulation over novel human models. Then, we deploy and evaluate it on our novel physical experiment setup that induces cognitive load on humans to observe their dynamic behaviors, including their mistakes, and their changing characteristics such as their expertise. We conduct user studies and show that our framework effectively collaborates non-stop for hours and adapts to various changing human behaviors and characteristics in real-time. That increases the efficiency and naturalness of the collaboration with a higher perceived collaboration, positive teammate traits, and human trust. We believe that such an extended human adaptation is key to the long-term use of cobots.Comment: The article is in review for publication in International Journal of Robotics Researc

Analyzing Infrastructure LiDAR Placement with Realistic LiDAR Simulation Library

Recently, Vehicle-to-Everything(V2X) cooperative perception has attracted increasing attention. Infrastructure sensors play a critical role in this research field; however, how to find the optimal placement of infrastructure sensors is rarely studied. In this paper, we investigate the problem of infrastructure sensor placement and propose a pipeline that can efficiently and effectively find optimal installation positions for infrastructure sensors in a realistic simulated environment. To better simulate and evaluate LiDAR placement, we establish a Realistic LiDAR Simulation library that can simulate the unique characteristics of different popular LiDARs and produce high-fidelity LiDAR point clouds in the CARLA simulator. Through simulating point cloud data in different LiDAR placements, we can evaluate the perception accuracy of these placements using multiple detection models. Then, we analyze the correlation between the point cloud distribution and perception accuracy by calculating the density and uniformity of regions of interest. Experiments show that when using the same number and type of LiDAR, the placement scheme optimized by our proposed method improves the average precision by 15%, compared with the conventional placement scheme in the standard lane scene. We also analyze the correlation between perception performance in the region of interest and LiDAR point cloud distribution and validate that density and uniformity can be indicators of performance. Both the RLS Library and related code will be released at https://github.com/PJLab-ADG/LiDARSimLib-and-Placement-Evaluation.Comment: 7 pages, 6 figures, accepted to the IEEE International Conference on Robotics and Automation (ICRA'23

Monitoring using Heterogeneous Autonomous Agents.

This dissertation studies problems involving different types of autonomous agents observing objects of interests in an area. Three types of agents are considered: mobile agents, stationary agents, and marsupial agents, i.e., agents capable of deploying other agents or being deployed themselves. Objects can be mobile or stationary. The problem of a mobile agent without fuel constraints revisiting stationary objects is formulated. Visits to objects are dictated by revisit deadlines, i.e., the maximum time that can elapse between two visits to the same object. The problem is shown to be NP-complete and heuristics are provided to generate paths for the agent. Almost periodic paths are proven to exist. The efficacy of the heuristics is shown through simulation. A variant of the problem where the agent has a finite fuel capacity and purchases fuel is treated. Almost periodic solutions to this problem are also shown to exist and an algorithm to compute the minimal cost path is provided. A problem where mobile and stationary agents cooperate to track a mobile object is formulated, shown to be NP-hard, and a heuristic is given to compute paths for the mobile agents. Optimal configurations for the stationary agents are then studied. Several methods are provided to optimally place the stationary agents; these methods are the maximization of Fisher information, the minimization of the probability of misclassification, and the minimization of the penalty incurred by the placement. A method to compute optimal revisit deadlines for the stationary agents is given. The placement methods are compared and their effectiveness shown using numerical results. The problem of two marsupial agents, one carrier and one passenger, performing a general monitoring task using a constrained optimization formulation is stated. Necessary conditions for optimal paths are provided for cases accounting for constrained release of the passenger, termination conditions for the task, as well as retrieval and constrained retrieval of the passenger. A problem involving two marsupial agents collecting information about a stationary object while avoiding detection is then formulated. Necessary conditions for optimal paths are provided and rectilinear motion is demonstrated to be optimal for both agents.PhDAerospace EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/111439/1/jfargeas_1.pd