Integrated Task and Motion Planning of Multi-Robot Manipulators in Industrial and Service Automation

Efficient coordination of several robot arms in order to carry out some given independent/cooperative tasks in a common workspace, avoiding collisions, is an appealing research problem that has been studied in different robotic fields, with industrial and service applications. Coordination of several robot arms in a shared environment is challenging because complexity of collision free path planning increases with the number of robots sharing the same workspace. Although research in different aspects of this problem such as task planning, motion planning and robot control has made great progress, the integration of these components is not well studied in the literature. This thesis focuses on integrating task and motion planning multi-robot-arm systems by introducing a practical and optimal interface layer for such systems. For a given set of speci fications and a sequence of tasks for a multi-arm system, the studied system design aims to automatically construct the necessary waypoints, the sequence of arms to be operated, and the algorithms required for the robots to reliably execute manipulation tasks. The contributions of the thesis are three-fold. First, an algorithm is introduced to integrate task and motion planning layers in order to achieve optimal and collision free task execution. Representation via shared space graph (SSG) is introduced to check whether two arms share certain parts of the workspace and to quantify cooperation of such arm pairs, which is essential in selection of arm sequence and scheduling of each arm in the sequence to perform a task or a sub-task. The introduced algorithm allows robots to autonomously reason about a structured environment, performs the sequence planning of robots to operate, and provides robots and objects path for each task to succeed a set of goals. Secondly, an integrated motion and task planning methodology is introduced for systems of multiple mobile and fixed base robot arms performing different tasks simultaneously in a shared workspace. We introduce concept of dynamic shared space graph (D-SSG) to continuously check whether two arms sharing certain parts of the workspace at different time steps and quantify cooperation of such arm pairs, which is essential to the selection of arm sequences and scheduling of each arm in the sequence to perform a task or a sub-task. The introduced algorithm allows robots to autonomously reason about complex human involving environments to plan the high level decisions (sequence planning) of robots to operate and calculates robots and objects path for each task to succeed a set of goals. The third contribution is design of an integration algorithm between low-level motion planning and high-level symbolic task planning layers to produce alternate plans in case of kinematic and geometric changes in the environment to prevent failure in the high-level task plan. In order to verify the methodological contributions of the thesis with a solid implementation basis, some implementations and tests are presented in the open-source robotics planning environments ROS, Moveit and Gazebo. Detailed analysis of these implementations and test results are provided as well

Humanoid robot painter: Visual perception and high-level planning

Abstract -This paper presents visual perception discovered in high-level manipulator planning for a robot to reproduce the procedure involved in human painting. First, we apply a technique of 2D object segmentation that considers region similarity as an objective function and edge as a constraint with artificial intelligent used as a criterion function. The system can segment images more effectively than most of existing methods, even if the foreground is very similar to the background. Second, we propose a novel color perception model that shows similarity to human perception. The method outperforms many existing color reduction algorithms. Third, we propose a novel global orientation map perception using a radial basis function. Finally, we use the derived model along with the brush's position-and force-sensing to produce a visual feedback drawing. Experiments show that our system can generate good paintings including portraits

双腕ロボットによる料理作業実現のための食材の切断に関する研究～抽象操作記述と対象認識に基づく動的軌道生成～

本研究の目的は，双腕ロボットシステムを用いて人のように料理作業を行うことを題材とし，その作業の中でも基本的な操作のひとつである，食材の切断操作を実現することである．具体的には，食材の大まかな位置と，それを切断するという命令が与えられたことを前提として，対象の食材を実際に切断する為に必要な対象食材の位置姿勢の認識と，切断操作を実行するときの手先軌道の動的な生成を実現することである．認識する食材や切断操作には多くの種類があるが，本研究では特定の対象や操作を対象とした作り込みではなく，それらを作り込む際に利用できる一般化された再利用性の高いスキルとして操作を実現することを目指した. また，抽象的な作業記述を操作という単位に分割し，抽象的な操作の記述から具体的なロボットの動作軌跡を実環境に適応させて生成するための実装をスキルとして定義し，さらにスキルの内部の記述をスキル層，応動層，動作層に分けて構造化することで，抽象作業記述から動的にロボットの動作生成を行う過程を体系立てた．本研究では，食材の切断という課題の達成に必要なセンシングの要件，操作の要件を明らかにし，これに基づいた食材認識手法と切断操作の手法を提案し，これらの実行要件を満たすロボットシステムを構築し．提案手法の有効性を実験によって検証した．食材の認識では，ロボットの手先に装備したRGB-Dカメラを用いて複数の視点から得られた点群情報を統合することで食材全体の形状取得し，安定的に位置姿勢を検出する手法を提案した．食材の切断操作では，切断中に刃先に加わる反力のフィードバックを利用して動的に軌道生成を行い，食材の大きさや硬さに関する個体差や実行環境の違いを適応的に吸収する手法を提案した．これらの手法を，構築したロボットシステムに実装し，実際に食材の切断に成功した．電気通信大学201

Following recipes with a cooking robot

Thesis (S.M.)--Massachusetts Institute of Technology, Dept. of Mechanical Engineering, 2012.Cataloged from PDF version of thesis.Includes bibliographical references (p. 141-143).In this thesis, we present BakeBot, a PR2 robot system that interprets natural language baking recipes into baking instructions which it follows to execute the recipe, from mise en place presentation of the ingredients through baking in a toaster oven. We developed parameterized motion primitives for baking. The motion primitives utilize the existing sensing and manipulation capabilities of the PR2 platform and also our new compliant control techniques to address environmental uncertainty. The system was first implemented as a static finite state machine, which was tested through 27 baking attempts, 16 of which successfully resulted in edible cookies. The system was then implemented as a dynamic state machine, in which the robot estimated the world state and planned sequences of motion primitives to follow the baking instructions inferred from the natural language recipe, which was tested thorough five baking attempts of two different recipes, all of which resulted in edible cookies.by Mario Attilio Bollini.S.M

Adapting robot behavior to user preferences in assistive scenarios

Aplicat embargament des de la data de defensa fins el 24 de juliol de 2020Robotic assistants have inspired numerous books and science fiction movies. In the real world, these kinds of devices are a growing need in amongst the elderly, who while life continue requiring more assistance. While life expectancy is increasing, life quality is not necessarily doing so. Thus, we may find ourselves and our loved ones being dependent and needing another person to perform the most basic tasks, which has a strong psychological impact. Accordingly, assistive robots may be the definitive tool to give more quality of life by empowering dependent people and extending their independent living. Assisting users to perform daily activities requires adapting to them and their needs, as they might not be able to adapt to the robot. This thesis tackles adaptation and personalization issues through user preferences. We 'focus on physical tasks that involve close contact, as these present interesting challenges, and are of great importance for he user. Therefore, three tasks are mainly used throughout the thesis: assistive feeding, shoe fitting, and jacket dressing. We first describe a framework for robot behavior adaptation that illustrates how robots should be personalized for and by end- users or their assistants. Using this framework, non-technical users determine how !he robot should behave. Then, we define the concept of preference for assistive robotics scenarios and establish a taxonomy, which includes hierarchies and groups of preferences, grounding definitions and concepts. We then show how the preferences in the taxonomy are used with Al planning systems to adapt the robot behavior to the preferences of the user obtained from simple questions. Our algorithms allow for long-term adaptations as well as to cope with misinformed user models. We further integrate the methods with low-level motion primitives that provide a more robust adaptation and behavior while lowering the number of needed actions and demonstrations. Moreover, we perform a deeper analysis in Planning and preferences with the introduction of new algorithms to provide preference suggestions in planning domains. The thesis then concludes with a user study that evaluates the use of the preferences in the three real assistive robotics scenarios. The experiments show a clear understanding of the preferences of users, who were able to assess the impact of their preferences on the behavior of the robot. In summary, we provide tools and algorithms to design the robotic assistants of the future. Assistants that should be able to adapt to the assisted user needs and preferences, just as human assistants do nowadays.Els assistents robòtics han inspirat nombrosos llibres i pel·lícules de ciència-ficció al llarg de la història. Però tornant al món real, aquest tipus de dispositius s'estan tornant una necessitat per a una societat que envelleix a un ritme ràpid i que, per tant, requerirà més i més assistència. Mentre l'esperança de vida augmenta, la qualitat de vida no necessàriament ho fa. Per tant, ens podem trobar a nosaltres mateixos i als nostres estimats en una situació de dependència, necessitant una altra persona per poder fer les tasques més bàsiques, cosa que té un gran impacte psicològic. En conseqüència, els robots assistencials poden ser l'eina definitiva per proporcionar una millor qualitat de vida empoderant els usuaris i allargant la seva capacitat de viure independentment. L'assistència a persones per realitzar tasques diàries requereix adaptar-se a elles i les seves necessitats, donat que aquests usuaris no poden adaptar-se al robot. En aquesta tesi, abordem el problema de l'adaptació i la personalització d'un robot mitjançant preferències de l'usuari. Ens centrem en tasques físiques, que involucren contacte amb la persona, per les seves dificultats i importància per a l'usuari. Per aquest motiu, la tesi utilitzarà principalment tres tasques com a exemple: donar menjar, posar una sabata i vestir una jaqueta. Comencem definint un marc (framework) per a la personalització del comportament del robot que defineix com s'han de personalitzar els robots per usuaris i pels seus assistents. Amb aquest marc, usuaris sense coneixements tècnics són capaços de definir com s'ha de comportar el robot. Posteriorment definim el concepte de preferència per a robots assistencials i establim una taxonomia que inclou jerarquies i grups de preferències, els quals fonamenten les definicions i conceptes. Després mostrem com les preferències de la taxonomia s'utilitzen amb sistemes planificadors amb IA per adaptar el comportament del robot a les preferències de l'usuari, que s'obtenen mitjançant preguntes simples. Els nostres algorismes permeten l'adaptació a llarg termini, així com fer front a models d'usuari mal inferits. Aquests mètodes són integrats amb primitives a baix nivell que proporcionen una adaptació i comportament més robusts a la mateixa vegada que disminueixen el nombre d'accions i demostracions necessàries. També fem una anàlisi més profunda de l'ús de les preferències amb planificadors amb la introducció de nous algorismes per fer suggeriments de preferències en dominis de planificació. La tesi conclou amb un estudi amb usuaris que avalua l'ús de les preferències en les tres tasques assistencials. Els experiments demostren un clar enteniment de les preferències per part dels usuaris, que van ser capaços de discernir quan les seves preferències eren utilitzades. En resum, proporcionem eines i algorismes per dissenyar els assistents robòtics del futur. Uns assistents que haurien de ser capaços d'adaptar-se a les preferències i necessitats de l'usuari que assisteixen, tal com els assistents humans fan avui en dia.Postprint (published version

Adapting robot behavior to user preferences in assistive scenarios

Robotic assistants have inspired numerous books and science fiction movies. In the real world, these kinds of devices are a growing need in amongst the elderly, who while life continue requiring more assistance. While life expectancy is increasing, life quality is not necessarily doing so. Thus, we may find ourselves and our loved ones being dependent and needing another person to perform the most basic tasks, which has a strong psychological impact. Accordingly, assistive robots may be the definitive tool to give more quality of life by empowering dependent people and extending their independent living. Assisting users to perform daily activities requires adapting to them and their needs, as they might not be able to adapt to the robot. This thesis tackles adaptation and personalization issues through user preferences. We 'focus on physical tasks that involve close contact, as these present interesting challenges, and are of great importance for he user. Therefore, three tasks are mainly used throughout the thesis: assistive feeding, shoe fitting, and jacket dressing. We first describe a framework for robot behavior adaptation that illustrates how robots should be personalized for and by end- users or their assistants. Using this framework, non-technical users determine how !he robot should behave. Then, we define the concept of preference for assistive robotics scenarios and establish a taxonomy, which includes hierarchies and groups of preferences, grounding definitions and concepts. We then show how the preferences in the taxonomy are used with Al planning systems to adapt the robot behavior to the preferences of the user obtained from simple questions. Our algorithms allow for long-term adaptations as well as to cope with misinformed user models. We further integrate the methods with low-level motion primitives that provide a more robust adaptation and behavior while lowering the number of needed actions and demonstrations. Moreover, we perform a deeper analysis in Planning and preferences with the introduction of new algorithms to provide preference suggestions in planning domains. The thesis then concludes with a user study that evaluates the use of the preferences in the three real assistive robotics scenarios. The experiments show a clear understanding of the preferences of users, who were able to assess the impact of their preferences on the behavior of the robot. In summary, we provide tools and algorithms to design the robotic assistants of the future. Assistants that should be able to adapt to the assisted user needs and preferences, just as human assistants do nowadays.Els assistents robòtics han inspirat nombrosos llibres i pel·lícules de ciència-ficció al llarg de la història. Però tornant al món real, aquest tipus de dispositius s'estan tornant una necessitat per a una societat que envelleix a un ritme ràpid i que, per tant, requerirà més i més assistència. Mentre l'esperança de vida augmenta, la qualitat de vida no necessàriament ho fa. Per tant, ens podem trobar a nosaltres mateixos i als nostres estimats en una situació de dependència, necessitant una altra persona per poder fer les tasques més bàsiques, cosa que té un gran impacte psicològic. En conseqüència, els robots assistencials poden ser l'eina definitiva per proporcionar una millor qualitat de vida empoderant els usuaris i allargant la seva capacitat de viure independentment. L'assistència a persones per realitzar tasques diàries requereix adaptar-se a elles i les seves necessitats, donat que aquests usuaris no poden adaptar-se al robot. En aquesta tesi, abordem el problema de l'adaptació i la personalització d'un robot mitjançant preferències de l'usuari. Ens centrem en tasques físiques, que involucren contacte amb la persona, per les seves dificultats i importància per a l'usuari. Per aquest motiu, la tesi utilitzarà principalment tres tasques com a exemple: donar menjar, posar una sabata i vestir una jaqueta. Comencem definint un marc (framework) per a la personalització del comportament del robot que defineix com s'han de personalitzar els robots per usuaris i pels seus assistents. Amb aquest marc, usuaris sense coneixements tècnics són capaços de definir com s'ha de comportar el robot. Posteriorment definim el concepte de preferència per a robots assistencials i establim una taxonomia que inclou jerarquies i grups de preferències, els quals fonamenten les definicions i conceptes. Després mostrem com les preferències de la taxonomia s'utilitzen amb sistemes planificadors amb IA per adaptar el comportament del robot a les preferències de l'usuari, que s'obtenen mitjançant preguntes simples. Els nostres algorismes permeten l'adaptació a llarg termini, així com fer front a models d'usuari mal inferits. Aquests mètodes són integrats amb primitives a baix nivell que proporcionen una adaptació i comportament més robusts a la mateixa vegada que disminueixen el nombre d'accions i demostracions necessàries. També fem una anàlisi més profunda de l'ús de les preferències amb planificadors amb la introducció de nous algorismes per fer suggeriments de preferències en dominis de planificació. La tesi conclou amb un estudi amb usuaris que avalua l'ús de les preferències en les tres tasques assistencials. Els experiments demostren un clar enteniment de les preferències per part dels usuaris, que van ser capaços de discernir quan les seves preferències eren utilitzades. En resum, proporcionem eines i algorismes per dissenyar els assistents robòtics del futur. Uns assistents que haurien de ser capaços d'adaptar-se a les preferències i necessitats de l'usuari que assisteixen, tal com els assistents humans fan avui en dia

Housekeeping with multiple autonomous robots: representation, reasoning, and execution

We consider a housekeeping domain with static or movable objects, where the goal is for multiple autonomous robots to tidy a house collaboratively in a given amount of time. This domain is challenging in the following ways: commonsense knowledge (e.g., expected locations of objects in the house) is required for intelligent behavior of robots; geometric constraints are required to find feasible plans (e.g., to avoid collisions); in case of plan failure while execution (e.g., due to a collision with movable objects whose presence and location are not known in advance or due to heavy objects that cannot be lifted by a single robot), recovery is required depending on the cause of failure; and collaboration of robots is required to complete some tasks (e.g., carrying heavy objects). We introduce a formal planning, execution and monitoring framework to address the challenges of this domain, by embedding knowledge representation and automated reasoning in each level of decision-making (that consists of discrete task planning, continuous motion planning, and plan execution), in such a way as to tightly integrate these levels. At the high-level, we represent not only actions and change but also commonsense knowledge in a logicbased formalism. Geometric reasoning is lifted to the high-level by embedding motion planning in the domain description. Then a discrete plan is computed for each robot using an automated reasoner. At the mid-level, if a continuous trajectory cannot be computed by a motion planner because the discrete plan is not feasible at the continuous-level, then a different plan is computed by the automated reasoner subject to some (temporal) conditions represented as formulas. At the low-level, if the plan execution fails, then a new continuous trajectory is computed by a motion planner at the mid-level or a new discrete plan is computed using an automated reasoner at the high-level. We illustrate the applicability of this formal framework with a simulation of a housekeeping domain

The Digital World – Essence and Dualism

This text (a part of a Doctoral Thesis development in progress) is focused on the recent challenges before philosophy, based on the necessity to synchronize our contemporary concepts with the latest outcomes in the studies of the Digital World. The key perspective is grounded on what R. Descartes and G. F. Leibniz foresaw centuries ago. The standpoint is physicalist and the analysis considers: the human Binary Approach and Binary System, the ontological aspects of the computer programming languages, and the philosophy within the hardware and software correlation.Този текст (част от докторска дисертация в развитие) се фокусира върху съвременните предизвикателства пред философията, в процеса на синхронизиране на концепциите с бурно развиващите се изследвания на дигиталния свят. Отправна точка са вижданията на Р. Декарт и Г. Лайбниц, заявени преди векове, но валидни в значителна степен и днес. Коментарът е от позиция на физикалиста, а анализът включва човешкия бинарен подход, онтологичните аспекти на компютърните програмни езици, както и философията във взаимоотношенията хардуер-софтуер