Hierarchical Goal Networks: Formalisms and Algorithms for Planning and Acting

In real-world applications of AI and automation such as in robotics, computer game playing and web-services, agents need to make decisions in unstructured environments that are open-world, dynamic and partially observable. In the AI and Robotics research communities in particular, there is much interest in equipping robots to operate with minimal human intervention in diverse scenarios such as in manufacturing plants, homes, hospitals, etc. Enabling agents to operate in these environments requires advanced planning and acting capabilities, some of which are not well supported by the current state of the art automated planning formalisms and algorithms. To address this problem, in my thesis I propose a new planning formalism that addresses some of the inadequacies in current planning frameworks, and a suite of planning and acting algorithms that operate under this planning framework. The main contributions of this thesis are: - Hierarchical Goal Network (HGN) Planning Formalism. This planning formalism combines aspects (and therefore harnesses advantages) of Classical Planning and Hierarchical Task Network (HTN) Planning, two of the most prominent planning formalisms currently in use. In particular, HGN planning algorithms, while retaining the efficiency and scalability advantages of HTNs, also allows incorporation of heuristics and other reasoning techniques from Classical Planning. - Planning Algorithms. Goal Decomposition Planner (GDP) and the Goal Decomposition with Landmarks (GoDeL) planner are two HGN planning algorithms that combines hierarchical decomposition with classical planning heuristics to outperform state-of-the-art HTN planners like SHOP and SHOP2. - Integration with Robotics. The Combined HGN and Motion Planning (CHaMP) algorithm integrates GoDeL with low-level motion and manipulation planning algorithms in Robotics to generate plans directly executable by robots. Given the need for autonomous agents to operate in open, dynamic and unstructured environments and the obvious need for high-level deliberation capabilities to enable intelligent behavior, the planning-and-acting systems that are developed as part of this thesis may provide unique insights into ways to realize these systems in the real world

Safe navigation and human-robot interaction in assistant robotic applications

L'abstract è presente nell'allegato / the abstract is in the attachmen

From human-human collaboration to human-robot collaboration: automated generation of assembly task knowledge model

Task knowledge is essential for robots to proactively perform collaborative assembly tasks with a human partner. Representation of task knowledge, such as task graphs, robot skill libraries, are usually manually defined by human experts. In this paper, different from learning from demonstrations of a single agent, we propose a system that automatically constructs task knowledge models from dual-human demonstrations in the real environment. Firstly, we track and segment video demonstrations into sequences of action primitives. Secondly, a graph-based algorithm is proposed to extract structure information of a task from action sequences, with task graphs as output. Finally, action primitives, along with interactive information between agents, temporal constraints, are modelled into a structured semantic model. The proposed system is validated in an IKEA table assembly task experiment

The concept of collaborative engineering: a systematic literature review

Collaborative engineering is not a new subject but it assumes a new importance in the Industry 4.0 (I4.0). There are other concepts frequently mismatched with collaboration. Thus, the main objective of this paper is to put forward a collaborative engineering concept, along its sub concepts, supported by an extensive systematic literature review. A critical analysis and discussion about the fundamental importance of learning, and the central human role in collaboration, in the I4.0, is presented, based on the main insights brought through the literature review. This study also enables to realize about the importance of collaboration in the current digitalization era, along with the importance of recent approaches and technology for enabling or promoting collaboration. Main current practices of human centered and autonomous machine-machine approaches and applications of collaboration in engineering, namely in manufacturing and management, are presented, along with main difficulties and further open research opportunities on collaboration.This work was supported by the Fundação para a Ciência e a Tecnologia [UIDB/00319/2020, UIDB/50014/2020, and EXPL/EME-SIS/1224/2021]

Robots learn to behave: improving human-robot collaboration in flexible manufacturing applications

L'abstract è presente nell'allegato / the abstract is in the attachmen

Trust-Based Control of Robotic Manipulators in Collaborative Assembly in Manufacturing

Human-robot interaction (HRI) is vastly addressed in the field of automation and manufacturing. Most of the HRI literature in manufacturing explored physical human-robot interaction (pHRI) and invested in finding means for ensuring safety and optimized effort sharing amongst a team of humans and robots. The recent emergence of safe, lightweight, and human-friendly robots has opened a new realm for human-robot collaboration (HRC) in collaborative manufacturing. For such robots with the new HRI functionalities to interact closely and effectively with a human coworker, new human-centered controllers that integrate both physical and social interaction are demanded. Social human-robot interaction (sHRI) has been demonstrated in robots with affective abilities in education, social services, health care, and entertainment. Nonetheless, sHRI should not be limited only to those areas. In particular, we focus on human trust in robot as a basis of social interaction. Human trust in robot and robot anthropomorphic features have high impacts on sHRI. Trust is one of the key factors in sHRI and a prerequisite for effective HRC. Trust characterizes the reliance and tendency of human in using robots. Factors within a robotic system (e.g. performance, reliability, or attribute), the task, and the surrounding environment can all impact the trust dynamically. Over-reliance or under-reliance might occur due to improper trust, which results in poor team collaboration, and hence higher task load and lower overall task performance. The goal of this dissertation is to develop intelligent control algorithms for the manipulator robots that integrate both physical and social HRI factors in the collaborative manufacturing. First, the evolution of human trust in a collaborative robot model is identified and verified through a series of human-in-the-loop experiments. This model serves as a computational trust model estimating an objective criterion for the evolution of human trust in robot rather than estimating an individual\u27s actual level of trust. Second, an HRI-based framework is developed for controlling the speed of a robot performing pick and place tasks. The impact of the consideration of the different level of interaction in the robot controller on the overall efficiency and HRI criteria such as human perceived workload and trust and robot usability is studied using a series of human-in-the-loop experiments. Third, an HRI-based framework is developed for planning and controlling the robot motion in performing hand-over tasks to the human. Again, series of human-in-the-loop experimental studies are conducted to evaluate the impact of implementation of the frameworks on overall efficiency and HRI criteria such as human workload and trust and robot usability. Finally, another framework is proposed for the cooperative manipulation of a common object by a team of a human and a robot. This framework proposes a trust-based role allocation strategy for adjusting the proactive behavior of the robot performing a cooperative manipulation task in HRC scenarios. For the mentioned frameworks, the results of the experiments show that integrating HRI in the robot controller leads to a lower human workload while it maintains a threshold level of human trust in robot and does not degrade robot usability and efficiency

Combining task and motion planning for mobile manipulators

Aplicat embargament des de la data de defensa fins el dia 31/12/2019Premi Extraordinari de Doctorat, promoció 2018-2019. Àmbit d’Enginyeria IndustrialThis thesis addresses the combination of task and motion planning which deals with different types of robotic manipulation problems. Manipulation problems are referred to as mobile manipulation, collaborative multiple mobile robots tasks, and even higher dimensional tasks (like bi-manual robots or mobile manipulators). Task and motion planning problems needs to obtain a geometrically feasible manipulation plan through symbolic and geometric search space. The combination of task and motion planning levels has emerged as a challenging issue as the failure leads robots to dead-end tasks due to geometric constraints. In addition, task planning is combined with physics-based motion planning and information to cope with manipulation tasks in which interactions between robots and objects are required, or also a low-cost feasible plan in terms of power is looked for. Moreover, combining task and motion planning frameworks is enriched by introducing manipulation knowledge. It facilitates the planning process and aids to provide the way of executing symbolic actions. Combining task and motion planning can be considered under uncertain information and with human-interaction. Uncertainty can be viewed in the initial state of the robot world or the result of symbolic actions. To deal with such issues, contingent-based task and motion planning is proposed using a perception system and human knowledge. Also, robots can ask human for those tasks which are difficult or infeasible for the purpose of collaboration. An implementation framework to combine different types of task and motion planning is presented. All the required modules and tools are also illustrated. As some task planning algorithms are implemented in Prolog or C++ languages and our geometric reasoner is developed in C++, the flow of information between different languages is explained.Aquesta tesis es centra en les eines de planificació combinada a nivell de tasca i a nivell de moviments per abordar diferents problemes de manipulació robòtica. Els problemes considerats són de navegació de robots mòbil enmig de obstacles no fixes, tasques de manipulació cooperativa entre varis robots mòbils, i tasques de manipulació de dimensió més elevada com les portades a terme amb robots bi-braç o manipuladors mòbils. La planificació combinada de tasques i de moviments ha de cercar un pla de manipulació que sigui geomètricament realitzable, a través de d'un espai de cerca simbòlic i geomètric. La combinació dels nivells de planificació de tasca i de moviments ha sorgit com un repte ja que les fallades degudes a les restriccions geomètriques poden portar a tasques sense solució. Addicionalment, la planificació a nivell de tasca es combina amb informació de la física de l'entorn i amb mètodes de planificació basats en la física, per abordar tasques de manipulació en les que la interacció entre el robot i els objectes és necessària, o també si es busca un pla realitzable i amb un baix cost en termes de potència. A més, el marc proposat per al combinació de la planificació a nivell de tasca i a nivell de moviments es millora mitjançant l'ús de coneixement, que facilita el procés de planificació i ajuda a trobar la forma d'executar accions simbòliques. La combinació de nivells de planificació també es pot considerar en casos d'informació incompleta i en la interacció humà-robot. La incertesa es considera en l'estat inicial i en el resultat de les accions simbòliques. Per abordar aquest problema, es proposa la planificació basada en contingències usant un sistema de percepció i el coneixement de l'operari humà. Igualment, els robots poden demanar col·laboració a l'operari humà per a que realitzi aquelles accions que són difícils o no realitzables pel robot. Es presenta també un marc d'implementació per a la combinació de nivells de planificació usant diferents mètodes, incloent tots els mòduls i eines necessàries. Com que alguns algorismes estan implementats en Prolog i d'altres en C++, i el mòdul de raonament geomètric proposat està desenvolupat en C++, es detalla el flux d'informació entre diferents llenguatges.Award-winningPostprint (published version