PlanCollabNL: Leveraging Large Language Models for Adaptive Plan Generation in Human-Robot Collaboration

"Hey, robot. Let's tidy up the kitchen. By the way, I have back pain today". How can a robotic system devise a shared plan with an appropriate task allocation from this abstract goal and agent condition? Classical AI task planning has been explored for this purpose, but it involves a tedious definition of an inflexible planning problem. Large Language Models (LLMs) have shown promising generalisation capabilities in robotics decision-making through knowledge extraction from Natural Language (NL). However, the translation of NL information into constrained robotics domains remains a challenge. In this paper, we use LLMs as translators between NL information and a structured AI task planning problem, targeting human-robot collaborative plans. The LLM generates information that is encoded in the planning problem, including specific subgoals derived from an NL abstract goal, as well as recommendations for subgoal allocation based on NL agent conditions. The framework, PlanCollabNL, is evaluated for a number of goals and agent conditions, and the results show that correct and executable plans are found in most cases. With this framework, we intend to add flexibility and generalisation to HRC plan generation, eliminating the need for a manual and laborious definition of restricted planning problems and agent models

Improved task planning through failure anticipation in human-robot collaboration

© 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting /republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other worksThe human state is defined in terms of capacity, knowledge and motivation. The system has been implemented in a standardised environment using the Planning Domain Definition Language (PDDL) and the modular ROSPlan framework, and we have validated the approach in multiple simulation settings. Our results show that using the human model fosters an appropriate task allocation while allowing failure anticipation, replanning in time to prevent it.Peer ReviewedPostprint (author's final draft

Planning Interactions as an Event Handling Solution for Successful and Balanced Human-Robot Collaboration

Trabajo presentado en el IROS2022 Workshop: Artificial Intelligence for Social Robots Interacting with Humans in the Real World, celebrado en Kyoto (Japón), el 27 de octubre de 2022Dealing with the stochastic nature of human behaviour in Human-Robot Collaboration (HRC) remains a well known challenge that needs to be tackled. Automated task planning techniques have been implemented in order to share the workload between the agents, but these still lack the necessary adaptability for real world applications. In this paper, we extend the work presented by the authors in [1], where an improved task planning framework integrating an agent model was presented, anticipating and avoiding failures in HRC by reallocating the actions in the plan when necessary. This work introduces the integration of interaction actions into the planning framework, in order to deal with situations where the issue reflected by a change in the agent state might be better handled with an interaction between the agents than by an action reallocation. Preliminary evaluation shows promising results of how this framework can help to increase the success in HRC plans, as well as the balance in workload distribution between the agents, which constitutes a key element in a collaboration.This work was partially financed by MCIN/ AEI /10.13039/501100011033 and by the ”European Union NextGenerationEU/PRTR” under the project ROB-IN (PLEC2021-007859) and by the Catalan Government through the funding grant ACCIO-Eurecat

Planning interactions as an event handling solution for successful and balanced human-robot collaboration

Dealing with the stochastic nature of human behaviour in Human-Robot Collaboration (HRC) remains a well known challenge that needs to be tackled. Automated task planning techniques have been implemented in order to share the workload between the agents, but these still lack the necessary adaptability for real world applications. In this paper, we extend the work presented by the authors in [1], where an improved task planning framework integrating an agent model was presented, anticipating and avoiding failures in HRC by reallocating the actions in the plan when necessary. This work introduces the integration of interaction actions into the planning framework, in order to deal with situations where the issue reflected by a change in the agent state might be better handled with an interaction between the agents than by an action reallocation. Preliminary evaluation shows promising results of how this framework can help to increase the success in HRC plans, as well as the balance in workload distribution between the agents, which constitutes a key element in a collaboration.Peer ReviewedPostprint (published version

Improved Task Planning through Failure Anticipation in Human-Robot Collaboration

Trabajo presentado en la International Conference on Robotics and Automation (ICRA), celebrada en Philadelphia (Estados Unidos), del 23 al 27 de mayo de 2022Human-Robot Collaboration (HRC) has become a major trend in robotics in recent years with the idea of combining the strengths from both humans and robots. In order to share the work to be done, many task planning approaches have been implemented. However, they don't fully satisfy the required adaptability in human-robot collaborative tasks, with most approaches not considering neither the state of the human partner nor the possibility of adapting the collaborative plan during execution or even anticipating failures. In this paper, we present a planning system for human-robot collaborative plans that takes into account the agents' states and deals with unforeseen human behaviour, by replanning in anticipation when the human state changes to prevent action failure. The human state is defined in terms of capacity, knowledge and motivation. The system has been implemented in a standardised environment using the Planning Domain Definition Language (PDDL) and the modular ROSPlan framework, and we have validated the approach in multiple simulation settings. Our results show that using the human model fosters an appropriate task allocation while allowing failure anticipation, replanning in time to prevent it.This work was financially supported by the Catalan Government through the funding grant ACCIO-Eurecat, and partially supported by MCIN/ AEI ´ /10.13039/501100011033 under the project CHLOE-GRAPH (PID2020- 118649RB-l00); by MCIN/ AEI /10.13039/501100011033 and by the European Union NextGenerationEU/PRTR under the project COHERENT (PCI2020-120718-2). Gerard Canal has been partially supported by EPSRC grant THuMP (EP/R033722/1) and by the Royal Academy of Engineering and the Office of the Chief Science Adviser for National Security under the UK Intelligence Community Postdoctoral Research Fellowship programme