Deep reinforcement learning from human preferences

For sophisticated reinforcement learning (RL) systems to interact usefully with real-world environments, we need to communicate complex goals to these systems. In this work, we explore goals defined in terms of (non-expert) human preferences between pairs of trajectory segments. We show that this approach can effectively solve complex RL tasks without access to the reward function, including Atari games and simulated robot locomotion, while providing feedback on less than one percent of our agent's interactions with the environment. This reduces the cost of human oversight far enough that it can be practically applied to state-of-the-art RL systems. To demonstrate the flexibility of our approach, we show that we can successfully train complex novel behaviors with about an hour of human time. These behaviors and environments are considerably more complex than any that have been previously learned from human feedback

Assessing the Decision-Making Process in Human-Robot Collaboration Using a Lego-like EEG Headset

Human-robot collaboration (HRC) has become an emerging field, where the use of a robotic agent has been shifted from a supportive machine to a decision-making collaborator. A variety of factors can influence the effectiveness of decision-making processes during HRC, including the system-related (e.g., robot capability) and human-related (e.g., individual knowledgeability) factors. As a variety of contextual factors can significantly impact the human-robot decision-making process in collaborative contexts, the present study adopts a Lego-like EEG headset to collect and examine human brain activities and utilizes multiple questionnaires to evaluate participants’ cognitive perceptions toward the robot. A user study was conducted where two levels of robot capabilities (high vs. low) were manipulated to provide system recommendations. The participants were also identified into two groups based on their computational thinking (CT) ability. The EEG results revealed that different levels of CT abilities trigger different brainwaves, and the participants’ trust calibration of the robot also varies the resultant brain activities

Effects of a Social Robot's Autonomy and Group Orientation on Human Decision-Making

Social attributes of intelligent robots are important for human-robot systems. This paper investigates influences of robot autonomy (i.e., high versus low) and group orientation (i.e., ingroup versus outgroup) on a human decision-making process. We conducted a laboratory experiment with 48 college students and tested the hypotheses with MANCOVA. We find that a robot with high autonomy has greater influence on human decisions than a robot with low autonomy. No significant effect is found on group orientation or on the interaction between group orientation and autonomy level. The results provide implications for social robot design

Human-Robot Collaboration in Automotive Assembly

In the past decades, automation in the automobile production line has significantly increased the efficiency and quality of automotive manufacturing. However, in the automotive assembly stage, most tasks are still accomplished manually by human workers because of the complexity and flexibility of the tasks and the high dynamic unconstructed workspace. This dissertation is proposed to improve the level of automation in automotive assembly by human-robot collaboration (HRC). The challenges that eluded the automation in automotive assembly including lack of suitable collaborative robotic systems for the HRC, especially the compact-size high-payload mobile manipulators; teaching and learning frameworks to enable robots to learn the assembly tasks, and how to assist humans to accomplish assembly tasks from human demonstration; task-driving high-level robot motion planning framework to make the trained robot intelligently and adaptively assist human in automotive assembly tasks. The technical research toward this goal has resulted in several peer-reviewed publications. Achievements include: 1) A novel collaborative lift-assist robot for automotive assembly; 2) Approaches of vision-based robot learning of placing tasks from human demonstrations in assembly; 3) Robot learning of assembly tasks and assistance from human demonstrations using Convolutional Neural Network (CNN); 4) Robot learning of assembly tasks and assistance from human demonstrations using Task Constraint-Guided Inverse Reinforcement Learning (TC-IRL); 5) Robot learning of assembly tasks from non-expert demonstrations via Functional Objective-Oriented Network (FOON); 6) Multi-model sampling-based motion planning for trajectory optimization with execution consistency in manufacturing contexts. The research demonstrates the feasibility of a parallel mobile manipulator, which introduces novel conceptions to industrial mobile manipulators for smart manufacturing. By exploring the Robot Learning from Demonstration (RLfD) with both AI-based and model-based approaches, the research also improves robots’ learning capabilities on collaborative assembly tasks for both expert and non-expert users. The research on robot motion planning and control in the dissertation facilitates the safety and human trust in industrial robots in HRC

L’implantation de la robotique collaborative et la gestion des ressources humaines dans le secteur manufacturier : soutenir le changement et l’adoption

Ce mémoire de maîtrise explore l’implantation de la robotique collaborative en entreprise sous l’angle des pratiques de gestion et des facteurs humains. La visée initiale de ce projet de recherche visait préalablement à circonscrire l’apport que peut prendre la gestion des ressources humaines (GRH) lors de ce type d’implantation technologique, qui implique une collaboration humain-machine plus accrue qu’auparavant. Initialement, l’objectif était donc d’identifier les pratiques de GRH à mettre en place lors de l’implantation de robots collaboratifs. Cela dit, comme ce projet de recherche présente une démarche exploratoire semi-inductive, l’objectif de recherche a évolué vers plusieurs objectifs. Cette ouverture sur de nouveaux objectifs est subséquente aux résultats obtenus lors de la revue systématique de la littérature et de la collecte de données afin de dresser un portrait plus juste, adapté à l’état des connaissances et au terrain. Les objectifs poursuivis sont les suivants : 1) identifier les pratiques de GRH et d’autres pratiques organisationnelles en matière de gestion du changement facilitant l’implantation et l’adoption des robots collaboratifs 2) identifier les facteurs associés à l’humain, au robot et à l’environnement qui influencent l’implantation des robots collaboratifs, l’adoption et la collaboration entre l’opérateur et le robot

Occupational health and safety issues in human-robot collaboration: State of the art and open challenges

Human-Robot Collaboration (HRC) refers to the interaction of workers and robots in a shared workspace. Owing to the integration of the industrial automation strengths with the inimitable cognitive capabilities of humans, HRC is paramount to move towards advanced and sustainable production systems. Although the overall safety of collaborative robotics has increased over time, further research efforts are needed to allow humans to operate alongside robots, with awareness and trust. Numerous safety concerns are open, and either new or enhanced technical, procedural and organizational measures have to be investigated to design and implement inherently safe and ergonomic automation solutions, aligning the systems performance and the human safety. Therefore, a bibliometric analysis and a literature review are carried out in the present paper to provide a comprehensive overview of Occupational Health and Safety (OHS) issues in HRC. As a result, the most researched topics and application areas, and the possible future lines of research are identified. Reviewed articles stress the central role played by humans during collaboration, underlining the need to integrate the human factor in the hazard analysis and risk assessment. Human-centered design and cognitive engineering principles also require further investigations to increase the worker acceptance and trust during collaboration. Deepened studies are compulsory in the healthcare sector, to investigate the social and ethical implications of HRC. Whatever the application context is, the implementation of more and more advanced technologies is fundamental to overcome the current HRC safety concerns, designing low-risk HRC systems while ensuring the system productivity