A socio-technical approach for assistants in human-robot collaboration in industry 4.0

The introduction of technologies disruptive of Industry 4.0 in the workplace integrated through human cyber-physical systems causes operators to face new challenges. These are reflected in the increased demands presented in the operator's capabilities physical, sensory, and cognitive demands. In this research, cognitive demands are the most interesting. In this perspective, assistants are presented as a possible solution, not as a tool but as a set of functions that amplify human capabilities, such as exoskeletons, collaborative robots for physical capabilities, virtual and augmented reality for sensory capabilities. Perhaps chatbots and softbots for cognitive capabilities, then the need arises to ask ourselves: How can operator assistance systems 4.0 be developed in the context of industrial manufacturing? In which capacities does the operator need more assistance? From the current paradigm of systematization, different approaches are used within the context of the workspace in industry 4.0. Thus, the functional resonance analysis method (FRAM) is used to model the workspace from the sociotechnical system approach, where the relationships between the components are the most important among the functions to be developed by the human-robot team. With the use of simulators for both robots and robotic systems, the behavior of the variability of the human-robot team is analyzed. Furthermore, from the perspective of cognitive systems engineering, the workspace can be studied as a joint cognitive system, where cognition is understood as distributed, in a symbiotic relationship between the human and technological agents. The implementation of a case study as a human-robot collaborative workspace allows evaluating the performance of the human-robot team, the impact on the operator's cognitive abilities, and the level of collaboration achieved in the human-robot team through a set of metrics and proven methods in other areas, such as cognitive systems engineering, human-machine interaction, and ergonomics. We conclude by discussing the findings and outlook regarding future research questions and possible developments.La introducción de tecnologías disruptivas de Industria 4.0 en el lugar de trabajo integradas a través de sistemas ciberfísicos humanos hace que los operadores enfrenten nuevos desafíos. Estos se reflejan en el aumento de las demandas presentadas en las capacidades físicas, sensoriales y cognitivas del operador. En esta investigación, las demandas cognitivas son las más interesantes. En esta perspectiva, los asistentes se presentan como una posible solución, no como una herramienta sino como un conjunto de funciones que amplifican las capacidades humanas, como exoesqueletos, robots colaborativos para capacidades físicas, realidad virtual y aumentada para capacidades sensoriales. Quizás chatbots y softbots para capacidades cognitivas, entonces surge la necesidad de preguntarnos: ¿Cómo se pueden desarrollar los sistemas de asistencia al operador 4.0 en el contexto de la fabricación industrial? ¿En qué capacidades el operador necesita más asistencia? A partir del paradigma actual de sistematización, se utilizan diferentes enfoques dentro del contexto del espacio de trabajo en la industria 4.0. Así, se utiliza el método de análisis de resonancia funcional (FRAM) para modelar el espacio de trabajo desde el enfoque del sistema sociotécnico, donde las relaciones entre los componentes son las más importantes entre las funciones a desarrollar por el equipo humano-robot. Con el uso de simuladores tanto para robots como para sistemas robóticos se analiza el comportamiento de la variabilidad del equipo humano-robot. Además, desde la perspectiva de la ingeniería de sistemas cognitivos, el espacio de trabajo puede ser estudiado como un sistema cognitivo conjunto, donde la cognición se entiende distribuida, en una relación simbiótica entre los agentes humanos y tecnológicos. La implementación de un caso de estudio como un espacio de trabajo colaborativo humano-robot permite evaluar el desempeño del equipo humano-robot, el impacto en las habilidades cognitivas del operador y el nivel de colaboración alcanzado en el equipo humano-robot a través de un conjunto de métricas y métodos probados en otras áreas, como la ingeniería de sistemas cognitivos, la interacción hombre-máquina y la ergonomía. Concluimos discutiendo los hallazgos y las perspectivas con respecto a futuras preguntas de investigación y posibles desarrollos.Postprint (published version

A human-oriented design process for collaborative robotics

The potential of collaborative robotics often does not materialize in an efficient design of the human-robot collaboration. Technology-oriented approaches are no longer enough in the Industry 4.0 era. This work proposes a set of methods to support manufacturing engineers in the human-oriented design process of integrated production systems to obtain satisfactory performance in the mass customization paradigm, without impacting the safety and health of workers. It founds the design criteria definition on five main pillars (safety, ergonomics, effectiveness, flexibility, and costs), favors the consideration of different design alternatives, and leads their selection. The dynamic impact of the design choices on the various elements of the system prevails over the static design constraints. The method has been experimented in collaboration with the major kitchen manufacturer in Italy, which introduced a collaborative robotics cell in the drawers' assembly line. It resulted in a more balanced production line (10% more), a verified risk minimization (RULA score reduced from 5 to 3 and OCRA score from 13.30 to 5.70), and a greater allocation of operators to high added value activities

Materials Handling in Production Systems: Design and Performance of Kit Preparation

This thesis focuses on processes for kit preparation, which are applied with the materials supply principle of kitting in production systems for mixed-model assembly. With kitting, assembly processes are supplied with portions of pre-sorted components, and each portion makes up a kit that holds the components needed for one assembly object at one or several assembly processes. When kitting is applied, picking activities, which are otherwise performed at assembly processes, are instead carried out in a process for kit preparation. Kit preparation involves collecting components designated for a particular assembly object into a single unit load that is delivered to assembly.Kitting is widely seen as beneficial for quality and flexibility in assembly processes when there are a large variety of components. Performance effects in assembly processes normally associated with kitting largely depend on the performance of kit preparation. Previous research indicates that a picking system’s design greatly impacts its performance. While research that has dealt with kit preparation points out several design aspects that can affect its performance, the available knowledge is far from exhaustive. The purpose of this thesis is to contribute to the knowledge of how kit preparation design aspects govern kit preparation performance.Case research, experiments, and modelling have been used to study how flexibility, kit quality and man-hour efficiency are affected by kit preparation design aspects related to work organisation, layout, policies, packaging, equipment, picking information, automation and control. Two case research studies respectively address kit preparation flexibility and kit quality, identifying how kit preparation design aspects can be configured to support these two performance areas. Two experiments focus on how picking information systems and confirmation methods affect kit preparation man-hour efficiency. One modelling study focuses on how collaborative robots can support man-hour efficient kit preparation. Through involvement in three research projects and an extensive review of the literature, this research has been guided by the needs of industry and by previously established knowledge.This thesis contributes to theory and to practice in the form of knowledge about relationships between kit preparation design aspects and the performance areas flexibility, kit quality and manhour efficiency. The theoretical contribution consists of building upon and underpinning the limitedknowledge about the topic that has been previously available, while also adding new knowledge. This includes, for example, glasses with integrated computer displays, RFID-scanning wristbands, and collaborative robots, and how they are linked to kit preparation performance. The practical contribution consists of concise yet holistic descriptions of relationships between kit preparation design and performance, which industry can readily adopt with some consideration to the situation’s characteristics

Development of collaborative strategies in joint action

Many tasks in daily life involve coordinating movements between two or more individuals. A couple of dancers, a team of players, two workers carrying a load or a therapist interacting with a patient are just a few examples. Acting in collaboration or joint action is a crucial human ability, and our sensorimotor system is shaped to support this capability efficiently. When two partners have different goals but may benefit from collaborating, they face the challenge of negotiating a joint strategy. To do this, first and foremost both subjects need to know their partner\u2019s state and current strategy. It is unclear how the collaboration would be affected if information about the partner is unreliable or incomplete. This work intends to investigate the development of collaborative strategies in joint action. To this purpose, I developed a dedicated experimental apparatus and task. I also developed a general computational framework \u2013 based on differential game theory \u2013 for the description and implementation of interactive behaviours of two subjects performing a joint motor task. The model allows to simulate any joint sensorimotor action in which the joint dynamics can be represented as a linear dynamical system and each agent\u2019s task is formulated in terms of a quadratic cost functional. The model also accounts for imperfect information about dyad dynamics and partner\u2019s actions, and can predict the development of joint action through repeated performance. A first experimental study, focused on how the development of joint action is affected by incomplete and unreliable information. We found that information about the partner not only affects the speed at which a collaborative strategy is achieved (less information, slower learning) but also optimality of the collaboration. In particular, when information about the partner is reduced, the learned strategy is characterised by the development of alternating patterns of leader-follower roles, whereas greater information leads to a more synchronous behaviour. Simulations with a computational model based on game theory suggest that synchronous behaviours are close to optimal in a game theoretic sense (Nash equilibrium). The emergence of roles is a compensation strategy which minimises the need to estimate partner\u2019s intentions and is, therefore, more robust to incomplete information. A second study addresses how physical interaction develops between adults with Autism spectrum disorder (ASD) and typically developing subjects. ASD remains mostly a mystery and has therefore generated some theories trying to explain their cognitive disabilities, which involve an impaired ability to interact with other human partners. Although preliminary due to the small number of subjects, our results suggest that ASD subjects display heterogeneity in establishing a collaboration, which can be only partly explained with their ability to perceive haptic force. This work is a first attempt to establish a sensorimotor theory of joint action. It may provide new insights into the development of robots that are capable of establishing optimal collaborations with human partners, for instance in the context of robot-assisted rehabilitation

Bimanual robotic manipulation based on potential fields

openDual manipulation is a natural skill for humans but not so easy to achieve for a robot. The presence of two end effectors implies the need to consider the temporal and spatial constraints they generate while moving together. Consequently, synchronization between the arms is required to perform coordinated actions (e.g., lifting a box) and to avoid self-collision between the manipulators. Moreover, the challenges increase in dynamic environments, where the arms must be able to respond quickly to changes in the position of obstacles or target objects. To meet these demands, approaches like optimization-based motion planners and imitation learning can be employed but they have limitations such as high computational costs, or the need to create a large dataset. Sampling-based motion planners can be a viable solution thanks to their speed and low computational costs but, in their basic implementation, the environment is assumed to be static. An alternative approach relies on improved Artificial Potential Fields (APF). They are intuitive, with low computational, and, most importantly, can be used in dynamic environments. However, they do not have the precision to perform manipulation actions, and dynamic goals are not considered. This thesis proposes a system for bimanual robotic manipulation based on a combination of improved Artificial Potential Fields (APF) and the sampling-based motion planner RRTConnect. The basic idea is to use improved APF to bring the end effectors near their target goal while reacting to changes in the surrounding environment. Only then RRTConnect is triggered to perform the manipulation task. In this way, it is possible to take advantage of the strengths of both methods. To improve this system APF have been extended to consider dynamic goals and a self-collision avoidance system has been developed. The conducted experiments demonstrate that the proposed system adeptly responds to changes in the position of obstacles and target objects. Moreover, the self-collision avoidance system enables faster dual manipulation routines compared to sequential arm movements

Productivity and flexibility improvement of assembly lines for high-mix low-volume production. A white goods industry case

Las tendencias globales de la personalización e individualización en masa impulsan la producción industrial en serie corta y variada; y por tanto una gran variedad de productos en pequeñas cantidades. Por ello, la customización en masa precisa de sistemas de ensamblaje que sean a la vez altamente productivos y flexibles, a diferencia de la tradicional oposición entre ambas características. La llamada cuarta revolución industrial trae diversas tecnologías habilitadoras que podrían ser útiles para abordar este problema. Sin embargo, las metodologías para implementar el ensamblaje 4.0 todavía no han sido resueltas. De hecho, para aprovechar todas las ventajas potenciales de la Industria 4.0, es necesario contar con un nivel previo de excelencia operacional y un análisis holístico de los sistemas productivos. Esta tesis tiene como objetivo entender y definir cómo mejorar la productividad y la flexibilidad de las operaciones de montaje en serie corta y variada.Esta meta se ha dividido en tres objetivos. El primer objetivo consiste en comprender las relaciones entre la Industria 4.0 y las operaciones de ensamblaje, así como sus implicaciones para los operarios. El segundo objetivo consiste en desarrollar una metodología y las herramientas necesarias para evaluar el rendimiento de diferentes configuraciones de cadenas de ensamblaje. El último objetivo consiste en el diseño de sistemas de ensamblaje que permitan incrementar su productividad al menos un 25 %, produciendo en serie corta y variada, mediante la combinación de puestos de montaje manual y estaciones automatizadas.Para abordar la fase de comprensión y definición del problema, se llevó a cabo una revisión bibliográfica sistemática y se desarrolló un marco conceptual para el Ensamblaje 4.0. Se desarrollaron, verificaron y validaron dos herramientas de evaluación del rendimiento: un modelo matemático analítico y varios modelos de simulación por eventos discretos. Para la verificación, y como punto de partida para los análisis, se ha utilizado un caso de estudio industrial de un fabricante global de electrodomésticos. Se han empleado múltiples escenarios de simulación y técnicas de diseño de experimentos para investigar tres cuestiones clave.En primer lugar, se identificaron los factores más críticos para el rendimiento de líneas de montaje manuales multi-modelo. En segundo lugar, se analizó el rendimiento de líneas de montaje semiautomáticas paralelas con operarios móviles en comparación con líneas semiautomáticas o manuales con operarios fijos, empleando diversos escenarios de demanda en serie corta y variada. Por último, se investigó el uso de trenes milkrun para la logística interna de líneas de ensamblaje multi-modelo bajo la influencia de perturbaciones.Los resultados de las simulaciones muestran que las líneas paralelas con operarios móviles pueden superar a las de operarios fijos en cualquier escenario de demanda, alcanzando como mínimo el objetivo de mejorar la productividad en un 25% o más. También permiten reducir cómodamente el número de operarios trabajando en la línea sin afectar negativamente al equilibrado de la misma, posibilitando la producción eficiente de bajo volumen. Los resultados de las simulaciones de logística interna indican que los milkrun pueden proteger las líneas de ensamblaje de las perturbaciones originadas en procesos aguas arriba.Futuras líneas de investigación en base a los resultados obtenidos en esta tesis podrían incluir la expansión e integración de los modelos de simulación actuales para analizar las cadenas de montaje paralelas con operarios móviles incorporando logística, averías y mantenimiento, problemas de control de calidad y políticas de gestión de los retrabajos. Otra línea podría ser el uso de diferentes herramienta para el análisis del desempeño como, por ejemplo, técnicas de programación de la producción que permitan evaluar el desempeño operacional de diferentes configuraciones de cadenas de montaje con operarios móviles, tanto en términos de automatización como de organización en planta. Podrían incorporarse tecnologías de la Industria 4.0 a los modelos de simulación para evaluar su impacto operacional global ¿como cobots para ensamblaje o para la manipulación de materiales, realidad aumentada para el apoyo cognitivo a los operarios, o AGVs para la conducciónde los trenes milkrun. Por último, el trabajo presentado en esta tesis acerca las líneas de ensamblaje semiautomáticas con operarios móviles a su implementación industrial.<br /

Multi-robot cooperative platform : a task-oriented teleoperation paradigm

This thesis proposes the study and development of a teleoperation system based on multi-robot cooperation under the task oriented teleoperation paradigm: Multi-Robot Cooperative Paradigm, MRCP. In standard teleoperation, the operator uses the master devices to control the remote slave robot arms. These arms reproduce the desired movements and perform the task. With the developed work, the operator can virtually manipulate an object. MRCP automatically generates the arms orders to perform the task. The operator does not have to solve situations arising from possible restrictions that the slave arms may have. The research carried out is therefore aimed at improving the accuracy teleoperation tasks in complex environments, particularly in the field of robot assisted minimally invasive surgery. This field requires patient safety and the workspace entails many restrictions to teleoperation. MRCP can be defined as a platform composed of several robots that cooperate automatically to perform a teleoperated task, creating a robotic system with increased capacity (workspace volume, accessibility, dexterity ...). The cooperation is based on transferring the task between robots when necessary to enable a smooth task execution. The MRCP control evaluates the suitability of each robot to continue with the ongoing task and the optimal time to execute a task transfer between the current selected robot and the best candidate to continue with the task. From the operator¿s point of view, MRCP provides an interface that enables the teleoperation though the task-oriented paradigm: operator orders are translated into task actions instead of robot orders. This thesis is structured as follows: The first part is dedicated to review the current solutions in the teleoperation of complex tasks and compare them with those proposed in this research. The second part of the thesis presents and reviews in depth the different evaluation criteria to determine the suitability of each robot to continue with the execution of a task, considering the configuration of the robots and emphasizing the criterion of dexterity and manipulability. The study reviews the different required control algorithms to enable the task oriented telemanipulation. This proposed teleoperation paradigm is transparent to the operator. Then, the Thesis presents and analyses several experimental results using MRCP in the field of minimally invasive surgery. These experiments study the effectiveness of MRCP in various tasks requiring the cooperation of two hands. A type task is used: a suture using minimally invasive surgery technique. The analysis is done in terms of execution time, economy of movement, quality and patient safety (potential damage produced by undesired interaction between the tools and the vital tissues of the patient). The final part of the thesis proposes the implementation of different virtual aids and restrictions (guided teleoperation based on haptic visual and audio feedback, protection of restricted workspace regions, etc.) using the task oriented teleoperation paradigm. A framework is defined for implementing and applying a basic set of virtual aids and constraints within the framework of a virtual simulator for laparoscopic abdominal surgery. The set of experiments have allowed to validate the developed work. The study revealed the influence of virtual aids in the learning process of laparoscopic techniques. It has also demonstrated the improvement of learning curves, which paves the way for its implementation as a methodology for training new surgeons.Aquesta tesi doctoral proposa l'estudi i desenvolupament d'un sistema de teleoperació basat en la cooperació multi-robot sota el paradigma de la teleoperació orientada a tasca: Multi-Robot Cooperative Paradigm, MRCP. En la teleoperació clàssica, l'operador utilitza els telecomandaments perquè els braços robots reprodueixin els seus moviments i es realitzi la tasca desitjada. Amb el treball realitzat, l'operador pot manipular virtualment un objecte i és mitjançant el MRCP que s'adjudica a cada braç les ordres necessàries per realitzar la tasca, sense que l'operador hagi de resoldre les situacions derivades de possibles restriccions que puguin tenir els braços executors. La recerca desenvolupada està doncs orientada a millorar la teleoperació en tasques de precisió en entorns complexos i, en particular, en el camp de la cirurgia mínimament invasiva assistida per robots. Aquest camp imposa condicions de seguretat del pacient i l'espai de treball comporta moltes restriccions a la teleoperació. MRCP es pot definir com a una plataforma formada per diversos robots que cooperen de forma automàtica per dur a terme una tasca teleoperada, generant un sistema robòtic amb capacitats augmentades (volums de treball, accessibilitat, destresa,...). La cooperació es basa en transferir la tasca entre robots a partir de determinar quin és aquell que és més adequat per continuar amb la seva execució i el moment òptim per realitzar la transferència de la tasca entre el robot actiu i el millor candidat a continuar-la. Des del punt de vista de l'operari, MRCP ofereix una interfície de teleoperació que permet la realització de la teleoperació mitjançant el paradigma d'ordres orientades a la tasca: les ordres es tradueixen en accions sobre la tasca en comptes d'estar dirigides als robots. Aquesta tesi està estructurada de la següent manera: Primerament es fa una revisió de l'estat actual de les diverses solucions desenvolupades actualment en el camp de la teleoperació de tasques complexes, comparant-les amb les proposades en aquest treball de recerca. En el segon bloc de la tesi es presenten i s'analitzen a fons els diversos criteris per determinar la capacitat de cada robot per continuar l'execució d'una tasca, segons la configuració del conjunt de robots i fent especial èmfasi en el criteri de destresa i manipulabilitat. Seguint aquest estudi, es presenten els diferents processos de control emprats per tal d'assolir la telemanipulació orientada a tasca de forma transparent a l'operari. Seguidament es presenten diversos resultats experimentals aplicant MRCP al camp de la cirurgia mínimament invasiva. En aquests experiments s'estudia l'eficàcia de MRCP en diverses tasques que requereixen de la cooperació de dues mans. S'ha escollit una tasca tipus: sutura amb tècnica de cirurgia mínimament invasiva. L'anàlisi es fa en termes de temps d'execució, economia de moviment, qualitat i seguretat del pacient (potencials danys causats per la interacció no desitjada entre les eines i els teixits vitals del pacient). Finalment s'ha estudiat l'ús de diferents ajudes i restriccions virtuals (guiat de la teleoperació via retorn hàptic, visual o auditiu, protecció de regions de l'espai de treball, etc) dins el paradigma de teleoperació orientada a tasca. S'ha definint un marc d'aplicació base i implementant un conjunt de restriccions virtuals dins el marc d'un simulador de cirurgia laparoscòpia abdominal. El conjunt d'experiments realitzats han permès validar el treball realitzat. Aquest estudi ha permès determinar la influencia de les ajudes virtuals en el procés d'aprenentatge de les tècniques laparoscòpiques. S'ha evidenciat una millora en les corbes d'aprenentatge i obre el camí a la seva implantació com a metodologia d'entrenament de nous cirurgians.Postprint (published version