Vision-based human presence detection pipeline by means of transfer learning approach

Over the last century, industrial robots have gained an immense amount of popularity in replacing the human workers due to their highly repetitive nature. It was a twist to the industries when the concept of cooperative robots, known as cobots, has been innovated. Sharing space between the cobots and human workers has considered as the most effective way of utilizing the cobots. Keeping in mind that the safety of the human workers is always the top priority of the cobot applications in the industries, many time and efforts have been invested to improve the safeness of the cobots deployments. Yet, the utilization of deep learning technologies is rarely found in accordance with human detection in the field of research, especially the transfer learning approach, providing that the visual perception has shown to be a unique sense that still cannot be replaced by other. Hence, this thesis aimed to leverage the transfer learning approach to fine-tune the deep learning-based object detection models in the human detection task. In relation to this main goal, the objectives of the study were as follows: establish an image dataset for cobot environment from the surveillance cameras in TT Vision Holdings Berhad, formulate deep learning-based object detection models by using the transfer learning approach, and evaluate the performance of various transfer learning models in detecting the presence of human workers with relevant evaluation metrics. Image dataset has acquired from the surveillance system of TT Vision Holdings Berhad and annotated accordingly. The variations of the dataset have been considered thoroughly to ensure the models can be well-trained on the distinct features of the human workers. TensorFlow Object Detection API was used in the study to perform the fine-tuning of the one-stage object detectors. Among all the transfer learning strategies, fine-tuning has chosen since it suits the study well after the interpretation on the size-similarity matrix. A total of four EfficientDet models, two SSD models, three RetinaNet models, and four CenterNet models were deployed in the present work. As a result, SSD-MobileNetV2-FPN model has achieved 81.1% AP with 32.82 FPS, which is proposed as the best well-balanced fine-tuned model between accuracy and speed. In other case where the consideration is taken solely on either accuracy or inference speed, SSD_MobileNetV1-FPN model has attained 87.2% AP with 28.28 FPS and CenterNet-ResNet50-V1-FPN has achieved 78.0% AP with 46.52 FPS, which is proposed to be the model with best accuracy and inference speed, respectively. As a whole, it could be deduced that the transfer learning models can handle the human detection task well via the fine-tuning on the COCO-pretrained weights

Dynamic Obstacle Avoidance for Omnidirectional Mobile Manipulators

The last decade witnessed an unprecedented spread of robotics. The production paradigm of Industry 4.0 and 5.0 yielded collaborative robots in production lines of all sizes. Also, the robots started leaving the industrial scenario to play a leading role in the field of personal assistance. These environments share a common challenge, i.e. the safety of people working and/or living around the robots. Collision avoidance control techniques are essential to improve such aspect, by preventing impacts that can occur between the robot and humans or objects. The paper extends algorithms already developed by the authors for robotic arms to the case of mobile manipulators. The control strategy, which has been refined in the contribution of the robot bodies, has then been tested in two simulated case studies involving an industrial mobile robot and the custom service robot Paquitop, developed at Politecnico di Torino

Adaptive Obstacle Avoidance for a Class of Collaborative Robots

In a human–robot collaboration scenario, operator safety is the main problem and must be guaranteed under all conditions. Collision avoidance control techniques are essential to improve operator safety and robot flexibility by preventing impacts that can occur between the robot and humans or with objects inadvertently left within the operational workspace. On this basis, collision avoidance algorithms for moving obstacles are presented in this paper: inspired by algorithms already developed by the authors for planar manipulators, algorithms are adapted for the 6-DOF collaborative manipulators by Universal Robots, and some new contributions are introduced. First, in this work, the safety region wrapping each link of the manipulator assumes a cylindrical shape whose radius varies according to the speed of the colliding obstacle, so that dynamical obstacles are avoided with increased safety regions in order to reduce the risk, whereas fixed obstacles allow us to use smaller safety regions, facilitating the motion of the robot. In addition, three different modalities for the collision avoidance control law are proposed, which differ in the type of motion admitted for the perturbation of the end-effector: the general mode allows for a 6-DOF perturbation, but restrictions can be imposed on the orientation part of the avoidance motion using 4-DOF or 3-DOF modes. In order to demonstrate the effectiveness of the control strategy, simulations with dynamic and fixed obstacles are presented and discussed. Simulations are also used to estimate the required computational effort in order to verify the transferability to a real system

Automation of product packaging for industrial applications

[EN] This work presents a robotic-based solution devised to automate the product packaging in industrial environments. Although the proposed approach is illustrated for the case of the shoe industry, it applies to many other products requiring similar packaging processes. The main advantage obtained with the automated task is that productivity could be significantly increased. The key algorithms for the developed robot system are: object detection using a computer vision system; object grasping; trajectory planning with collision avoidance; and operator interaction using a force/torque sensor. All these algorithms have been experimentally tested in the laboratory to show the effectiveness and applicability of the proposed approach.This work has been partly supported by Ministerio de Economia y Competitividad of the Spanish Government [Grant No. RTC201654086 and PRI-AIBDE-2011-1219], by the Deutscher Akademischer Austauschdienst (DAAD) of the German Government (Projekt-ID 54368155) and by ROBOFOOT project [Grant No. 260159] of the European Commission.Perez-Vidal, C.; Gracia, L.; De Paco, J.; Wirkus, M.; Azorin, J.; De Gea, J. (2018). Automation of product packaging for industrial applications. International Journal of Computer Integrated Manufacturing. 31(2):129-137. https://doi.org/10.1080/0951192X.2017.1369165S12913731

Integrating intelligence and knowledge of human factors to facilitate collaboration in manufacturing

The implementation of automation has become a common occurrence in recent years, and automated robotic systems are actively used in many manufacturing processes. However, fully automated manufacturing systems are far less common, and human operators remain prevalent. The resulting scenario is one where human and robotic operators work in close proximity, and directly affect the behavior of one another. Conversely to their robotic counterparts, human beings do not share the same level of repeatability or accuracy, and as such can be a source of uncertainty in such processes. Concurrently, the emergence of intelligent manufacturing has presented opportunities for adaptability within robotic control. This work examines relevant human factors and develops a learning model to examine how to utilize this knowledge and provide appropriate adaptability to robotic elements, with the intention of improving collaborative interaction with human colleagues, and optimized performance. The work is supported by an example case-study, which explores the application of such a control system, and its performance in a real-world production scenario

Occupational health and safety issues in human-robot collaboration: State of the art and open challenges

Human-Robot Collaboration (HRC) refers to the interaction of workers and robots in a shared workspace. Owing to the integration of the industrial automation strengths with the inimitable cognitive capabilities of humans, HRC is paramount to move towards advanced and sustainable production systems. Although the overall safety of collaborative robotics has increased over time, further research efforts are needed to allow humans to operate alongside robots, with awareness and trust. Numerous safety concerns are open, and either new or enhanced technical, procedural and organizational measures have to be investigated to design and implement inherently safe and ergonomic automation solutions, aligning the systems performance and the human safety. Therefore, a bibliometric analysis and a literature review are carried out in the present paper to provide a comprehensive overview of Occupational Health and Safety (OHS) issues in HRC. As a result, the most researched topics and application areas, and the possible future lines of research are identified. Reviewed articles stress the central role played by humans during collaboration, underlining the need to integrate the human factor in the hazard analysis and risk assessment. Human-centered design and cognitive engineering principles also require further investigations to increase the worker acceptance and trust during collaboration. Deepened studies are compulsory in the healthcare sector, to investigate the social and ethical implications of HRC. Whatever the application context is, the implementation of more and more advanced technologies is fundamental to overcome the current HRC safety concerns, designing low-risk HRC systems while ensuring the system productivity