Dynamic Grasping of Unknown Objects with a Multi-Fingered Hand

An important prerequisite for autonomous robots is their ability to reliably grasp a wide variety of objects. Most state-of-the-art systems employ specialized or simple end-effectors, such as two-jaw grippers, which severely limit the range of objects to manipulate. Additionally, they conventionally require a structured and fully predictable environment while the vast majority of our world is complex, unstructured, and dynamic. This paper presents an implementation to overcome both issues. Firstly, the integration of a five-finger hand enhances the variety of possible grasps and manipulable objects. This kinematically complex end-effector is controlled by a deep learning based generative grasping network. The required virtual model of the unknown target object is iteratively completed by processing visual sensor data. Secondly, this visual feedback is employed to realize closed-loop servo control which compensates for external disturbances. Our experiments on real hardware confirm the system's capability to reliably grasp unknown dynamic target objects without a priori knowledge of their trajectories. To the best of our knowledge, this is the first method to achieve dynamic multi-fingered grasping for unknown objects. A video of the experiments is available at https://youtu.be/Ut28yM1gnvI.Comment: ICRA202

Industrial Robotics

This book covers a wide range of topics relating to advanced industrial robotics, sensors and automation technologies. Although being highly technical and complex in nature, the papers presented in this book represent some of the latest cutting edge technologies and advancements in industrial robotics technology. This book covers topics such as networking, properties of manipulators, forward and inverse robot arm kinematics, motion path-planning, machine vision and many other practical topics too numerous to list here. The authors and editor of this book wish to inspire people, especially young ones, to get involved with robotic and mechatronic engineering technology and to develop new and exciting practical applications, perhaps using the ideas and concepts presented herein

Utilizing Reinforcement Learning and Computer Vision in a Pick-And-Place Operation for Sorting Objects in Motion

This master's thesis studies the implementation of advanced machine learning (ML) techniques in industrial automation systems, focusing on applying machine learning to enable and evolve autonomous sorting capabilities in robotic manipulators. In particular, Inverse Kinematics (IK) and Reinforcement Learning (RL) are investigated as methods for controlling a UR10e robotic arm for pick-and-place of moving objects on a conveyor belt within a small-scale sorting facility. A camera-based computer vision system applying YOLOv8 is used for real-time object detection and instance segmentation. Perception data is utilized to ascertain optimal grip points, specifically through an implemented algorithm that outputs optimal grip position, angle, and width. As the implemented system includes testing and evaluation on a physical system, the intricacies of hardware control, specifically the reverse engineering of an OnRobot RG6 gripper is elaborated as part of this study. The system is implemented on the Robotic Operating System (ROS), and its design is in particular driven by high modularity and scalability in mind. The camera-based vision system serves as the primary input, while the robot control is the output. The implemented system design allows for the evaluation of motion control employing both IK and RL. Computation of IK is conducted via MoveIt2, while the RL model is trained and computed in NVIDIA Isaac Sim. The high-level control of the robotic manipulator was accomplished with use of Proximal Policy Optimization (PPO). The main result of the research is a novel reward function for the pick-and-place operation that takes into account distance and orientation from the target object. In addition, the provided system administers task control by independently initializing pick-and-place operation phases for each environment. The findings demonstrate that PPO was able to significantly enhance the velocity, accuracy, and adaptability of industrial automation. Our research shows that accurate control of the robot arm can be reached by training the PPO Model purely by applying a digital twin simulation

Imitation Learning-based Visual Servoing for Tracking Moving Objects

In everyday life collaboration tasks between human operators and robots, the former necessitate simple ways for programming new skills, the latter have to show adaptive capabilities to cope with environmental changes. The joint use of visual servoing and imitation learning allows us to pursue the objective of realizing friendly robotic interfaces that (i) are able to adapt to the environment thanks to the use of visual perception and (ii) avoid explicit programming thanks to the emulation of previous demonstrations. This work aims to exploit imitation learning for the visual servoing paradigm to address the specific problem of tracking moving objects. In particular, we show that it is possible to infer from data the compensation term required for realizing the tracking controller, avoiding the explicit implementation of estimators or observers. The effectiveness of the proposed method has been validated through simulations with a robotic manipulator.Comment: International Workshop on Human-Friendly Robotics (HFR), 202

Safer hybrid workspace using human-robot interaction while sharing production activities

In a near future, human and industrial manipulator will work together sharing a common workspace and production activities leading to a potential increase of accident. The research project concerns the adaptation of industrial robot already installed in a flexible manufacturing system in order to make it more interactive with human. The aim concerns the reduction of potential risk of injuries while working with an industrial robot. This paper presents a new inexpensive, non-intrusive, non-invasive, and non-vision-based system, for human detection and collision avoidance. One method investigated for improving safety concerns planning of safe path. This system recognizes human activities and locates operator's position in real time through an instrumented safety helmet. This safety helmet includes an IMU (Inertial Measurement Unit) and an indoor localization system such as RSSI (Received Signal Strength Indication) using industrial wireless equipment. A hybrid workspace including a flexible manufacturing system has been designed in order to practice experiments in an industrial-like environment

Real-time control of industrial robots in multiple microcomputers

Imperial Users onl

Integration of robotic systems in a packaging machine: A tool for design and simulation of efficient motion trajectories

In this paper, the advantages of CACSD (Computer Aided Control System Design) tools for integrating a robotic system in a packaging machine are illustrated. Beside the mechanical integration of the robot into the machine architecture, it is necessary a functional integration, that requires a precise synchronization with the other parts of the system. In the proposed application, a robot with a parallel kinematics is used for pick-and-place tasks between two conveyor belts. It is therefore necessary a proper motion planning which allows to synchronize the grasp and release phases with the conveyor belts, avoiding obstacles and guaranteeing the compliance with bounds on velocity, acceleration and limits in the workspace. A trajectory composed by quintic polynomials has been considered and a specific tool has been designed in the Matlab environment, which allows to modify the parameters of the trajectory and to analyze the obtained motion profiles from both the kinematic and dynamic point of view