4 research outputs found
Classification and Grip of Occluded Objects
The present paper exposes a system for detection, classification, and grip of occluded objects by machine vision, artificial intelligence, and an anthropomorphic robot, to generate a solution for the subjection of elements that present occlusions. The deep learning algorithm used is based on Convolutional Neural Networks (CNN), specifically Fast R-CNN (Fast Region-Based CNN) and DAG-CNN (Directed Acyclic Graph CNN) for pattern recognition, the three-dimensional information of the environment was collected through Kinect V1, and tests simulations by the tool VRML. A sequence of detection, classification, and grip was programmed to determine which elements present occlusions and which type of tool generates the occlusion. According to the user's requirements, the desired elements are delivered (occluded or not), and the unwanted elements are removed. It was possible to develop a program with 88.89% accuracy in gripping and delivering occluded objects using networks Fast R-CNN and DAG-CNN with achieving of 70.9% and 96.2% accuracy respectively, detecting elements without occlusions for the first net and classifying the objects into five tools (Scalpel, Scissor, Screwdriver, Spanner, and Pliers), with the second net. The grip of occluded objects requires accurate detection of the element located at the top of the pile of objects to remove it without affecting the rest of the environment. Additionally, the detection process requires that a part of the occluded tool be visible to determine the existence of occlusions in the stac
Recommended from our members
RGBD Camera Pose Estimation Techniques, Slip Detection, and Occluded Object Search Strategies for Deformable Linear Object Features in Autonomous Robotic Space Task Execution
This thesis studies Robotic handling of Deformable Linear Objects (DLO). Many habitats used for space exploration include panels with multiple wires and connections which can be easily reconfigured by humans but very difficult to be handled autonomously by robotic systems due to the flexible nature of the wires. In some situations, the wires can come loose and get separated from their connections resulting in malfunctioning of some onboard systems. This thesis develops methods for autonomous handling of flexible wires (deformable linear objects) involving the unplugging and re-plugging or stowing of one end of the wire from a connection point. An anomaly situation may arise when the end of a gripped DLO slips away from the robotic end effector into the environment while being maneuvered, entering the object into anunknown state. The objective of the research presented herein was to use purely visual sensing to detect this DLO slip locating the loose connector end, estimating its pose, and autonomously developing a motion plan for retrieval and delivery of the connector end to its originally intended destination. Three pose estimation methods are implemented: employing fiducial markers, RGBD image processing, and machine learning algorithms to generate the pose of the end of the DLO being manipulated.
Experiments are performed using two cooperating robotic arms that show identification rates of 48.1%, 100.0%, and 77.8% and arm retrieval grasp rates of 48.1%, 74.1%, and 64.0% respectively among 27 trials. The identification rate varied based on the level of occlusion of the DLO end within the workspace. Slip detection is accomplished by comparing this estimated position’s distance to the manipulating arm’s end effector against a threshold quantifying a slip, producing a success rate of 77.2% from 18 slip trials. In the event that the loose connector settles out of the camera’s view, a spiral search pattern was designed to maneuver the secondary camera for further workspace inspection, with a search identification rate of 91.7% in 36 trials. The effectiveness of the overall system as a solution for anomaly detection and resolution is exhibited through three demonstrations with varying environmental configurations