Object Detection and Tracking in Cooperative Multi-Robot Transportation

Contemporary manufacturing systems imply the utilization of autonomous robotic systems, mainly for the execution of manipulation and transportation tasks. With a goal to reduce transportation and manipulation time, improve efficiency, and achieve flexibility of intelligent manufacturing systems, two or more intelligent mobile robots can be exploited. Such multi-robot systems require coordination and some level of communication between heterogeneous or homogeneous robotic systems. In this paper, we propose the utilization of two heterogeneous robotic systems, original intelligent mobile robots RAICO (Robot with Artificial Intelligence based COgnition) and DOMINO (Deep learning-based Omnidirectional Mobile robot with Intelligent cOntrol), for transportation tasks within a laboratory model of a manufacturing environment. In order to reach an adequate cooperation level and avoid collision while moving along predefined paths, our own developed intelligent mobile robots RAICO and DOMINO will communicate their current poses, and object detection and tracking system is developed. A stereo vision system equipped with two parallelly placed industrial-grade cameras is used for image acquisition, while convolutional neural networks are utilized for object detection, classification, and tracking. The proposed object detection and tracking system enables real-time tracking of another mobile robot within the same manufacturing environment. Furthermore, continuous information about mobile robot poses and the size of the bounding box generated by the convolutional neural network in the process of detection of another mobile robot is used for estimation of object movement and collision avoidance. Mobile robot localization through time is performed based on kinematic models of two intelligent mobile robots, and conducted experiments within a laboratory model of manufacturing environment confirm the applicability of the proposed framework for object detection and collision avoidance