Real time Visual SLAM for mobile robot navigation and object detection : a thesis presented in partial fulfilment of the requirement for the degree of Doctor of Philosophy in Engineering at Massey University, Albany, New Zealand

Abstract

This study developed a real-time Visual Simultaneous Localization and Mapping (SLAM) method for mobile robot navigation and object detection (SLAM-O), in order to establish the position of a mobile robot and interesting objects in an unknown indoor environment. The VEX Robotics Competition (VRC) is one of the largest, fastest growing educational programmes in the world, and it is designed to increase student interest and involvement in Science, Technology, Engineering, and Mathematics (STEM). This study aims to enable an autonomous robot to compete with human participants in the VRC match, where robots are programmed to respond to a user’s remote control. To win the competition, the robot needs to optimise between detection of goal objects, navigation and scoring. This thesis presents a Visual SLAM technique for robot localization and field objects’ mapping, and aims to provide an innovative and practical approach to control robot navigation and maximise its scoring. For visual observation, this study consists of an evaluation and comparison of widely used RGB-D cameras. Also, this thesis describes the integration of an iterated video frames module with the Extended Kalman Filter (EKF) for accurate SLAM estimation; where, a new frame selection method is employed. A novel SLAM-O method is developed for detecting objects in a robot’s navigation process. The SLAM-O method uses a new K-Means-based colour identification method for semi-transparent object detection and a new concave-based object separation method for multi-connected objects, which outperform traditional methods. Through conducting an investigation into RGB-D cameras’ performance, in terms of repeatability and accuracy, colour images and depth point clouds accuracy from different cameras are evaluated and compared. These comparison’s results provide a reference for choosing a camera for robot localisation. Depth errors and covariance are obtained from the investigation. The obtained results provide important parameters for a RGB-D camera related computation, such as a SLAM problem, etc. An Extended Kalman Filter (EKF) based Visual SLAM method and an iterated video frames module integrated in the EKF are developed. The Visual SLAM method can handle feature detection and corresponding depth measurements in an efficient way, and the iterated video frames module is capable of maximise robot self-localization accuracy. Experimental results demonstrate the accuracy of states estimate. These two method enables a mobile robot navigates accurately in an indoor environment with low computation cost. In addition, this study also presents a SLAM-O method by integrating object detection into Visual SLAM. SLAM-O enables robots to locate objects of interest, which can be used in robotic applications, such as navigation, object grasp, etc. To locate objects which are semi-transparent or closely connected, a K-Means method to clustering pixels on a semi-transparent object’s surface and a concave based method for object separation are developed. Experimental results prove the methods efficiency. These two methods are efficient and useful tools for object detection in SLAM-O framework