Kinect-Based Vision System of Mine Rescue Robot for Low Illuminous Environment

This paper presents Kinect-based vision system of mine rescue robot working in illuminous underground environment. The somatosensory system of Kinect is used to realize the hand gesture recognition involving static hand gesture and action. A K-curvature based convex detection method is proposed to fit the hand contour with polygon. In addition, the hand action is completed by using the NiTE library with the framework of hand gesture recognition. In addition, the proposed method is compared with BP neural network and template matching. Furthermore, taking advantage of the information of the depth map, the interface of hand gesture recognition is established for human machine interaction of rescue robot. Experimental results verify the effectiveness of Kinect-based vision system as a feasible and alternative technology for HMI of mine rescue robot

A Consolidated Review of Path Planning and Optimization Techniques: Technical Perspectives and Future Directions

In this paper, a review on the three most important communication techniques (ground, aerial, and underwater vehicles) has been presented that throws light on trajectory planning, its optimization, and various issues in a summarized way. This kind of extensive research is not often seen in the literature, so an effort has been made for readers interested in path planning to fill the gap. Moreover, optimization techniques suitable for implementing ground, aerial, and underwater vehicles are also a part of this review. This paper covers the numerical, bio-inspired techniques and their hybridization with each other for each of the dimensions mentioned. The paper provides a consolidated platform, where plenty of available research on-ground autonomous vehicle and their trajectory optimization with the extension for aerial and underwater vehicles are documented

Object Detection and Obstacle Avoidance for Mobile Robot using Stereo Camera

The objective of this research is to develop a real time obstacle detection and obstacle avoidance for autonomous navigation of mobile robots using a stereo camera in an unstructured environment. Autonomous navigation of mobile robots demands a) Exact determination of position and orientation of Robot and b) Accurate determination of size, shape, depth and range of potential obstacles in the environment. Simple kinematic model is used for mobile robot and stereo camera with pan and tilt provision is considered for long range operation. Complete 3D reconstruction of object/obstacle is obtained from the stereo matching algorithm and with triangulation method. The pose ( position and orientation) of mobile robot is formulated from the static object observation with stereo reference matching points using RANSAC (RANdom SAmple Consensus) in successive frames. Potential field based obstacle avoidance formulation is carried out by using the obstacle range, size information, mobile robot position and orientation. Finally, proportional derivative navigation control loop along with obstacle avoidance algorithm is formulated and verified

Pose estimation and data fusion algorithms for an autonomous mobile robot based on vision and IMU in an indoor environment

Thesis (PhD(Computer Engineering))--University of Pretoria, 2021.Autonomous mobile robots became an active research direction during the past few years, and they are emerging in different sectors such as companies, industries, hospital, institutions, agriculture and homes to improve services and daily activities. Due to technology advancement, the demand for mobile robot has increased due to the task they perform and services they render such as carrying heavy objects, monitoring, delivering of goods, search and rescue missions, performing dangerous tasks in places like underground mines. Instead of workers being exposed to hazardous chemicals or environments that could affect health and put lives at risk, humans are being replaced with mobile robot services. It is with these concerns that the enhancement of mobile robot operation is necessary, and the process is assisted through sensors. Sensors are used as instrument to collect data or information that aids the robot to navigate and localise in its environment. Each sensor type has inherent strengths and weaknesses, therefore inappropriate combination of sensors could result into high cost of sensor deployment with low performance. Regardless, the potential and prospect of autonomous mobile robot, they are yet to attain optimal performance, this is because of integral challenges they are faced with most especially localisation. Localisation is one the fundamental issues encountered in mobile robot which demands attention and the challenging part is estimating the robot position and orientation of which this information can be acquired from sensors and other relevant systems. To tackle the issue of localisation, a good technique should be proposed to deal with errors, downgrading factors, improper measurement and estimations. Different approaches are recommended in estimating the position of a mobile robot. Some studies estimated the trajectory of the mobile robot and indoor scene reconstruction using a monocular visual odmometry. This approach cannot be feasible for large zone and complex environment. Radio frequency identification (RFID) technology on the other hand provides accuracy and robustness, but the method depend on the distance between the tags, and the distance between the tags and the reader. To increase the localisation accuracy, the number of RFID tags per unit area has to be increased. Therefore, this technique may not result in economical and easily scalable solution because of the increasing number of required tags and the associated cost of their deployment. Global Positioning System (GPS) is another approach that offers proved results in most scenarios, however, indoor localization is one of the settings in which GPS cannot be used because the signal strength is not reliable inside a building. Most approaches are not able to precisely localise autonomous mobile robot even with the high cost of equipment and complex implementation. Most the devices and sensors either requires additional infrastructures or they are not suitable to be used in an indoor environment. Therefore, this study proposes using data from vision and inertial sensors which comprise 3-axis of accelerometer and 3-axis of gyroscope, also known as 6-degree of freedom (6-DOF) to determine pose estimation of mobile robot. The inertial measurement unit (IMU) based tracking provides fast response, therefore, they can be considered to assist vision whenever it fails due to loss of visual features. The use of vision sensor helps to overcome the characteristic limitation of the acoustic sensor for simultaneous multiple object tracking. With this merit, vision is capable of estimating pose with respect to the object of interest. A singular sensor or system is not reliable to estimate the pose of a mobile robot due to limitations, therefore, data acquired from sensors and sources are combined using data fusion algorithm to estimate position and orientation within specific environment. The resulting model is more accurate because it balances the strengths of the different sensors. Information provided through sensor or data fusion can be used to support more-intelligent actions. The proposed algorithms are expedient to combine data from each of the sensor types to provide the most comprehensive and accurate environmental model possible. The algorithms use a set of mathematical equations that provides an efficient computational means to estimate the state of a process. This study investigates the state estimation methods to determine the state of a desired system that is continuously changing given some observations or measurements. From the performance and evaluation of the system, it can be observed that the integration of sources of information and sensors is necessary. This thesis has provided viable solutions to the challenging problem of localisation in autonomous mobile robot through its adaptability, accuracy, robustness and effectiveness.NRFUniversity of PretoriaElectrical, Electronic and Computer EngineeringPhD(Computer Engineering)Unrestricte