A virtual object point model for the calibration of underwater stereo cameras to recover accurate 3D information

The focus of this thesis is on recovering accurate 3D information from underwater images. Underwater 3D reconstruction differs significantly from 3D reconstruction in air due to the refraction of light. In this thesis, the concepts of stereo 3D reconstruction in air get extended for underwater environments by an explicit consideration of refractive effects with the aid of a virtual object point model. Within underwater stereo 3D reconstruction, the focus of this thesis is on the refractive calibration of underwater stereo cameras

A Comprehensive Review on Autonomous Navigation

The field of autonomous mobile robots has undergone dramatic advancements over the past decades. Despite achieving important milestones, several challenges are yet to be addressed. Aggregating the achievements of the robotic community as survey papers is vital to keep the track of current state-of-the-art and the challenges that must be tackled in the future. This paper tries to provide a comprehensive review of autonomous mobile robots covering topics such as sensor types, mobile robot platforms, simulation tools, path planning and following, sensor fusion methods, obstacle avoidance, and SLAM. The urge to present a survey paper is twofold. First, autonomous navigation field evolves fast so writing survey papers regularly is crucial to keep the research community well-aware of the current status of this field. Second, deep learning methods have revolutionized many fields including autonomous navigation. Therefore, it is necessary to give an appropriate treatment of the role of deep learning in autonomous navigation as well which is covered in this paper. Future works and research gaps will also be discussed

Underwater 3D Reconstruction Based on Physical Models for Refraction and Underwater Light Propagation

In recent years, underwater imaging has gained a lot of popularity partly due to the availability of off-the-shelf consumer cameras, but also due to a growing interest in the ocean floor by science and industry. Apart from capturing single images or sequences, the application of methods from the area of computer vision has gained interest as well. However, water affects image formation in two major ways. First, while traveling through the water, light is attenuated and scattered, depending on the light's wavelength causing the typical strong green or blue hue in underwater images. Second, cameras used in underwater scenarios need to be confined in an underwater housing, viewing the scene through a flat or dome-shaped glass port. The inside of the housing is filled with air. Consequently, the light entering the housing needs to pass a water-glass interface, then a glass-air interface, thus is refracted twice, affecting underwater image formation geometrically. In classic Structure-from-Motion (SfM) approaches, the perspective camera model is usually assumed, however, it can be shown that it becomes invalid due to refraction in underwater scenarios. Therefore, this thesis proposes an adaptation of the SfM algorithm to underwater image formation with flat port underwater housings, i.e. introduces a method where refraction at the underwater housing is modeled explicitly. This includes a calibration approach, algorithms for relative and absolute pose estimation, an efficient, non-linear error function that is utilized in bundle adjustment, and a refractive plane sweep algorithm. Finally, if calibration data for an underwater light propagation model exists, the dense depth maps can be used to correct texture colors. Experiments with a perspective and the proposed refractive approach to 3D reconstruction revealed that the perspective approach does indeed suffer from a systematic model error depending on the distance between camera and glass and a possible tilt of the glass with respect to the image sensor. The proposed method shows no such systematic error and thus provides more accurate results for underwater image sequences

Autonomous navigation for guide following in crowded indoor environments

The requirements for assisted living are rapidly changing as the number of elderly patients over the age of 60 continues to increase. This rise places a high level of stress on nurse practitioners who must care for more patients than they are capable. As this trend is expected to continue, new technology will be required to help care for patients. Mobile robots present an opportunity to help alleviate the stress on nurse practitioners by monitoring and performing remedial tasks for elderly patients. In order to produce mobile robots with the ability to perform these tasks, however, many challenges must be overcome. The hospital environment requires a high level of safety to prevent patient injury. Any facility that uses mobile robots, therefore, must be able to ensure that no harm will come to patients whilst in a care environment. This requires the robot to build a high level of understanding about the environment and the people with close proximity to the robot. Hitherto, most mobile robots have used vision-based sensors or 2D laser range finders. 3D time-of-flight sensors have recently been introduced and provide dense 3D point clouds of the environment at real-time frame rates. This provides mobile robots with previously unavailable dense information in real-time. I investigate the use of time-of-flight cameras for mobile robot navigation in crowded environments in this thesis. A unified framework to allow the robot to follow a guide through an indoor environment safely and efficiently is presented. Each component of the framework is analyzed in detail, with real-world scenarios illustrating its practical use. Time-of-flight cameras are relatively new sensors and, therefore, have inherent problems that must be overcome to receive consistent and accurate data. I propose a novel and practical probabilistic framework to overcome many of the inherent problems in this thesis. The framework fuses multiple depth maps with color information forming a reliable and consistent view of the world. In order for the robot to interact with the environment, contextual information is required. To this end, I propose a region-growing segmentation algorithm to group points based on surface characteristics, surface normal and surface curvature. The segmentation process creates a distinct set of surfaces, however, only a limited amount of contextual information is available to allow for interaction. Therefore, a novel classifier is proposed using spherical harmonics to differentiate people from all other objects. The added ability to identify people allows the robot to find potential candidates to follow. However, for safe navigation, the robot must continuously track all visible objects to obtain positional and velocity information. A multi-object tracking system is investigated to track visible objects reliably using multiple cues, shape and color. The tracking system allows the robot to react to the dynamic nature of people by building an estimate of the motion flow. This flow provides the robot with the necessary information to determine where and at what speeds it is safe to drive. In addition, a novel search strategy is proposed to allow the robot to recover a guide who has left the field-of-view. To achieve this, a search map is constructed with areas of the environment ranked according to how likely they are to reveal the guide’s true location. Then, the robot can approach the most likely search area to recover the guide. Finally, all components presented are joined to follow a guide through an indoor environment. The results achieved demonstrate the efficacy of the proposed components

Control and visual navigation for unmanned underwater vehicles

Ph. D. Thesis.Control and navigation systems are key for any autonomous robot. Due to environmental disturbances, model uncertainties and nonlinear dynamic systems, reliable functional control is essential and improvements in the controller design can significantly benefit the overall performance of Unmanned Underwater Vehicles (UUVs). Analogously, due to electromagnetic attenuation in underwater environments, the navigation of UUVs is always a challenging problem. In this thesis, control and navigation systems for UUVs are investigated. In the control field, four different control strategies have been considered: Proportional-Integral-Derivative Control (PID), Improved Sliding Mode Control (SMC), Backstepping Control (BC) and customised Fuzzy Logic Control (FLC). The performances of these four controllers were initially simulated and subsequently evaluated by practical experiments in different conditions using an underwater vehicle in a tank. The results show that the improved SMC is more robust than the others with small settling time, overshoot, and error. In the navigation field, three underwater visual navigation systems have been developed in the thesis: ArUco Underwater Navigation systems, a novel Integrated Visual Odometry with Monocular camera (IVO-M), and a novel Integrated Visual Odometry with Stereo camera (IVO-S). Compared with conventional underwater navigation, these methods are relatively low-cost solutions and unlike other visual or inertial-visual navigation methods, they are able to work well in an underwater sparse-feature environment. The results show the following: the ArUco underwater navigation system does not suffer from cumulative error, but some segments in the estimated trajectory are not consistent; IVO-M suffers from cumulative error (error ratio is about 3 - 4%) and is limited by the assumption that the “seabed is locally flat”; IVO-S suffers from small cumulative errors (error ratio is less than 2%). Overall, this thesis contributes to the control and navigation systems of UUVs, presenting the comparison between controllers, the improved SMC, and low-cost underwater visual navigation methods

A collaborative monocular visual simultaneous localization and mapping solution to generate a semi-dense 3D map.

The utilization and generation of indoor maps are critical in accurate indoor tracking. Simultaneous Localization and Mapping (SLAM) is one of the main techniques used for such map generation. In SLAM, an agent generates a map of an unknown environment while approximating its own location in it. The prevalence and afford-ability of cameras encourage the use of Monocular Visual SLAM, where a camera is the only sensing device for the SLAM process. In modern applications, multiple mobile agents may be involved in the generation of indoor maps, thus requiring a distributed computational framework. Each agent generates its own local map, which can then be combined with those of other agents into a map covering a larger area. In doing so, they cover a given environment faster than a single agent. Furthermore, they can interact with each other in the same environment, making this framework more practical, especially for collaborative applications such as augmented reality. One of the main challenges of collaborative SLAM is identifying overlapping maps, especially when the relative starting positions of the agents are unknown. We propose a system comprised of multiple monocular agents with unknown relative starting positions to generate a semi-dense global map of the environment

Indoor Localization and Mapping Using Deep Learning Networks

Over the past several decades, robots have been used extensively in environments that pose high risk to human operators and in jobs that are repetitive and monotonous. In recent years, robot autonomy has been exploited to extend their use in several non-trivial tasks such as space exploration, underwater exploration, and investigating hazardous environments. Such tasks require robots to function in unstructured environments that can change dynamically. Successful use of robots in these tasks requires them to be able to determine their precise location, obtain maps and other information about their environment, navigate autonomously, and operate intelligently in the unknown environment. The process of determining the location of the robot and generating a map of its environment has been termed in the literature as Simultaneous Localization and Mapping (SLAM). Light Detection and Ranging (LiDAR), Sound Navigation and Ranging (SONAR) sensors, and depth cameras are typically used to generate a representation of the environment during the SLAM process. However, the real-time localization and generation of map information are still challenging tasks. Therefore, there is a need for techniques to speed up the approximate localization and mapping process while using fewer computational resources. This thesis presents an alternative method based on deep learning and computer vision algorithms for generating approximate localization information for mobile robots. This approach has been investigated to obtain approximate localization information captured by monocular cameras. Approximate localization can subsequently be used to develop coarse maps where a priori information is not available. Experiments were conducted to verify the ability of the proposed technique to determine the approximate location of the robot. The approximate location of the robot was qualitatively denoted in terms of its location in a building, a floor of the building, and interior corridors. ArUco markers were used to determine the quantitative location of the robot. The use of this approximate location of the robot in determining the location of key features in the vicinity of the robot was also studied. The results of the research reported in this thesis demonstrate that low cost, low resolution techniques can be used in conjunction with deep learning techniques to obtain approximate localization of an autonomous robot. Further such approximate information can be used to determine coarse position information of key features in the vicinity. It is anticipated that this approach can be subsequently extended to develop low-resolution maps of the environment that are suitable for autonomous navigation of robots