3D registration and integrated segmentation framework for heterogeneous unmanned robotic systems

The paper proposes a novel framework for registering and segmenting 3D point clouds of large-scale natural terrain and complex environments coming from a multisensor heterogeneous robotics system, consisting of unmanned aerial and ground vehicles. This framework involves data acquisition and pre-processing, 3D heterogeneous registration and integrated multi-sensor based segmentation modules. The first module provides robust and accurate homogeneous registrations of 3D environmental models based on sensors' measurements acquired from the ground (UGV) and aerial (UAV) robots. For 3D UGV registration, we proposed a novel local minima escape ICP (LME-ICP) method, which is based on the well known iterative closest point (ICP) algorithm extending it by the introduction of our local minima estimation and local minima escape mechanisms. It did not require any prior known pose estimation information acquired from sensing systems like odometry, global positioning system (GPS), or inertial measurement units (IMU). The 3D UAV registration has been performed using the Structure from Motion (SfM) approach. In order to improve and speed up the process of outliers removal for large-scale outdoor environments, we introduced the Fast Cluster Statistical Outlier Removal (FCSOR) method. This method was used to filter out the noise and to downsample the input data, which will spare computational and memory resources for further processing steps. Then, we co-registered a point cloud acquired from a laser ranger (UGV) and a point cloud generated from images (UAV) generated by the SfM method. The 3D heterogeneous module consists of a semi-automated 3D scan registration system, developed with the aim to overcome the shortcomings of the existing fully automated 3D registration approaches. This semi-automated registration system is based on the novel Scale Invariant Registration Method (SIRM). The SIRM provides the initial scaling between two heterogenous point clouds and provides an adaptive mechanism for tuning the mean scale, based on the difference between two consecutive estimated point clouds' alignment error values. Once aligned, the resulting homogeneous ground-aerial point cloud is further processed by a segmentation module. For this purpose, we have proposed a system for integrated multi-sensor based segmentation of 3D point clouds. This system followed a two steps sequence: ground-object segmentation and color-based region-growing segmentation. The experimental validation of the proposed 3D heterogeneous registration and integrated segmentation framework was performed on large-scale datasets representing unstructured outdoor environments, demonstrating the potential and benefits of the proposed semi-automated 3D registration system in real-world environments

3D Registration and Integrated Segmentation Framework for Heterogeneous Unmanned Robotic Systems

The paper proposes a novel framework for registering and segmenting 3D point clouds of large-scale natural terrain and complex environments coming from a multisensor heterogeneous robotics system, consisting of unmanned aerial and ground vehicles. This framework involves data acquisition and pre-processing, 3D heterogeneous registration and integrated multi-sensor based segmentation modules. The first module provides robust and accurate homogeneous registrations of 3D environmental models based on sensors' measurements acquired from the ground (UGV) and aerial (UAV) robots. For 3D UGV registration, we proposed a novel local minima escape ICP (LME-ICP) method, which is based on the well known iterative closest point (ICP) algorithm extending it by the introduction of our local minima estimation and local minima escape mechanisms. It did not require any prior known pose estimation information acquired from sensing systems like odometry, global positioning system (GPS), or inertial measurement units (IMU). The 3D UAV registration has been performed using the Structure from Motion (SfM) approach. In order to improve and speed up the process of outliers removal for large-scale outdoor environments, we introduced the Fast Cluster Statistical Outlier Removal (FCSOR) method. This method was used to filter out the noise and to downsample the input data, which will spare computational and memory resources for further processing steps. Then, we co-registered a point cloud acquired from a laser ranger (UGV) and a point cloud generated from images (UAV) generated by the SfM method. The 3D heterogeneous module consists of a semi-automated 3D scan registration system, developed with the aim to overcome the shortcomings of the existing fully automated 3D registration approaches. This semi-automated registration system is based on the novel Scale Invariant Registration Method (SIRM). The SIRM provides the initial scaling between two heterogenous point clouds and provides an adaptive mechanism for tuning the mean scale, based on the difference between two consecutive estimated point clouds' alignment error values. Once aligned, the resulting homogeneous ground-aerial point cloud is further processed by a segmentation module. For this purpose, we have proposed a system for integrated multi-sensor based segmentation of 3D point clouds. This system followed a two steps sequence: ground-object segmentation and color-based region-growing segmentation. The experimental validation of the proposed 3D heterogeneous registration and integrated segmentation framework was performed on large-scale datasets representing unstructured outdoor environments, demonstrating the potential and benefits of the proposed semi-automated 3D registration system in real-world environments

Deep-LK for Efficient Adaptive Object Tracking

In this paper we present a new approach for efficient regression based object tracking which we refer to as Deep- LK. Our approach is closely related to the Generic Object Tracking Using Regression Networks (GOTURN) framework of Held et al. We make the following contributions. First, we demonstrate that there is a theoretical relationship between siamese regression networks like GOTURN and the classical Inverse-Compositional Lucas & Kanade (IC-LK) algorithm. Further, we demonstrate that unlike GOTURN IC-LK adapts its regressor to the appearance of the currently tracked frame. We argue that this missing property in GOTURN can be attributed to its poor performance on unseen objects and/or viewpoints. Second, we propose a novel framework for object tracking - which we refer to as Deep-LK - that is inspired by the IC-LK framework. Finally, we show impressive results demonstrating that Deep-LK substantially outperforms GOTURN. Additionally, we demonstrate comparable tracking performance to current state of the art deep-trackers whilst being an order of magnitude (i.e. 100 FPS) computationally efficient

Local Mismatch Location and Spatial Scale Detection in Image Registration

Image registration is now a well understood problem and several techniques using a combination of cost functions, transformation models and optimizers have been reported in medical imaging literature. Parametric methods often rely on the efficient placement of control points in the images, that is, depending on the location and scale at which images are mismatched. Poor choice of parameterization results in deformations not being modeled accurately or over parameterization, where control points may lie in homogeneous regions with low sensitivity to cost. This lowers computational efficiency due to the high complexity of the search space and might also provide transformations that are not physically meaningful, and possibly folded. Adaptive methods that parameterize based on mismatch in images have been proposed. In such methods, the cost measure must be normalized, heuristics such as how many points to pick, resolution of the grids, choosing gradient thresholds and when to refine scale would have to be ascertained in addition to the limitation of working only at a few discrete scales. In this paper we identify mismatch by searching the entire image and a wide range of smooth spatial scales. The mismatch vector, containing location and scale of mismatch is computed from peaks in the local joint entropy. Results show that this method can be used to quickly and effectively locate mismatched regions in images where control points can be placed in preference to other regions speeding up registration.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/85931/1/Fessler223.pd

3D Object Detection Via 2D LiDAR Corrected Pseudo LiDAR Point Clouds

The age of automation has led to significant research in the field of Machine Learning and Computer Vision. Computer Vision tasks fundamentally rely on information from digital images, videos, texts and sensors to build intelligent systems. In recent times, deep neural networks combined with computer vision algorithms have been successful in developing 2D object detection methods with a potential to be applied in real-time systems. However, performing fast and accurate 3D object detection is still a challenging problem. The automotive industry is shifting gears towards building electric vehicles, connected cars, sustainable vehicles and is expected to have a high growth potential in the coming years. 3D object detection is a critical task for autonomous driving vehicles and robots as it helps moving objects in the scene to effectively plan their motion around other objects. 3D object detection tasks leverage image data from camera and/or 3D point clouds obtained from expensive 3D LiDAR sensors to achieve high detection accuracy. The 3D LiDAR sensor provides accurate depth information that is required to estimate the third dimension of the objects in the scene. Typically, a 64 beam LiDAR sensor mounted on a self-driving car cost around $75000. In this thesis, we propose a cost-effective approach for 3D object detection using a low-cost 2D LiDAR sensor. We collectively use the single beam point cloud data from 2D LiDAR for depth correction in pseudo-LiDAR. The proposed methods are tested on the KITTI 3D object detection dataset

RMNv2: Reduced Mobilenet V2 an Efficient Lightweight Model for Hardware Deployment

Indiana University-Purdue University Indianapolis (IUPUI)Humans can visually see things and can differentiate objects easily but for computers, it is not that easy. Computer Vision is an interdisciplinary field that allows computers to comprehend, from digital videos and images, and differentiate objects. With the Introduction to CNNs/DNNs, computer vision is tremendously used in applications like ADAS, robotics and autonomous systems, etc. This thesis aims to propose an architecture, RMNv2, that is well suited for computer vision applications such as ADAS, etc. RMNv2 is inspired by its original architecture Mobilenet V2. It is a modified version of Mobilenet V2. It includes changes like disabling downsample layers, Heterogeneous kernel-based convolutions, mish activation, and auto augmentation. The proposed model is trained from scratch in the CIFAR10 dataset and produced an accuracy of 92.4% with a total number of parameters of 1.06M. The results indicate that the proposed model has a model size of 4.3MB which is like a 52.2% decrease from its original implementation. Due to its less size and competitive accuracy the proposed model can be easily deployed in resource-constrained devices like mobile and embedded devices for applications like ADAS etc. Further, the proposed model is also implemented in real-time embedded devices like NXP Bluebox 2.0 and NXP i.MX RT1060 for image classification tasks

Smart Cage Active Contours and their application to brain image segmentation

In this work we present a new segmentation method named Smart Cage Active Contours (SCAC) that combines a parametrized active contour framework named Cage Active Contours (CAC), based on a ne trans- formations, with Active Shape Models (ASM). Our method e ectively restricts the shapes the evolving contours can take without the need of the training images to be manually landmarked. We apply our method to segment the caudate nuclei subcortical structure of a set of 40 subjects in magnetic resonance brain images, with promising results