While robotic systems may have once been relegated to structured environments and automation style tasks, in recent years these boundaries have begun to erode. As robots begin to operate in largely unstructured environments, it becomes more difficult for them to effectively interpret their surroundings. As sensor technology improves, the amount of data these robots must utilize can quickly become intractable. Additional challenges include environmental noise, dynamic obstacles, and inherent sensor non-linearities. Deep learning techniques have emerged as a way to efficiently deal with these challenges. While end-to-end deep learning can be convenient, challenges such as validation and training requirements can be prohibitive to its use.
In order to address these issues, we propose augmenting the power of deep learning techniques with tools such as optimization methods, physics based models, and human expertise. In this work, we present a principled framework for approaching a problem that allows a user to identify the types of augmentation methods and deep learning techniques best suited to their problem. To validate our framework, we consider three different domains: LIDAR based odometry estimation, hybrid soft robotic control, and sonar based underwater mapping.
First, we investigate LIDAR based odometry estimation which can be characterized with both high data precision and availability; ideal for augmenting with optimization methods. We propose using denoising autoencoders (DAEs) to address the challenges presented by modern LIDARs. Our proposed approach is comprised of two stages: a novel pre-processing stage for robust feature identification and a scan matching stage for motion estimation. Using real-world data from the University of Michigan North Campus long-term vision and LIDAR dataset (NCLT dataset) as well as the KITTI dataset, we show that our approach generalizes across domains; is capable of reducing the per-estimate error of standard ICP methods on average by 25.5% for the translational component and 57.53% for the rotational component; and is capable of reducing the computation time of state-of-the-art ICP methods by a factor of 7.94 on average while achieving competitive performance.
Next, we consider hybrid soft robotic control which has lower data precision due to real-world noise (e.g., friction and manufacturing imperfections). Here, augmenting with model based methods is more appropriate. We present a novel approach for modeling, and classifying between, the system load states introduced when constructing staged soft arm configurations. Our proposed approach is comprised of two stages: an LSTM calibration routine used to identify the current load state and a control input generation step that combines a generalized quasistatic model with the learned load model. We show our method is capable of classifying between different arm configurations at a rate greater than 95%. Additionally, our method is capable of reducing the end-effector error of quasistatic model only control to within 1 cm of our controller baseline.
Finally, we examine sonar based underwater mapping. Here, data is so noisy that augmenting with human experts and incorporating some global context is required. We develop a novel framework that enables the real-time 3D reconstruction of underwater environments using features from 2D sonar images. In our approach, a convolutional neural network (CNN) analyzes sonar imagery in real-time and only proposes a small subset of high-quality frames to the human expert for feature annotation. We demonstrate that our approach provides real-time reconstruction capability without loss in classification performance on datasets captured onboard our underwater vehicle while operating in a variety of environments
