The Hierarchic treatment of marine ecological information from spatial networks of benthic platforms

Measuring biodiversity simultaneously in different locations, at different temporal scales, and over wide spatial scales is of strategic importance for the improvement of our understanding of the functioning of marine ecosystems and for the conservation of their biodiversity. Monitoring networks of cabled observatories, along with other docked autonomous systems (e.g., Remotely Operated Vehicles [ROVs], Autonomous Underwater Vehicles [AUVs], and crawlers), are being conceived and established at a spatial scale capable of tracking energy fluxes across benthic and pelagic compartments, as well as across geographic ecotones. At the same time, optoacoustic imaging is sustaining an unprecedented expansion in marine ecological monitoring, enabling the acquisition of new biological and environmental data at an appropriate spatiotemporal scale. At this stage, one of the main problems for an effective application of these technologies is the processing, storage, and treatment of the acquired complex ecological information. Here, we provide a conceptual overview on the technological developments in the multiparametric generation, storage, and automated hierarchic treatment of biological and environmental information required to capture the spatiotemporal complexity of a marine ecosystem. In doing so, we present a pipeline of ecological data acquisition and processing in different steps and prone to automation. We also give an example of population biomass, community richness and biodiversity data computation (as indicators for ecosystem functionality) with an Internet Operated Vehicle (a mobile crawler). Finally, we discuss the software requirements for that automated data processing at the level of cyber-infrastructures with sensor calibration and control, data banking, and ingestion into large data portals.Peer ReviewedPostprint (published version

Marine Vessel Inspection as a Novel Field for Service Robotics: A Contribution to Systems, Control Methods and Semantic Perception Algorithms.

This cumulative thesis introduces a novel field for service robotics: the inspection of marine vessels using mobile inspection robots. In this thesis, three scientific contributions are provided and experimentally verified in the field of marine inspection, but are not limited to this type of application. The inspection scenario is merely a golden thread to combine the cumulative scientific results presented in this thesis. The first contribution is an adaptive, proprioceptive control approach for hybrid leg-wheel robots, such as the robot ASGUARD described in this thesis. The robot is able to deal with rough terrain and stairs, due to the control concept introduced in this thesis. The proposed system is a suitable platform to move inside the cargo holds of bulk carriers and to deliver visual data from inside the hold. Additionally, the proposed system also has stair climbing abilities, allowing the system to move between different decks. The robot adapts its gait pattern dynamically based on proprioceptive data received from the joint motors and based on the pitch and tilt angle of the robot's body during locomotion. The second major contribution of the thesis is an independent ship inspection system, consisting of a magnetic wall climbing robot for bulkhead inspection, a particle filter based localization method, and a spatial content management system (SCMS) for spatial inspection data representation and organization. The system described in this work was evaluated in several laboratory experiments and field trials on two different marine vessels in close collaboration with ship surveyors. The third scientific contribution of the thesis is a novel approach to structural classification using semantic perception approaches. By these methods, a structured environment can be semantically annotated, based on the spatial relationships between spatial entities and spatial features. This method was verified in the domain of indoor perception (logistics and household environment), for soil sample classification, and for the classification of the structural parts of a marine vessel. The proposed method allows the description of the structural parts of a cargo hold in order to localize the inspection robot or any detected damage. The algorithms proposed in this thesis are based on unorganized 3D point clouds, generated by a LIDAR within a ship's cargo hold. Two different semantic perception methods are proposed in this thesis. One approach is based on probabilistic constraint networks; the second approach is based on Fuzzy Description Logic and spatial reasoning using a spatial ontology about the environment

On Advanced Mobility Concepts for Intelligent Planetary Surface Exploration

Surface exploration by wheeled rovers on Earth's Moon (the two Lunokhods) and Mars (Nasa's Sojourner and the two MERs) have been followed since many years already very suc-cessfully, specifically concerning operations over long time. However, despite of this success, the explored surface area was very small, having in mind a total driving distance of about 8 km (Spirit) and 21 km (Opportunity) over 6 years of operation. Moreover, ESA will send its ExoMars rover in 2018 to Mars, and NASA its MSL rover probably this year. However, all these rovers are lacking sufficient on-board intelligence in order to overcome longer dis-tances, driving much faster and deciding autonomously on path planning for the best trajec-tory to follow. In order to increase the scientific output of a rover mission it seems very nec-essary to explore much larger surface areas reliably in much less time. This is the main driver for a robotics institute to combine mechatronics functionalities to develop an intelligent mo-bile wheeled rover with four or six wheels, and having specific kinematics and locomotion suspension depending on the operational terrain of the rover to operate. DLR's Robotics and Mechatronics Center has a long tradition in developing advanced components in the field of light-weight motion actuation, intelligent and soft manipulation and skilled hands and tools, perception and cognition, and in increasing the autonomy of any kind of mechatronic systems. The whole design is supported and is based upon detailed modeling, optimization, and simula-tion tasks. We have developed efficient software tools to simulate the rover driveability per-formance on various terrain characteristics such as soft sandy and hard rocky terrains as well as on inclined planes, where wheel and grouser geometry plays a dominant role. Moreover, rover optimization is performed to support the best engineering intuitions, that will optimize structural and geometric parameters, compare various kinematics suspension concepts, and make use of realistic cost functions like mass and consumed energy minimization, static sta-bility, and more. For self-localization and safe navigation through unknown terrain we make use of fast 3D stereo algorithms that were successfully used e.g. in unmanned air vehicle ap-plications and on terrestrial mobile systems. The advanced rover design approach is applica-ble for lunar as well as Martian surface exploration purposes. A first mobility concept ap-proach for a lunar vehicle will be presented

Automation and Control Architecture for Hybrid Pipeline Robots

The aim of this research project, towards the automation of the Hybrid Pipeline Robot (HPR), is the development of a control architecture and strategy, based on reconfiguration of the control strategy for speed-controlled pipeline operations and self-recovering action, while performing energy and time management. The HPR is a turbine powered pipeline device where the flow energy is converted to mechanical energy for traction of the crawler vehicle. Thus, the device is flow dependent, compromising the autonomy, and the range of tasks it can perform. The control strategy proposes pipeline operations supervised by a speed control, while optimizing the energy, solved as a multi-objective optimization problem. The states of robot cruising and self recovering, are controlled by solving a neuro-dynamic programming algorithm for energy and time optimization, The robust operation of the robot includes a self-recovering state either after completion of the mission, or as a result of failures leading to the loss of the robot inside the pipeline, and to guaranteeing the HPR autonomy and operations even under adverse pipeline conditions Two of the proposed models, system identification and tracking system, based on Artificial Neural Networks, have been simulated with trial data. Despite the satisfactory results, it is necessary to measure a full set of robot’s parameters for simulating the complete control strategy. To solve the problem, an instrumentation system, consisting on a set of probes and a signal conditioning board, was designed and developed, customized for the HPR’s mechanical and environmental constraints. As a result, the contribution of this research project to the Hybrid Pipeline Robot is to add the capabilities of energy management, for improving the vehicle autonomy, increasing the distances the device can travel inside the pipelines; the speed control for broadening the range of operations; and the self-recovery capability for improving the reliability of the device in pipeline operations, lowering the risk of potential loss of the robot inside the pipeline, causing the degradation of pipeline performance. All that means the pipeline robot can target new market sectors that before were prohibitive

Deep Reinforcement Learning for the Velocity Control of a Magnetic, Tethered Differential-Drive Robot

The ROBOPLANET Altiscan crawler is a magnetic-wheeled, differential-drive robot being explored as an option to aid, if not completely replace, humans in the inspection and maintenance of marine vessels. Velocity control of the crawler is a crucial part in establishing trust and reliability amongst its operators. However, thanks to the crawler's elongated, magnetic wheels and umbilical tether, it operates in a complex environment rich with nonlinear dynamics which makes control challenging. Model-based approaches for the control of a robot that aim to mathematically formalize the physics of the system require an in-depth knowledge of the domain. Reinforcement learning (RL) is a trial-and-error-based approach that can solve control problems in nonlinear systems. To accommodate for high-dimensionality and continuous state spaces, deep neural networks (DNNs) can be used as nonlinear function approximators to extend RL, creating a method known as deep reinforcement learning (DRL). DRL coupled with a simulated environment provides a way for a model to learn physics-naive control. The research conducted in this thesis explored the efficacy of a DRL algorithm, proximal policy optimization (PPO), to learn the velocity control of the Altiscan crawler by modeling its operating environment in a novel, GPU-accelerated simulation software called Isaac Gym. The approaches evaluated the error between measured base velocities of the crawler as a result of the actions provided by the DRL model and target velocities in six different environments. Two variants of PPO, standard and recurrent, were compared against the inverse velocity kinematics model of a differential-drive robot. The results show that velocity control in simulation is possible using PPO, but evaluation on the real crawler is needed to come to a meaningful conclusion.M.S

A Flexible Autonomous Robotic Observatory Infrastructure for Bentho-Pelagic Monitoring

This paper presents the technological developments and the policy contexts for the project “Autonomous Robotic Sea-Floor Infrastructure for Bentho-Pelagic Monitoring” (ARIM). The development is based on the national experience with robotic component technologies that are combined and merged into a new product for autonomous and integrated ecological deep-sea monitoring. Traditional monitoring is often vessel-based and thus resource demanding. It is economically unviable to fulfill the current policy for ecosystem monitoring with traditional approaches. Thus, this project developed platforms for bentho-pelagic monitoring using an arrangement of crawler and stationary platforms at the Lofoten-Vesterålen (LoVe) observatory network (Norway). Visual and acoustic imaging along with standard oceanographic sensors have been combined to support advanced and continuous spatial-temporal monitoring near cold water coral mounds. Just as important is the automatic processing techniques under development that have been implemented to allow species (or categories of species) quantification (i.e., tracking and classification). At the same time, real-time outboard processed three-dimensional (3D) laser scanning has been implemented to increase mission autonomy capability, delivering quantifiable information on habitat features (i.e., for seascape approaches). The first version of platform autonomy has already been tested under controlled conditions with a tethered crawler exploring the vicinity of a cabled stationary instrumented garage. Our vision is that elimination of the tether in combination with inductive battery recharge trough fuel cell technology will facilitate self-sustained long-term autonomous operations over large areas, serving not only the needs of science, but also sub-sea industries like subsea oil and gas, and mining

Ultra-wideband Localization on Manifolds for Autonomous Metal Structure Inspection

A robot that can probabalistically infer its state and uncertainties while exploiting differential geometry is capable of achieving more consistent, more accurate, robust state estimation. It is being proposed that ultra-wideband, a cutting-edge technology, that is also highly unpredictable, can be used to give autonomy to a magnetic-wheeled crawler robot for the application of metal structure inspection. Thus, ultra-wideband technology is evaluated based on its sensitivity to metal surfaces at varying heights, as well as its response to varying grid sizes between receivers in experiments featuring a Turtlebot and an RTK-GPS. Then, a novel methodology for ultra-wideband grid initialization is presented featuring a simulation of a ship hull with an ultra-wideband grid. Finally, a metal structure is considered as a parallelizable manifold with a bivariate b-spline representation, and the matrix exponential correspondence between a Lie group and its Lie algebra for the Special Orthogonal Group is applied within the Extended Kalman Filter framework. These considerations constitute the Manifold Invariant Extended Kalman Filter (M-IEKF), a novel approach to more robust state estimation. The filter is derived, presented, and evaluated in comparison with a modified standard approach: the Manifold-Constrained Extended Kalman Filter (MC-EKF), which uses zero-noise virtual measurements to constrain the state estimate. Then, for a real proof of concept, an experiment using a magnetic-wheeled crawler robot with ultra-wideband localization on a surface consisting of curved metal plates is carried out giving viability to the approach in the real-world application of autonomous metal structure inspection.M.S