TRADE: Object Tracking with 3D Trajectory and Ground Depth Estimates for UAVs

We propose TRADE for robust tracking and 3D localization of a moving target in cluttered environments, from UAVs equipped with a single camera. Ultimately TRADE enables 3d-aware target following. Tracking-by-detection approaches are vulnerable to target switching, especially between similar objects. Thus, TRADE predicts and incorporates the target 3D trajectory to select the right target from the tracker's response map. Unlike static environments, depth estimation of a moving target from a single camera is a ill-posed problem. Therefore we propose a novel 3D localization method for ground targets on complex terrain. It reasons about scene geometry by combining ground plane segmentation, depth-from-motion and single-image depth estimation. The benefits of using TRADE are demonstrated as tracking robustness and depth accuracy on several dynamic scenes simulated in this work. Additionally, we demonstrate autonomous target following using a thermal camera by running TRADE on a quadcopter's board computer

Motivations and Preliminary Design for Mid-Air Deployment of a Science Rotorcraft on Mars

Mid-Air Deployment (MAD) of a rotorcraft during Entry, Descent and Landing (EDL) on Mars eliminates the need to carry a propulsion or airbag landing system. This reduces the total mass inside the aeroshell by more than 100 kg and simplifies the aeroshell architecture. MAD’s lighter and simpler design is likely to bring the risk and cost associated with the mission down. Moreover, the lighter entry mass enables landing in the Martian highlands, at elevations inaccessible to current EDL technologies. This paper proposes a novel MAD concept for a Mars helicopter. We suggest a minimum science payload package to perform relevant science in the highlands. A variant of the Ingenuity helicopter is proposed to provide increased deceleration during MAD, and enough lift to fly the science payload in the highlands. We show in simulation that the lighter aeroshell results in a lower terminal velocity (30 m/s) at the end of the parachute phase of the EDL, and at higher altitudes than other approaches. After discussing the aerodynamics, controls, guidance, and mechanical challenges associated with deploying at such speed, we propose a backshell architecture that addresses them to release the helicopter in the safest conditions. Finally, we implemented the helicopter model and aerodynamic descent perturbations in the JPL Dynamics and Real-Time Simulation (DARTS)framework. Preliminary performance evaluation indicates landing and helicopter operations can be achieved up to +5 km MOLA (Mars Orbiter Laser Altimeter reference)

Vision-based navigation for pinpoint planetary landing on any relief

Cette thèse présente Lion, un système de navigation utilisant des informations visuelles et inertielles pour l’atterrissage planétaire de précision. Lion est conçu pour voler au-dessus de n’importe quel type de terrain, plat ou accidenté, et ne fait pas d’hypothèse sur sa topographie. Faire un atterrir un véhicule d’exploration planétaire autonome à moins de 100 mètres d’un objectif cartographié est un défi pour la navigation. Les approches basées vision tentent d’apparrier des détails 2D détectés dans une image avec des amers 3D cartographiés pour atteindre la précision requise. Lion utilise de façon serrée des mesures venant d’un nouvel algorithme d’appariement imagecarteafin de mettre à jour l’état d’un filtre de Kalman étendu intégrant des données inertielles. Le traitement d’image utilise les prédictions d’état et de covariance du filtre dans le but de déterminer les régions et échelles d’extraction dans l’image où trouver des amers non-ambigus. Le traitement local par amer de l’échelle image permet d’améliorer de façon significative la répétabilité de leur détection entre l’image de descente et l’image orbitale de référence. Nous avons également conçu un banc d’essai matériel appelé Visilab pour évaluer Lion dans des conditions représentatives d’une mission lunaire. L’observabilité des performances de navigation absolue dans Visilab est évaluée à l’aide d’un nouveau modèle d’erreur. Les performances du systèmes sont évaluées aux altitudes clés de la descente, en terme de précision de navigation et robustesse au changement de capteurs ou d’illumination, inclinaison de la caméra de descente, et sur différents types de relief. Lion converge jusqu’à une erreur de 4 mètres de moyenne et 47 mètres de dispersion 3 RMS à 3 kilomètres d’altitude à l’échelle.This thesis introduces Lion, a vision-aided inertial navigation system for pinpoint planetary landing. Lion can fly over any type of terrain, whatever its topography, flat or not. Landing an autonomous spacecraft within 100 meters of a mapped target is a navigation challenge in planetary exploration. Vision-based approaches attemptto pair 2D features detected in camera images with 3D mapped landmarks to reach the required precision. Lion tightly uses measurements from a novel image-tomapmatcher in order to update the state of an extended Kalman filter propagated with inertial data. The image processing uses the state and covariance predictionsfrom the filter to determine the regions and extraction scales in which to search for non-ambiguous landmarks in the image. The individual image scale managementprocess per landmark greatly improves the repeatability rate between the map and descent images. We also designed a lunar-representative optical test bench called Visilab to test Lion on. The observability of absolute navigation performances in Visilab is evaluated with a novel error budget model. Finally, the system performances areevaluated at the key altitudes of a lunar landing, in terms of accuracy and robustness to sensor or illumination changes, off-nadir camera angle, and non-planar topography. We demonstrate error convergence down to a mean of 4 meters and a 3-RMS dispersion of 47 meters at 3 kilometers of altitude in hardware conditions at scale

Optimizing Terrain Mapping and Landing Site Detection for Autonomous UAVs

The next generation of Mars rotorcrafts requires on-board autonomous hazard avoidance landing. To this end, this work proposes a system that performs continuous multi-resolution height map reconstruction and safe landing spot detection. Structure-from-Motion measurements are aggregated in a pyramid structure using a novel Optimal Mixture of Gaussians formulation that provides a comprehensive uncertainty model. Our multiresolution pyramid is built more efficiently and accurately than past work by decoupling pyramid filling from the measurement updates of different resolutions. To detect the safest landing location, after an optimized hazard segmentation, we use a mean shift algorithm on multiple distance transform peaks to account for terrain roughness and uncertainty. The benefits of our contributions are evaluated on real and synthetic flight data.Comment: Accepted to ICRA 202

Estimating vector magnitude from its direction and derivative, with application to bearing-only SLAM filter problem

For some problems, such as monocular visual odometry (VO), vector measurements are given with unknown magnitude. In VO, the magnitude can be found by recognizing features with known position, or with an extra sensor such as an altimeter. This article presents a nonlinear observer that uses the derivative of the vector as an additional measurement for estimating the magnitude of a vector. For the VO example, this means that the velocity can be estimated by fusing the normalized velocity vector with acceleration measurements. The observer exploits the fact that the dynamics of the normalized vector is dependent on the magnitude of the vector. The observer employs methods from nonlinear/adaptive estimation; filters the unit vector on the unit sphere, and retrieves the magnitude of the vector. The observer is shown to be uniformly semi-globally asymptotically (USGAS) stable and uniformly exponentially stable (UES) in a defined region. The observer is applied to the bearing-only SLAM filter problem as an example

Globally Stable Velocity Estimation Using Normalized Velocity Measurement

The problem of estimating velocity from a monocular camera and calibrated inertial measurement unit (IMU) measurements is revisited. For the presented setup, it is assumed that normalized velocity measurements are available from the camera. By applying results from nonlinear observer theory, we present velocity estimators with proven global stability under defined conditions, and without the need to observe features from several camera frames. Several nonlinear methods are compared with each other, also against an extended Kalman filter (EKF), where the robustness of the nonlinear methods compared with the EKF are demonstrated in simulations and experiments

Challenges of pin-point landing for planetary landing: the LION absolute vision-based navigation approach and experimental results

After ExoMars in 2016 and 2018, future ESA missions to Mars, the Moon, or asteroids will require safe and pinpoint precision landing capabilities, with for example a specified accuracy of typically 100 m at touchdown for a Moon landing. The safe landing requirement can be met thanks to state-of-the-art Terrain-Relative Navigation (TRN) sensors such as Wide-Field-of-View vision-based navigation cameras (VBNC), with appropriate hazard detection and avoidance algorithms. To reach the pinpoint precision requirement, on-board absolute navigation with respect to the landing site is mandatory, with a typical accuracy better than 100 m at touchdown for a Lunar mission, or below 10 km at entry interface for a Mars landing missions. In this paper, we present the validation approach and experimental results of an Absolute Visual Navigation system (AVN) known as Lion. The Lion functional architecture will be first presented, as well as the implemented incremental validation and verification approach ; experimental set-up and end-to-end tests results will be summarized. Finally, way forward and lessons learned will be discussed

Multi-Resolution Elevation Mapping and Safe Landing Site Detection with Applications to Planetary Rotorcraft

In this paper, we propose a resource-efficient approach to provide an autonomous UAV with an on-board perception method to detect safe, hazard-free landing sites during flights over complex 3D terrain. We aggregate 3D measurements acquired from a sequence of monocular images by a Structure-from-Motion approach into a local, robot-centric, multi-resolution elevation map of the overflown terrain, which fuses depth measurements according to their lateral surface resolution (pixel-footprint) in a probabilistic framework based on the concept of dynamic Level of Detail. Map aggregation only requires depth maps and the associated poses, which are obtained from an on-board Visual Odometry algorithm. An efficient landing site detection method then exploits the features of the underlying multi-resolution map to detect safe landing sites based on slope, roughness, and quality of the reconstructed terrain surface. The evaluation of the performance of the mapping and landing site detection modules are analyzed independently and jointly in simulated and real-world experiments in order to establish the efficacy of the proposed approach

Tillage, Manure, and Biochar Short-Term Effects on Soil Characteristics in Forage Systems

Manure, a globally used soil amendment, can contribute to excessive N and P runoff, leading to water pollution. Biochar (BC) shows promise in mitigating nutrient loss by retaining soil nutrients. However, there is limited research exploring the combined effects of tillage practices, biochar, manure, forage crops, and soil types on soil nutrient characteristics in a single field study. Our objectives are to determine if, in North Central Texas, differing soil types, soil amendments, forage crops, and tillage practices affect soil nutrients when applied short term, and whether correlations exist among soil nutrient characteristics as affected by soil amendments, tillage practices, and the presence of forage crops. The study encompasses three field sites with five factors, including soil types, manure rates, biochar rates, tillage practices, and forage crop types. Soil samples were assayed for pH, electrical conductivity (EC), macronutrients, and micronutrients. Data analyses involved variance analysis, Fisher’s tests, and Pearson’s correlations using R in Rstudio (the IDE). Microplots treated with manure (average 2.16 ppm) retained 60% greater average nitrate levels at the end of the growing season than those treated with a synthetic fertilizer (average 1.35 ppm) (p ≤ 0.05). Moderate and strong correlations were observed between EC and S (r (106) = 0.43, p p p p < 0.001 in sandy loam soil) across different soil types. Soil type (texture) emerged as the primary influencing factor on plant-available soil nutrients and characteristics, followed by manure application and tillage practices. The impact of BC and forage crop type varied depending on other experimental factors. Understanding the influence of soil type, amendment application, and tillage on soil nutrient characteristics can guide sustainable forage production practices and soil nutrient management strategies