An Open Source, Autonomous, Vision-Based Algorithm for Hazard Detection and Avoidance for Celestial Body Landing

Planetary exploration is one of the main goals that humankind has established as a must for space exploration in order to be prepared for colonizing new places and provide scientific data for a better understanding of the formation of our solar system. In order to provide a safe approach, several safety measures must be undertaken to guarantee not only the success of the mission but also the safety of the crew. One of these safety measures is the Autonomous Hazard, Detection, and Avoidance (HDA) sub-system for celestial body landers that will enable different spacecraft to complete solar system exploration. The main objective of the HDA sub-system is to assemble a map of the local terrain during the descent of the spacecraft so that a safe landing site can be marked down. This thesis will be focused on a passive method using a monocular camera as its primary detection sensor due to its form factor and weight, which enables its implementation alongside the proposed HDA algorithm in the Intuitive Machines lunar lander NOVA-C as part of the Commercial Lunar Payload Services technological demonstration in 2021 for the NASA Artemis program to take humans back to the moon. This algorithm is implemented by including two different sources for making decisions, a two-dimensional (2D) vision-based HDA map and a three-dimensional (3D) HDA map obtained through a Structure from Motion process in combination with a plane fitting sequence. These two maps will provide different metrics in order to provide the lander a better probability of performing a safe touchdown. These metrics are processed to optimize a cost function

Probabilistic Surface Characterization for Safe Landing Hazard Detection and Avoidance (HDA)

Apparatuses, systems, computer programs and methods for performing hazard detection and avoidance for landing vehicles are provided. Hazard assessment takes into consideration the geometry of the lander. Safety probabilities are computed for a plurality of pixels in a digital elevation map. The safety probabilities are combined for pixels associated with one or more aim points and orientations. A worst case probability value is assigned to each of the one or more aim points and orientations

Robust vision based slope estimation and rocks detection for autonomous space landers

As future robotic surface exploration missions to other planets, moons and asteroids become more ambitious in their science goals, there is a rapidly growing need to significantly enhance the capabilities of entry, descent and landing technology such that landings can be carried out with pin-point accuracy at previously inaccessible sites of high scientific value. As a consequence of the extreme uncertainty in touch-down locations of current missions and the absence of any effective hazard detection and avoidance capabilities, mission designers must exercise extreme caution when selecting candidate landing sites. The entire landing uncertainty footprint must be placed completely within a region of relatively flat and hazard free terrain in order to minimise the risk of mission ending damage to the spacecraft at touchdown. Consequently, vast numbers of scientifically rich landing sites must be rejected in favour of safer alternatives that may not offer the same level of scientific opportunity. The majority of truly scientifically interesting locations on planetary surfaces are rarely found in such hazard free and easily accessible locations, and so goals have been set for a number of advanced capabilities of future entry, descent and landing technology. Key amongst these is the ability to reliably detect and safely avoid all mission critical surface hazards in the area surrounding a pre-selected landing location. This thesis investigates techniques for the use of a single camera system as the primary sensor in the preliminary development of a hazard detection system that is capable of supporting pin-point landing operations for next generation robotic planetary landing craft. The requirements for such a system have been stated as the ability to detect slopes greater than 5 degrees and surface objects greater than 30cm in diameter. The primary contribution in this thesis, aimed at achieving these goals, is the development of a feature-based,self-initialising, fully adaptive structure from motion (SFM) algorithm based on a robust square-root unscented Kalman filtering framework and the fusion of the resulting SFM scene structure estimates with a sophisticated shape from shading (SFS) algorithm that has the potential to produce very dense and highly accurate digital elevation models (DEMs) that possess sufficient resolution to achieve the sensing accuracy required by next generation landers. Such a system is capable of adapting to potential changes in the external noise environment that may result from intermittent and varying rocket motor thrust and/or sudden turbulence during descent, which may translate to variations in the vibrations experienced by the platform and introduce varying levels of motion blur that will affect the accuracy of image feature tracking algorithms. Accurate scene structure estimates have been obtained using this system from both real and synthetic descent imagery, allowing for the production of accurate DEMs. While some further work would be required in order to produce DEMs that possess the resolution and accuracy needed to determine slopes and the presence of small objects such as rocks at the levels of accuracy required, this thesis presents a very strong foundation upon which to build and goes a long way towards developing a highly robust and accurate solution

Attitude Determination Using Imaging Lidar

The purpose of this study is to determine the attitude of an out of control object using a new technology called lidar (Light Ranging and Detection). As the number of spacecraft continues to grow, it is paramount to introduce a new type of autonomous on-orbit satellite inspection and repair involving docking. Traditional space vision technology is based on video systems. This method is limited by the necessity of operating when the target is illuminated by the sunlight or using its own source of illumination. The use of laser imaging technology offers an elegant solution to these challenges. This approach allows the collection of range data, while scanning the lidar field-of-view together with the transmitted laser beam across the required solid angle. A lidar simulator was implemented to generate point clouds of digital 3D models. This thesis describes methods that can be used to detect features such as edges, boundaries, surfaces and corners in the point cloud. From those features it was possible to define a reference frame and associate it to the object. Observing the evolution of this body frame, the changes in orientation can be deduced in the direction cosine matrix form. It was desired to retrieve angular rates in Euler angle form but since the conversion from rotation matrix to Euler is not a bijection, no satisfying results were obtained. The results are therefore expressed in terms of rotation matrix. It was found that depending on the orientation of the spacecraft the accuracy of the results varied. The results indicate that filtering of the direction cosine matrices might yield good data for determining attitude rates

라이다기반 달 착륙지 선정기법 성능분석 및 쿼드로터를 이용한 성능검증

학위논문 (석사)-- 서울대학교 대학원 : 기계항공공학부, 2016. 2. 박찬국.달착륙선의 라이다 기반 위험회피 착륙시스템은 기본적으로 착륙지형 파라미터인 경사와 험준도로 위험도를 계산하고 위험도가 최소값을 갖는 점을 안전한 착륙 지점으로 선정한다. 이때, 경사와 험준도만을 고려할 경우 라이다 측정오차에 의해 착륙지가 위험요소 근처로 선정될 수 있으며 이는 착륙선에 위협적이다. 이러한 문제를 해결하고 최대한 안전한 착륙지점을 선정하기 위하여 위험상대거리 기반의 위험도를 기존의 지형파리미터 기반의 위험도와 함께 고려하여야 한다. 이 때 위험상대거리 기반 위험도와 지형 특성간의 관계가 위험회피 착륙기법 성능에 미치는 영향에 대한 면밀한 분석이 필요하다. 논문에서는 경사와 험준도 각각에 대한 위험상대거리 기반 위험도가 지형 특성에 따라 착륙지 선정결과에 미치는 영향을 분석하였다. 또한 시뮬레이션과 삼차원 뎁스 카메라를 장착한 쿼드로터 기반 실험을 통해 두 가지 위험상대거리를 동시에 고려하였을 때 가장 좋은 위험회피 착륙 성능을 나타냄을 확인하였다.A Lidar-based safe landing technique fundamentally estimates the slope and roughness of lunar terrain using three-dimensional range measurement, and designates the landing target that terrain parameters have the minimum value. In the case of hazard cost calculation only with the parameters, however, the selected target can be located near hazards on the surface such as craters, rocks and slants which are able to damage lunar lander. Therefore, relative distance to hazard should also be considered simultaneously in order to choose the landing point which is not only gentle but also as far from hazard as possible. In this case, the effect of terrain condition on the safe landing performance should be closely analyzed since it is influenced by the level of the terrain parameters. In this thesis, the relation between terrain condition and the performance of landing site designation with the relative distance to hazard was analyzed, and it was confirmed that the best landing target can be selected by the weighted sum of the parameters and relative distance to hazard reflecting the terrain characteristics of landing area through simulations and experiments with a quadrotor with TOF camera.Chapter 1.Introduction 1 1.1 Motivation and Background 1 1.2 Objectives and Contributions 6 1.3 Organization 7 Chapter 2. Hazard Detection and Avoidance Landing 8 2.1 Introduction 8 2.2 Uniform DEM generation 9 2.3 Lunar Terrain Parameter Estimation 12 2.3.1 Slope and Roughenss 12 2.3.2 Terrain Approximation 12 2.4 Hazard Cost Basd on Lunar Terrain Parameters 16 2.5 Hazard Cost Basd on Relative Distance to Hazard 16 2.6 Hazard Cost Integration 18 Chapter 3. Performance Analysis of Landing Point Designation 20 3.1 Lidar Model 20 3.2 Simulation of Safe Landing Point Designation 23 3.2.1 Lidar Measurement Generation 25 3.2.2 Uniform DEM Generation 26 3.2.3 Terrain Parameter and Integrated Hazard Cost Calculation 28 3.3 Performance Analysis of Landing Target Selection 30 3.3.1 Landing Point Designation with respect to a terrain with Extreme Slope Change 30 3.3.2 Landing Point Designation with respect to a Rough Terrain 31 3.3.3 Landing Point Designation with respect to Lunar Terrain 33 3.4 Comparison of the HDA Performance according to Terrain Condition 35 3.5 Summary 38 Chapter 4. HDA Experiment Using Quadrotor Equipped with TOF Camera 39 4.1 Introduction 39 4.2 Experiment Environment 40 4.3 HDA Experiment Based on Quadrotor Autopilot system with TOF Camera 44 4.4 Experiment Result of Quadrotor Safe Landing Based on HDA 47 4.5 Summary 48 Chapter 5. Conclusions 49 Bibliography 51 국문초록 53Maste

Real-time landing place assessment in man-made environments

We propose a novel approach to the real-time landing site detection and assessment in unconstrained man-made environments using passive sensors. Because this task must be performed in a few seconds or less, existing methods are often limited to simple local intensity and edge variation cues. By contrast, we show how to efficiently take into account the potential sites' global shape, which is a critical cue in man-made scenes. Our method relies on a new segmentation algorithm and shape regularity measure to look for polygonal regions in video sequences. In this way, we enforce both temporal consistency and geometric regularity, resulting in very reliable and consistent detections. We demonstrate our approach for the detection of landable sites such as rural fields, building rooftops and runways from color and infrared monocular sequences significantly outperforming the state-of-the-art

Real-time landing place assessment in man-made environments

We propose a novel approach to the real-time landing site detection and assessment in unconstrained man-made environments using passive sensors. Because this task must be performed in a few seconds or less, existing methods are often limited to simple local intensity and edge variation cues. By contrast, we show how to efficiently take into account the potential sites' global shape, which is a critical cue in man-made scenes. Our method relies on a new segmentation algorithm and shape regularity measure to look for polygonal regions in video sequences. In this way, we enforce both temporal consistency and geometric regularity, resulting in very reliable and consistent detections. We demonstrate our approach for the detection of landable sites such as rural fields, building rooftops and runways from color and infrared monocular sequences significantly outperforming the state-of-the-art