NOAH-H, a deep-learning, terrain classification system for Mars: Results for the ExoMars Rover candidate landing sites

In this investigation a deep learning terrain classification system, the “Novelty or Anomaly Hunter – HiRISE” (NOAH-H), was used to classify High Resolution Imaging Science Experiment (HiRISE) images of Oxia Planum and Mawrth Vallis. A set of ontological classes was developed that covered the variety of surface textures and aeolian bedforms present at both sites. Labelled type-examples of these classes were used to train a Deep Neural Network (DNN) to perform semantic segmentation in order to identify these classes in further HiRISE images. This contribution discusses the methods and results of the study from a geomorphologists perspective, providing a case study applying machine learning to a landscape classification task. Our aim is to highlight considerations about how to compile training datasets, select ontological classes, and understand what such systems can and cannot do. We highlight issues that arise when adapting a traditional planetary mapping workflow to the production of training data. We discuss both the pixel scale accuracy of the model, and how qualitative factors can influence the reliability and usability of the output. We conclude that “landscape level” reliability is critical for the use of the output raster by humans. The output can often be more useful than pixel scale accuracy statistics would suggest, however the product must be treated with caution, and not considered a final arbiter of geological origin. A good understanding of how and why the model classifies different landscape features is vital to interpreting it reliably. When used appropriately the classified raster provides a good indication of the prevalence and distribution of different terrain types, and informs our understanding of the study areas. We thus conclude that it is fit for purpose, and suitable for use in further work

Jezero crater, Mars: application of the deep learning NOAH-H terrain classification system

We applied a deep learning terrain classification system, the ‘Novelty or Anomaly Hunter – HiRISE’ (NOAH-H), originally developed for the ExoMars landing sites in Oxia Planum and Mawrth Vallis, to the Mars 2020 Perseverance rover landing site in Jezero crater. NOAH-H successfully classified the terrain in four HiRISE images of Jezero even though the landforms in the Jezero study area were slightly different from those in the training dataset. We mosaicked the NOAH-H classified rasters and compared them with a manually generated photogeological map, and with Perseverance rover and Ingenuity helicopter images. We find that grouped NOAH-H classes correspond well with the humanmade map and that individual classes are corroborated by the available ground-truth images. We conclude that our NOAH-H products can be refined for feeding into traversability analysis of the ExoMars Rosalind Franklin rover landing site at Oxia Planum and that they can also be used to aid the photogeological mapping process

Using the ERGO Framework in a Planetary and an Orbital Scenario

The European Robotic Goal-Oriented Autonomous Controller ERGO [1] is one of the six space robotic projects in the frame of the PERASPERA SRC [2]. Its main objective is to provide an autonomous framework for future space robots that will be able to perform its activities without the need of constant human supervision and control. Future space missions, in particular those aimed at Deep Space or planetary exploration, such as Exomars [3], or Mars2020 [4] demand a greater level of autonomy. The concept of autonomy applies here to a whole set of operations to be performed on-board without human supervision; for instance, a Martian rover has to avoid getting stuck in the sand while traversing, autonomously recharge its batteries periodically, and communicate with Earth occasionally each sol[5]. Additionally, it will need tobe ableto detect serendipitous events (e.g. a rock that has a specific property). A deep space probe[6] has to take the right measurements to approach an asteroid, and due to the latency of the communication with Ground, these measurements need to be taken autonomously on board. Orbital space missions have already successfully applied autonomy concepts on board, in particular for autonomous event detection and on-board activities planning [7]. In ERGO we provide a framework for autonomy aimed to cover a wide set of a capabilities, ranging from reactive capabilities (i.e. capabilities that demand a quick response) to deliberative capabilities (that consider different courses of actions, and evaluate among the different possibilities the best alternative). This paper will discuss the process of the design of robotic systems using the paradigm provided by this framework applied to two different scenarios: a Sample Fetching Rover (SFR), and also an On-Orbit Servicing mission, where a damaged spacecraft can have one or several of its modules replaced autonomously by a servicer spacecraft. We will describe the methodology, the main problems found, the design decisions taken to overcome these problems, as well as an overview of the final design of both systems

Oxia Planum, Mars, classified using the NOAH-H deep-learning terrain classification system

We present a map of Oxia Planum, Mars, the landing site for the ExoMars Rover. This shows surface texture and aeolian bedform distribution, classified using a deep learning (DL) system. A hierarchical classification scheme was developed, categorising the surface textures observed at the site. This was then used to train a DL network, the ‘Novelty or Anomaly Hunter – HiRISE’ (NOAH-H). The DL applied the classification scheme across a wider area than could have been mapped manually. The result showed strong agreement with human-mapped areas reserved for validation. The resulting product is presented in two ways, representing the two principle levels of the classification scheme. ‘Descriptive classes’ are purely textural in nature, making them compatible with a machine learning approach. These are then combined into ‘interpretive groups’, broader thematic classes, which provide an interpretation of the landscape. This step allows for a more intuitive analysis of the results by human users

Oxia Planum, Mars, classified using the NOAH-H deep-learning terrain classification system

ABSTRACTWe present a map of Oxia Planum, Mars, the landing site for the ExoMars Rover. This shows surface texture and aeolian bedform distribution, classified using a deep learning (DL) system. A hierarchical classification scheme was developed, categorising the surface textures observed at the site. This was then used to train a DL network, the ‘Novelty or Anomaly Hunter – HiRISE’ (NOAH-H). The DL applied the classification scheme across a wider area than could have been mapped manually. The result showed strong agreement with human-mapped areas reserved for validation. The resulting product is presented in two ways, representing the two principle levels of the classification scheme. ‘Descriptive classes’ are purely textural in nature, making them compatible with a machine learning approach. These are then combined into ‘interpretive groups’, broader thematic classes, which provide an interpretation of the landscape. This step allows for a more intuitive analysis of the results by human users

Mawrth Vallis, Mars, classified using the NOAH-H deep-learning terrain classification system

A deep learning (DL) terrain classification system, the Novelty and Anomaly Hunter – HiRISE (NOAH-H) was used to produce a terrain map of Mawrth Vallis, Mars. With it, we digitised the extent and distribution of transverse aeolian ridges (TARs), a common type of martian aeolian bedform. We present maps of the site, classifying terrain into descriptive classes and interpretive groups. TAR density maps are calculated, and the network output is compared to a manually produced map of TAR density, highlighting the differences in approach and results between these methods. Even when mapping on a small scale, humans must divide the terrain into coherent patches in order to map a large area in a reasonable time frame. Conversely, the speed of DL systems enables mapping on the pixel scale, producing a more detailed product, but one which is also “noisier”, and less immediately informative. There are pros and cons to both approaches

The ERGO Framework and its Use in Planetary/Orbital Scenarios

ERGO (European Robotic Goal-Oriented Autonomous Controller) is one of the six space robotic projects in the frame of the first call of the PERASPERA SRC. ERGO is aimed to future space missions, in which space robots will require a higher level of autonomy (e.g. Exomars or Mars2020). As a framework, ERGO provides a set of components that can be reused and tailored for robots space missions (Orbital, Deep Space or Planetary Explorations) in which the on-board system has to work autonomously, performing complex operations in hazardous environments without human intervention. The concept of autonomy can be applied to a whole set of operations to be performed on-board with no human supervision, such as Martian exploration rovers, deep space probes, or in- orbit assembly robots. In the last decades, the advantages of increasing the level of autonomy in spacecraft have been demonstrated in planetary rovers. At the same time, orbital space missions have already successfully applied autonomy concepts on board, in particular for autonomous event detection and on-board activities planning. ERGO provides a framework for on-board autonomy systems based on a specific paradigm aimed to facilitate an easy integration and/or expansion covering future mission needs; by using this paradigm, both reactive and deliberative capabilities can be orchestrated on-board. In ERGO, deliberative capabilities are provided via AI techniques: automated planning and machine-learning based vision systems. ERGO also provides a set of tools for developing safety-critical space mission applications and FDIR systems. Moreover, specific components for motion planning, path planning, hazard avoidance and trajectory control are also part of the framework. Finally, ERGO is integrated with the TASTE middleware. All ERGO components are now being tested in an orbital and a planetary scenario. This paper discusses the ERGO components, its main characteristics, and how they have been applied to an orbital and a planetary scenario. It provides an overview of the evolution of the ERGO system; its main components and the future extensions planned for it