Recent Advances in Image Restoration with Applications to Real World Problems

In the past few decades, imaging hardware has improved tremendously in terms of resolution, making widespread usage of images in many diverse applications on Earth and planetary missions. However, practical issues associated with image acquisition are still affecting image quality. Some of these issues such as blurring, measurement noise, mosaicing artifacts, low spatial or spectral resolution, etc. can seriously affect the accuracy of the aforementioned applications. This book intends to provide the reader with a glimpse of the latest developments and recent advances in image restoration, which includes image super-resolution, image fusion to enhance spatial, spectral resolution, and temporal resolutions, and the generation of synthetic images using deep learning techniques. Some practical applications are also included

Fusion of Landsat and Worldview Images

Pansharpened Landsat images have 15 m spatial resolution with 16-day revisit periods. On the other hand, Worldview images have 0.5 m resolution after pansharpening but the revisit times are uncertain. We present some preliminary results for a challenging image fusion problem that fuses Landsat and Worldview (WV) images to yield a high temporal resolution image sequence at the same spatial resolution of WV images. Since the spatial resolution between Landsat and Worldview is 30 to 1, our preliminary results are mixed in that the objective performance metrics such as peak signal-to-noise ratio (PSNR), correlation coefficient (CC), etc. sometimes showed good fusion performance, but at other times showed poor results. This indicates that more fusion research is still needed in this niche application

Detection and Analysis of Martian Low-Temperature Geochemistry

The history of Mars is encoded in the geochemistry of ancient sedimentary Martian rocks and secondary phases. Recent landed missions have provided unparalleled datasets with which to investigate this geochemistry. Accordingly, this thesis is concerned primarily with the in-situ analysis of low-temperature Martian geochemical processes by landed missions, and the attendant Earth-based studies which enrich those in-situ investigations. There are five main studies reported here. The first is an analysis of a Mars-analog environment on Earth. Datasets similar to those that will be produced by the upcoming Mars-2020 rover are used to evaluate the ability of the rover to reconstruct a known paleoenvironment, to identify reference datasets that require further development, and to suggest operational modes that most efficiently use the rover’s resources. The second study is an in-situ noble gas analysis using the SAM instrument on the Curiosity rover to investigate a jarosite-containing sample using a two-step heating analysis for K-Ar dating. The jarosite likely formed at 2.12±0.36 Ga while plagioclase in the sample formed at 4.07 ± 0.63 Ga, indicating that liquid water interactions continued in Gale crater well past the end of the Hesperian period. The following chapter details another noble gas analysis, focusing on cosmogenic dating of surface exposure. In contrast to <100 Ma exposure ages observed at the floor of Gale crater, exposure ages exceeding 1 Ga are detected on the flanks of Mount Sharp. These ages indicate Mount Sharp formed during the Hesperian and has been largely unchanged in the intervening 3.1 Ga. The next study is a reevaluation of the data used to identify the presence of perchlorate in Gale crater. These data suggest that perchlorate is indeed present, but that it must be Amazonian in age, suggesting that rare surface wetting events have caused leaching of this soluble ion into the bedrock. The final study reports the development of a technique for measuring the isotopes in perchlorate using Orbitrap mass spectrometry on Earth, allowing investigation of the formation processes that impact the chlorine isotope ratio of this molecule on both Earth and Mars

On Multi-Resolution 3D Orbital Imagery and Visualisation for Mars Geological Analysis

Mars Science Laboratory has revealed a dynamic history of water as the rover has ascended the mysterious Mount Sharp in Gale crater. Because rovers only “see” their local environment, planetary scientists rely on satellite-based orbital imagery to understand the regional geology of Gale crater. However, orbital imagery is map-view—viewed from above, lacking perspective—which presents challenges to interpretation of stratigraphy. 3D visualisation is an emerging opportunity to study orbital images in more intuitive, field-like environments, but has had limited application to Mars. In this work, I formulate and analyse 3D orbital imagery over Gale crater, Mars to investigate the stratigraphy of Mount Sharp 700 m above and 40 km away from MSL. First, I process orbital imagery from the HRSC, CTX, and HiRISE cameras into 3D digital terrain models (DTMs). I then co-register and evaluate these DTMs using statistical tools and existing products to build a new, validated, multi-resolution basemap tied down to MOLA. Sakarya Vallis, a 400-m deep canyon on Mount Sharp, was then analysed in a 3D environment at 1 m/px. From measurements of exposed rock layers, I construct cross-sections, stratigraphic logs, and a geological unit map to capture this geology. Seven geological units are interpreted across 1 km of exposure, varying in thicknesses (10–174 m) and dips (3–12º). These units may reveal a cyclic depositional environment; a progradational sequence and channel; and unconformities. This work therefore suggests two periods of sub-aqueous deposition in this region during the Late Noachian to Early Hesperian. These results further provide geological context of Gale crater as MSL ascends Mount Sharp, and future inputs for palaeoenvironmental models of Gale crater

Engineering a Low-Cost Remote Sensing Capability for Deep-Space Applications

Systems engineering (SE) has been a useful tool for providing objective processes to breaking down complex technical problems to simpler tasks, while concurrently generating metrics to provide assurance that the solution is fit-for-purpose. Tailored forms of SE have also been used by cubesat mission designers to assist in reducing risk by providing iterative feedback and key artifacts to provide managers with the evidence to adjust resources and tasking for success. Cubesat-sized spacecraft are being planned, built and in some cases, flown to provide a lower-cost entry point for deep-space exploration. This is particularly important for agencies and countries with lower space exploration budgets, where specific mission objectives can be used to develop tailored payloads within tighter constraints, while also returning useful scientific results or engineering data. In this work, a tailored SE tradespace approach was used to help determine how a 6 unit (6U) cubesat could be built from commercial-off-the-shelf (COTS)-based components and undertake remote sensing missions near Mars or near-Earth Asteroids. The primary purpose of these missions is to carry a hyperspectral sensor sensitive to 600-800nm wavelengths (hereafter defined as “red-edge”), that will investigate mineralogy characteristics commonly associated with oxidizing and hydrating environments in red-edge. Minerals of this type remain of high interest for indicators of present or past habitability for life, or active geologic processes. Implications of operating in a deep-space environment were considered as part of engineering constraints of the design, including potential reduction of available solar energy, changes in thermal environment and background radiation, and vastly increased communications distances. The engineering tradespace analysis identified realistic COTS options that could satisfy mission objectives for the 6U cubesat bus while also accommodating a reasonable degree of risk. The exception was the communication subsystem, in which case suitable capability was restricted to one particular option. This analysis was used to support an additional trade investigation into the type of sensors that would be most suitable for building the red-edge hyperspectral payload. This was in part constrained by ensuring not only that readily available COTS sensors were used, but that affordability, particularly during a geopolitical environment that was affecting component supply surety and access to manufacturing facilities, was optimized. It was found that a number of sensor options were available for designing a useful instrument, although the rapid development and life-of-type issues with COTS sensors restricted the ability to obtain useful metrics on their performance in the space environment. Additional engineering testing was conducted by constructing hyperspectral sensors using sensors popular in science, technology, engineering and mathematics (STEM) contexts. Engineering and performance metrics of the payload containing the sensors was conducted; and performance of these sensors in relevant analogous environments. A selection of materials exhibiting spectral phenomenology in the red-edge portion of the spectrum was used to produce metrics on the performance of the sensors. It was found that low-cost cameras were able to distinguish between most minerals, although they required a wider spectral range to do so. Additionally, while Raspberry Pi cameras have been popular with scientific applications, a low-cost camera without a Bayer filter markedly improved spectral sensitivity. Consideration for space-environment testing was also trialed in additional experiments using high-altitude balloons to reach the near-space environment. The sensor payloads experienced conditions approximating the surface of Mars, and results were compared with Landsat 7, a heritage Earth sensing satellite, using a popular vegetation index. The selected Raspberry Pi cameras were able to provide useful results from near-space that could be compared with space imagery. Further testing incorporated comparative analysis of custom-built sensors using readily available Raspberry Pi and astronomy cameras, and results from Mastcam and Mastcam/z instruments currently on the surface of Mars. Two sensor designs were trialed in field settings possessing Mars-analogue materials, and a subset of these materials were analysed using a laboratory-grade spectro-radiometer. Results showed the Raspberry Pi multispectral camera would be best suited for broad-scale indications of mineralogy that could be targeted by the pushbroom sensor. This sensor was found to possess a narrower spectral range than the Mastcam and Mastcam/z but was sensitive to a greater number of bands within this range. The pushbroom sensor returned data on spectral phenomenology associated with attributes of Minerals of the type found on Mars. The actual performance of the payload in appropriate conditions was important to provide critical information used to risk reduce future designs. Additionally, the successful outcomes of the trials reduced risk for their application in a deep space environment. The SE and practical performance testing conducted in this thesis could be developed further to design, build and fly a hyperspectral sensor, sensitive to red-edge wavelengths, on a deep-space cubesat mission. Such a mission could be flown at reasonable cost yet return useful scientific and engineering data

High-performance hardware accelerators for image processing in space applications

Mars is a hard place to reach. While there have been many notable success stories in getting probes to the Red Planet, the historical record is full of bad news. The success rate for actually landing on the Martian surface is even worse, roughly 30%. This low success rate must be mainly credited to the Mars environment characteristics. In the Mars atmosphere strong winds frequently breath. This phenomena usually modifies the lander descending trajectory diverging it from the target one. Moreover, the Mars surface is not the best place where performing a safe land. It is pitched by many and close craters and huge stones, and characterized by huge mountains and hills (e.g., Olympus Mons is 648 km in diameter and 27 km tall). For these reasons a mission failure due to a landing in huge craters, on big stones or on part of the surface characterized by a high slope is highly probable. In the last years, all space agencies have increased their research efforts in order to enhance the success rate of Mars missions. In particular, the two hottest research topics are: the active debris removal and the guided landing on Mars. The former aims at finding new methods to remove space debris exploiting unmanned spacecrafts. These must be able to autonomously: detect a debris, analyses it, in order to extract its characteristics in terms of weight, speed and dimension, and, eventually, rendezvous with it. In order to perform these tasks, the spacecraft must have high vision capabilities. In other words, it must be able to take pictures and process them with very complex image processing algorithms in order to detect, track and analyse the debris. The latter aims at increasing the landing point precision (i.e., landing ellipse) on Mars. Future space-missions will increasingly adopt Video Based Navigation systems to assist the entry, descent and landing (EDL) phase of space modules (e.g., spacecrafts), enhancing the precision of automatic EDL navigation systems. For instance, recent space exploration missions, e.g., Spirity, Oppurtunity, and Curiosity, made use of an EDL procedure aiming at following a fixed and precomputed descending trajectory to reach a precise landing point. This approach guarantees a maximum landing point precision of 20 km. By comparing this data with the Mars environment characteristics, it is possible to understand how the mission failure probability still remains really high. A very challenging problem is to design an autonomous-guided EDL system able to even more reduce the landing ellipse, guaranteeing to avoid the landing in dangerous area of Mars surface (e.g., huge craters or big stones) that could lead to the mission failure. The autonomous behaviour of the system is mandatory since a manual driven approach is not feasible due to the distance between Earth and Mars. Since this distance varies from 56 to 100 million of km approximately due to the orbit eccentricity, even if a signal transmission at the light speed could be possible, in the best case the transmission time would be around 31 minutes, exceeding so the overall duration of the EDL phase. In both applications, algorithms must guarantee self-adaptability to the environmental conditions. Since the Mars (and in general the space) harsh conditions are difficult to be predicted at design time, these algorithms must be able to automatically tune the internal parameters depending on the current conditions. Moreover, real-time performances are another key factor. Since a software implementation of these computational intensive tasks cannot reach the required performances, these algorithms must be accelerated via hardware. For this reasons, this thesis presents my research work done on advanced image processing algorithms for space applications and the associated hardware accelerators. My research activity has been focused on both the algorithm and their hardware implementations. Concerning the first aspect, I mainly focused my research effort to integrate self-adaptability features in the existing algorithms. While concerning the second, I studied and validated a methodology to efficiently develop, verify and validate hardware components aimed at accelerating video-based applications. This approach allowed me to develop and test high performance hardware accelerators that strongly overcome the performances of the actual state-of-the-art implementations. The thesis is organized in four main chapters. Chapter 2 starts with a brief introduction about the story of digital image processing. The main content of this chapter is the description of space missions in which digital image processing has a key role. A major effort has been spent on the missions in which my research activity has a substantial impact. In particular, for these missions, this chapter deeply analizes and evaluates the state-of-the-art approaches and algorithms. Chapter 3 analyzes and compares the two technologies used to implement high performances hardware accelerators, i.e., Application Specific Integrated Circuits (ASICs) and Field Programmable Gate Arrays (FPGAs). Thanks to this information the reader may understand the main reasons behind the decision of space agencies to exploit FPGAs instead of ASICs for high-performance hardware accelerators in space missions, even if FPGAs are more sensible to Single Event Upsets (i.e., transient error induced on hardware component by alpha particles and solar radiation in space). Moreover, this chapter deeply describes the three available space-grade FPGA technologies (i.e., One-time Programmable, Flash-based, and SRAM-based), and the main fault-mitigation techniques against SEUs that are mandatory for employing space-grade FPGAs in actual missions. Chapter 4 describes one of the main contribution of my research work: a library of high-performance hardware accelerators for image processing in space applications. The basic idea behind this library is to offer to designers a set of validated hardware components able to strongly speed up the basic image processing operations commonly used in an image processing chain. In other words, these components can be directly used as elementary building blocks to easily create a complex image processing system, without wasting time in the debug and validation phase. This library groups the proposed hardware accelerators in IP-core families. The components contained in a same family share the same provided functionality and input/output interface. This harmonization in the I/O interface enables to substitute, inside a complex image processing system, components of the same family without requiring modifications to the system communication infrastructure. In addition to the analysis of the internal architecture of the proposed components, another important aspect of this chapter is the methodology used to develop, verify and validate the proposed high performance image processing hardware accelerators. This methodology involves the usage of different programming and hardware description languages in order to support the designer from the algorithm modelling up to the hardware implementation and validation. Chapter 5 presents the proposed complex image processing systems. In particular, it exploits a set of actual case studies, associated with the most recent space agency needs, to show how the hardware accelerator components can be assembled to build a complex image processing system. In addition to the hardware accelerators contained in the library, the described complex system embeds innovative ad-hoc hardware components and software routines able to provide high performance and self-adaptable image processing functionalities. To prove the benefits of the proposed methodology, each case study is concluded with a comparison with the current state-of-the-art implementations, highlighting the benefits in terms of performances and self-adaptability to the environmental conditions

Mars, invisible vision and the virtual landscape: immersive encounters with contemporary rover images

How do contemporary imaging devices and the forms in which images are displayed affect our perception of Mars? How are scientists and engineers visually exploring, experiencing and navigating this uninhabitable terrain? Can we better understand this virtual landscape through immersive imaging techniques, or are these simply illusions? At what point does the glitch invade these immersive spaces, throwing us back into the realm of the image? And finally, can the glitch be seen as a method towards another kind of visibility, enabling us to ‘see’ and encounter Mars in productive ways? Through the analysis of contemporary representations of the Martian terrain, Mars, Invisible Vision and the Virtual Landscape: Immersive Encounters with Contemporary Rover Images offers a new contribution to studies of the digital and virtual image. Specifically addressing immersive image forms used in Mars exploration the research is structured around four main case studies: life-size illusions such as panoramas; 3D imaging; false colour imaging; and the concept of a ‘Mars Yard’. The thesis offers a new understanding of human interaction with a landscape only visible through a screen, and how contemporary scientific imaging devices aim to collapse the frame and increase a sense of immersion in the image. Arguing that these representations produce inherently virtual experiences, their transportive power is questioned, highlighting the image as reconstructed – through the presence of a glitch, illusion is broken, revealing the image-as-image. This thesis takes an interdisciplinary approach in which scientific images are analysed through the prism of photography’s relationship to reality, theories of vision and perception, representations of landscape, and digital and virtual image theory. At the heart of this thesis is the act of looking; critical and speculative writing is used to convey immersive encounters with images at NASA and the Jet Propulsion Laboratory (USA); University College London’s Regional Planetary Imaging Facility; Airbus Defence and Space (UK); the photographic archive at the V&A; and the Panorama Mesdag (Netherlands). The research re-examines scientific forms of images against examples from the history of visual culture (be it art or popular culture) to draw parallels between different ways of seeing, representing and discovering the unknown. The eyes of the Mars rovers provide viewpoints through which we regard an alien terrain: windows upon unknown worlds. Rover images bridge a gap between what is known and unknown, between what is visible and invisible. The rover is our surrogate, an extension of our vision that portrays an intuitively comprehensible landscape. Yet this landscape remains totally out of reach, millions of miles away. This distance is an impenetrable boundary – both physically and metaphorically – that new technologies are trying to break. Mars, Invisible Vision and the Virtual Landscape offers a two-way impact, constituting a new approach to the relationship between real and imagined images in order to demonstrate that the real Mars, however it is represented and perceived, remains distant and detached

3rd International Workshop on Instrumentation for Planetary Missions : October 24–27, 2016, Pasadena, California

The purpose of this workshop is to provide a forum for collaboration, exchange of ideas and information, and discussions in the area of the instruments, subsystems, and other payload-related technologies needed to address planetary science questions. The agenda will compose a broad survey of the current state-of-the-art and emerging capabilities in instrumentation available for future planetary missions.Universities Space Research Association (USRA); Lunar and Planetary Institute (LPI); Jet Propulsion Laboratory (JPL)Conveners: Sabrina Feldman, Jet Propulsion Laboratory, David Beaty, Jet Propulsion Laboratory ; Science Organizing Committee: Carlton Allen, Johnson Space Center (retired) [and 12 others