High-performance hardware accelerators for image processing in space applications

Mars is a hard place to reach. While there have been many notable success stories in getting probes to the Red Planet, the historical record is full of bad news. The success rate for actually landing on the Martian surface is even worse, roughly 30%. This low success rate must be mainly credited to the Mars environment characteristics. In the Mars atmosphere strong winds frequently breath. This phenomena usually modifies the lander descending trajectory diverging it from the target one. Moreover, the Mars surface is not the best place where performing a safe land. It is pitched by many and close craters and huge stones, and characterized by huge mountains and hills (e.g., Olympus Mons is 648 km in diameter and 27 km tall). For these reasons a mission failure due to a landing in huge craters, on big stones or on part of the surface characterized by a high slope is highly probable. In the last years, all space agencies have increased their research efforts in order to enhance the success rate of Mars missions. In particular, the two hottest research topics are: the active debris removal and the guided landing on Mars. The former aims at finding new methods to remove space debris exploiting unmanned spacecrafts. These must be able to autonomously: detect a debris, analyses it, in order to extract its characteristics in terms of weight, speed and dimension, and, eventually, rendezvous with it. In order to perform these tasks, the spacecraft must have high vision capabilities. In other words, it must be able to take pictures and process them with very complex image processing algorithms in order to detect, track and analyse the debris. The latter aims at increasing the landing point precision (i.e., landing ellipse) on Mars. Future space-missions will increasingly adopt Video Based Navigation systems to assist the entry, descent and landing (EDL) phase of space modules (e.g., spacecrafts), enhancing the precision of automatic EDL navigation systems. For instance, recent space exploration missions, e.g., Spirity, Oppurtunity, and Curiosity, made use of an EDL procedure aiming at following a fixed and precomputed descending trajectory to reach a precise landing point. This approach guarantees a maximum landing point precision of 20 km. By comparing this data with the Mars environment characteristics, it is possible to understand how the mission failure probability still remains really high. A very challenging problem is to design an autonomous-guided EDL system able to even more reduce the landing ellipse, guaranteeing to avoid the landing in dangerous area of Mars surface (e.g., huge craters or big stones) that could lead to the mission failure. The autonomous behaviour of the system is mandatory since a manual driven approach is not feasible due to the distance between Earth and Mars. Since this distance varies from 56 to 100 million of km approximately due to the orbit eccentricity, even if a signal transmission at the light speed could be possible, in the best case the transmission time would be around 31 minutes, exceeding so the overall duration of the EDL phase. In both applications, algorithms must guarantee self-adaptability to the environmental conditions. Since the Mars (and in general the space) harsh conditions are difficult to be predicted at design time, these algorithms must be able to automatically tune the internal parameters depending on the current conditions. Moreover, real-time performances are another key factor. Since a software implementation of these computational intensive tasks cannot reach the required performances, these algorithms must be accelerated via hardware. For this reasons, this thesis presents my research work done on advanced image processing algorithms for space applications and the associated hardware accelerators. My research activity has been focused on both the algorithm and their hardware implementations. Concerning the first aspect, I mainly focused my research effort to integrate self-adaptability features in the existing algorithms. While concerning the second, I studied and validated a methodology to efficiently develop, verify and validate hardware components aimed at accelerating video-based applications. This approach allowed me to develop and test high performance hardware accelerators that strongly overcome the performances of the actual state-of-the-art implementations. The thesis is organized in four main chapters. Chapter 2 starts with a brief introduction about the story of digital image processing. The main content of this chapter is the description of space missions in which digital image processing has a key role. A major effort has been spent on the missions in which my research activity has a substantial impact. In particular, for these missions, this chapter deeply analizes and evaluates the state-of-the-art approaches and algorithms. Chapter 3 analyzes and compares the two technologies used to implement high performances hardware accelerators, i.e., Application Specific Integrated Circuits (ASICs) and Field Programmable Gate Arrays (FPGAs). Thanks to this information the reader may understand the main reasons behind the decision of space agencies to exploit FPGAs instead of ASICs for high-performance hardware accelerators in space missions, even if FPGAs are more sensible to Single Event Upsets (i.e., transient error induced on hardware component by alpha particles and solar radiation in space). Moreover, this chapter deeply describes the three available space-grade FPGA technologies (i.e., One-time Programmable, Flash-based, and SRAM-based), and the main fault-mitigation techniques against SEUs that are mandatory for employing space-grade FPGAs in actual missions. Chapter 4 describes one of the main contribution of my research work: a library of high-performance hardware accelerators for image processing in space applications. The basic idea behind this library is to offer to designers a set of validated hardware components able to strongly speed up the basic image processing operations commonly used in an image processing chain. In other words, these components can be directly used as elementary building blocks to easily create a complex image processing system, without wasting time in the debug and validation phase. This library groups the proposed hardware accelerators in IP-core families. The components contained in a same family share the same provided functionality and input/output interface. This harmonization in the I/O interface enables to substitute, inside a complex image processing system, components of the same family without requiring modifications to the system communication infrastructure. In addition to the analysis of the internal architecture of the proposed components, another important aspect of this chapter is the methodology used to develop, verify and validate the proposed high performance image processing hardware accelerators. This methodology involves the usage of different programming and hardware description languages in order to support the designer from the algorithm modelling up to the hardware implementation and validation. Chapter 5 presents the proposed complex image processing systems. In particular, it exploits a set of actual case studies, associated with the most recent space agency needs, to show how the hardware accelerator components can be assembled to build a complex image processing system. In addition to the hardware accelerators contained in the library, the described complex system embeds innovative ad-hoc hardware components and software routines able to provide high performance and self-adaptable image processing functionalities. To prove the benefits of the proposed methodology, each case study is concluded with a comparison with the current state-of-the-art implementations, highlighting the benefits in terms of performances and self-adaptability to the environmental conditions

ReSCon '09, Research Student Conference: Book of Abstracts

The second SED Research Student Conference (ReSCon2009) was hosted over three days, 22-24 June 2009, in the Lecture Centre at Brunel University. The conference consisted of technical presentations, a poster session and social events. The abstracts and presentations were the result of ongoing research by postgraduate research students from the School of Engineering and Design at Brunel University. The conference is held annually, and ReSCon plays a key role in contributing to research and innovations within the School

Robustesse par conception de circuits implantés sur FPGA SRAM et validation par injection de fautes

Cette thèse s'intéresse en premier lieu à l'évaluation des effets fonctionnels des erreurs survenant dans la mémoire SRAM de configuration de certains FPGAs. La famille Virtex II Pro de Xilinx est utilisée comme premier cas pratique d'expérimentation. Des expérimentations sous faisceau laser nous ont permis d'avoir une bonne vue d'ensemble sur les motifs d'erreurs réalistes qui sont obtenus par des sources de perturbations réelles. Une méthodologie adaptée d'injection de fautes a donc été définie pour permettre une meilleure évaluation, en phase de conception, de la robustesse d'un circuit implanté sur ce type de technologie. Cette méthodologie est basée sur de la reconfiguration dynamique. Le même type d'approche a ensuite été évalué sur plusieurs cibles technologiques, ce qui a nécessité le développement de plusieurs environnements d'injection de fautes. L'étude a pour la première fois inclus la famille AT40K de ATMEL, qui permet un type de reconfiguration unique et efficace. Le second type de contribution concerne l'augmentation à faible coût de la robustesse de circuits implantés sur des plateformes FPGA SRAM. Nous proposons une approche de protection sélective exploitant les ressources du FPGA inutilisées par l'application. L'approche a été automatisée sur plusieurs cibles technologiques (Xilinx, Altera) et l'efficacité est analysée en utilisant les méthodes d'injection de fautes précédemment développées.This thesis focuses primarily on the evaluation of the functional effects of errors occurring in the SRAM configuration memory of some FPGAs. Xilinx Virtex II Pro family is used as a first case study. Experiments under laser beam allowed us to have a good overview of realistic error patterns, related to real disturbance sources. A suited fault injection methodology has thus been defined to improve design-time robustness evaluations of a circuit implemented on this type of technology. This methodology is based on runtime reconfiguration. The approach has then been evaluated on several technological targets, requiring the development of several fault injection environments. The study included for the first time the ATMEL AT40K family, with a unique and efficient reconfiguration mode. The second type of contribution is focused on the improvement at low cost of the robustness of designs implemented on SRAM-based FPGA platforms. We propose a selective protection approach exploiting resources unused by the application. The approach has been automated on several technological targets (Xilinx, Altera) and the efficiency has been analyzed by taking advantage of the fault injection techniques previously developed.SAVOIE-SCD - Bib.électronique (730659901) / SudocGRENOBLE1/INP-Bib.électronique (384210012) / SudocGRENOBLE2/3-Bib.électronique (384219901) / SudocSudocFranceF

Belle II Technical Design Report

The Belle detector at the KEKB electron-positron collider has collected almost 1 billion Y(4S) events in its decade of operation. Super-KEKB, an upgrade of KEKB is under construction, to increase the luminosity by two orders of magnitude during a three-year shutdown, with an ultimate goal of 8E35 /cm^2 /s luminosity. To exploit the increased luminosity, an upgrade of the Belle detector has been proposed. A new international collaboration Belle-II, is being formed. The Technical Design Report presents physics motivation, basic methods of the accelerator upgrade, as well as key improvements of the detector.Comment: Edited by: Z. Dole\v{z}al and S. Un

Real-Time High-Resolution Multiple-Camera Depth Map Estimation Hardware and Its Applications

Depth information is used in a variety of 3D based signal processing applications such as autonomous navigation of robots and driving systems, object detection and tracking, computer games, 3D television, and free view-point synthesis. These applications require high accuracy and speed performances for depth estimation. Depth maps can be generated using disparity estimation methods, which are obtained from stereo matching between multiple images. The computational complexity of disparity estimation algorithms and the need of large size and bandwidth for the external and internal memory make the real-time processing of disparity estimation challenging, especially for high resolution images. This thesis proposes a high-resolution high-quality multiple-camera depth map estimation hardware. The proposed hardware is verified in real-time with a complete system from the initial image capture to the display and applications. The details of the complete system are presented. The proposed binocular and trinocular adaptive window size disparity estimation algorithms are carefully designed to be suitable to real-time hardware implementation by allowing efficient parallel and local processing while providing high-quality results. The proposed binocular and trinocular disparity estimation hardware implementations can process 55 frames per second on a Virtex-7 FPGA at a 1024 x 768 XGA video resolution for a 128 pixel disparity range. The proposed binocular disparity estimation hardware provides best quality compared to existing real-time high-resolution disparity estimation hardware implementations. A novel compressed-look up table based rectification algorithm and its real-time hardware implementation are presented. The low-complexity decompression process of the rectification hardware utilizes a negligible amount of LUT and DFF resources of the FPGA while it does not require the existence of external memory. The first real-time high-resolution free viewpoint synthesis hardware utilizing three-camera disparity estimation is presented. The proposed hardware generates high-quality free viewpoint video in real-time for any horizontally aligned arbitrary camera positioned between the leftmost and rightmost physical cameras. The full embedded system of the depth estimation is explained. The presented embedded system transfers disparity results together with synchronized RGB pixels to the PC for application development. Several real-time applications are developed on a PC using the obtained RGB+D results. The implemented depth estimation based real-time software applications are: depth based image thresholding, speed and distance measurement, head-hands-shoulders tracking, virtual mouse using hand tracking and face tracking integrated with free viewpoint synthesis. The proposed binocular disparity estimation hardware is implemented in an ASIC. The ASIC implementation of disparity estimation imposes additional constraints with respect to the FPGA implementation. These restrictions, their implemented efficient solutions and the ASIC implementation results are presented. In addition, a very high-resolution (82.3 MP) 360°x90° omnidirectional multiple camera system is proposed. The hemispherical camera system is able to view the target locations close to horizontal plane with more than two cameras. Therefore, it can be used in high-resolution 360° depth map estimation and its applications in the future

Galaxy evolution probe

The Galaxy Evolution Probe (GEP) is a concept for a mid- and far-infrared space observatory to measure key properties of large samples of galaxies with large and unbiased surveys. GEP will attempt to achieve zodiacal light and Galactic dust emission photon background-limited observations by utilizing a 6-K, 2.0-m primary mirror and sensitive arrays of kinetic inductance detectors (KIDs). It will have two instrument modules: a 10 to 400  μm hyperspectral imager with spectral resolution R  =  λ  /  Δλ  ≥  8 (GEP-I) and a 24 to 193  μm, R  =  200 grating spectrometer (GEP-S). GEP-I surveys will identify star-forming galaxies via their thermal dust emission and simultaneously measure redshifts using polycyclic aromatic hydrocarbon emission lines. Galaxy luminosities derived from star formation and nuclear supermassive black hole accretion will be measured for each source, enabling the cosmic star formation history to be measured to much greater precision than previously possible. Using optically thin far-infrared fine-structure lines, surveys with GEP-S will measure the growth of metallicity in the hearts of galaxies over cosmic time and extraplanar gas will be mapped in spiral galaxies in the local universe to investigate feedback processes. The science case and mission architecture designed to meet the science requirements is described, and the KID and readout electronics state of the art and needed developments are described. This paper supersedes the GEP concept study report cited in it by providing new content, including: a summary of recent mid-infrared KID development, a discussion of microlens array fabrication for mid-infrared KIDs, and additional context for galaxy surveys. The reader interested in more technical details may want to consult the concept study report

NIM: The Neutral Gas and Ion Mass Spectrometer to Explore the Galilean Ice Worlds

The JUpiter ICy moons Explorer (JUICE) of the European Space Agency (ESA) has the purpose to investigate Jupiter and its icy moons Europa, Ganymede and Callisto in great detail. Among other scientific goals, JUICE will investigate the Jupiter system as a potential habitable system because the three icy moons have subsurface oceans where life might be possible. On board of JUICE is the Particle Environment Package (PEP), which consists of six individual instruments measuring electrons, ions and neutral particles in an energy range from meV up to MeV. One of these six instruments is the Neutral gas and Ion Mass spectrometer (NIM) from the University of Bern. The NIM instrument is designed to measure the chemical and isotope composition of the icy moons’ exospheres during the flybys of JUICE of the icy moons and also during JUICE final destination in Ganymede’s orbit. Knowing the chemical and isotope composition allows to investigate the origin and evolution processes involved in the formation processes of the icy moons, Jupiter and our solar system. NIM is a time-of-flight mass spectrometer able to measure thermal neutral molecules and ionospheric ions. This thesis shows the journey from finalising the flight design of the NIM instrument to the actual testing, qualification, and calibration until delivery of the NIM Proto-Flight (PFM) instrument in December 2020 to the JUICE spacecraft. On this journey, different flight components were tested and analysed as they became available during the development and finalisation of the PFM and Flight-Spare (FS) instrument. From the foreseen scientific scope for the NIM instrument, a list of measurement requirements was delivered. This work shows that NIM PFM and NIM FS meet all these requirements

Optimising the NAOMI adaptive optics real-time control system

This thesis describes the author's research in the field of Real-Time Control (RTC) for Adaptive Optics (AO) instrumentation. The research encompasses experiences and knowledge gained working in the area of RTC on astronomical instrumentation projects whilst at the Optical Science Laboratories (OSL), University College London (UCL), the Isaac Newton Groups of Telescopes (ING) and the Centre for Advanced Instrumentation (СfAI), Durham University. It begins by providing an extensive introduction to the field of Astronomical Adaptive Optics covering Image Correction Theory, Atmospheric Theory, Control Theory and Adaptive Optics Component Theory. The following chapter contains a review of the current state of world wide AO instruments and facilities. The Nasmyth Adaptive Optics Multi-purpose Instrument (NAOMI), the common user AO facility at the 4.2 William Herschel Telescope (WHT), is subsequently described. Results of NAOMI component characterisation experiments are detailed to provide a system understanding of the improvement optimisation could offer. The final chapter investigates how upgrading the RTCS could increase NAOMI'S spatial and temporal performance and examines the RTCS in the context of Extremely Large Telescope (ELT) class telescopes

Lunar University Network for Astrophysics Research: Comprehensive Report to The NASA Lunar Science Institute. March 1, 2012

The Lunar University Network for Astrophysics Research (LUNAR) is a team of researchers and students at leading universities, NASA centers, and federal research laboratories undertaking investigations aimed at using the Moon as a platform for space science. LUNAR research includes Lunar Interior Physics & Gravitation using Lunar Laser Ranging (LLR), Low Frequency Cosmology and Astrophysics (LFCA), Planetary Science and the Lunar Ionosphere, Radio Heliophysics, and Exploration Science. The LUNAR team is exploring technologies that are likely to have a dual purpose, serving both exploration and science. There is a certain degree of commonality in much of LUNAR’s research. Specifically, the technology development for a lunar radio telescope involves elements from LFCA, Heliophysics, Exploration Science, and Planetary Science; similarly the drilling technology developed for LLR applies broadly to both Exploration and Lunar Science

Topical Workshop on Electronics for Particle Physics

The purpose of the workshop was to present results and original concepts for electronics research and development relevant to particle physics experiments as well as accelerator and beam instrumentation at future facilities; to review the status of electronics for the LHC experiments; to identify and encourage common efforts for the development of electronics; and to promote information exchange and collaboration in the relevant engineering and physics communities