Multi-wavelength infrared imaging computer systems and applications

This dissertation presents the development of three computer systems for multi-wavelength thermal imaging. Two computer systems were developed for the multi-wavelength imaging pyrometers (M-WIPs) that yield non-contact temperature measurements by remotely sensing the surface of objects with unknown wavelength-dependent emissivity. These M-WIP computer systems represent the state-of-art development in remote temperature measurement system based on the multi-wavelength approach. The dissertation research includes M-WIP computer system integration, software development, performance evaluation, and also applications in monitoring and control of temperature distribution of silicon wafers in a rapid thermal process system. The two M-WIPs are capable of data acquisition, signal processing, system calibration, radiometric measurement, parallel processing and process control. Temperature measurement experiments demonstrated the accuracy of ±1°C against blackbody and ±4°C for colorbody objects. Various algorithms were developed and implemented, including real-time two-point non-uniformity correction, thermal image pseudocoloring, PC to SUN workstation data transfer, automatic IR camera integration time control, and radiometric measurement parallel processing. A third computer system was developed for the demonstration of a 3-color InGaAs FPA which can provide images with information in three different IR wavelength range simultaneously. Numbers of functions were developed to demonstrate and characterize 3-color FPAs, and the system was delivered to be used by the 3-color FPA manufacturer

Evaluation of Single-Chip, Real-Time Tomographic Data Processing on FPGA - SoC Devices

A novel approach to tomographic data processing has been developed and evaluated using the Jagiellonian PET (J-PET) scanner as an example. We propose a system in which there is no need for powerful, local to the scanner processing facility, capable to reconstruct images on the fly. Instead we introduce a Field Programmable Gate Array (FPGA) System-on-Chip (SoC) platform connected directly to data streams coming from the scanner, which can perform event building, filtering, coincidence search and Region-Of-Response (ROR) reconstruction by the programmable logic and visualization by the integrated processors. The platform significantly reduces data volume converting raw data to a list-mode representation, while generating visualization on the fly.Comment: IEEE Transactions on Medical Imaging, 17 May 201

Ein flexibles, heterogenes Bildverarbeitungs-Framework für weltraumbasierte, rekonfigurierbare Datenverarbeitungsmodule

Scientific instruments as payload of current space missions are often equipped with high-resolution sensors. Thereby, especially camera-based instruments produce a vast amount of data. To obtain the desired scientific information, this data usually is processed on ground. Due to the high distance of missions within the solar system, the data rate for downlink to the ground station is strictly limited. The volume of scientific relevant data is usually less compared to the obtained raw data. Therefore, processing already has to be carried out on-board the spacecraft. An example of such an instrument is the Polarimetric and Helioseismic Imager (PHI) on-board Solar Orbiter. For acquisition, storage and processing of images, the instrument is equipped with a Data Processing Module (DPM). It makes use of heterogeneous computing based on a dedicated LEON3 processor in combination with two reconfigurable Xilinx Virtex-4 Field-Programmable Gate Arrays (FPGAs). The thesis will provide an overview of the available space-grade processing components (processors and FPGAs) which fulfill the requirements of deepspace missions. It also presents existing processing platforms which are based upon a heterogeneous system combining processors and FPGAs. This also includes the DPM of the PHI instrument, whose architecture will be introduced in detail. As core contribution of this thesis, a framework will be presented which enables high-performance image processing on such hardware-based systems while retaining software-like flexibility. This framework mainly consists of a variety of modules for hardware acceleration which are integrated seamlessly into the data flow of the on-board software. Supplementary, it makes extensive use of the dynamic in-flight reconfigurability of the used Virtex-4 FPGAs. The flexibility of the presented framework is proven by means of multiple examples from within the image processing of the PHI instrument. The framework is analyzed with respect to processing performance as well as power consumption.Wissenschaftliche Instrumente auf aktuellen Raumfahrtmissionen sind oft mit hochauflösenden Sensoren ausgestattet. Insbesondere kamerabasierte Instrumente produzieren dabei eine große Menge an Daten. Diese werden üblicherweise nach dem Empfang auf der Erde weiterverarbeitet, um daraus wissenschaftlich relevante Informationen zu gewinnen. Aufgrund der großen Entfernung von Missionen innerhalb unseres Sonnensystems ist die Datenrate zur Übertragung an die Bodenstation oft sehr begrenzt. Das Volumen der wissenschaftlich relevanten Daten ist meist deutlich kleiner als die aufgenommenen Rohdaten. Daher ist es vorteilhaft, diese bereits an Board der Sonde zu verarbeiten. Ein Beispiel für solch ein Instrument ist der Polarimetric and Helioseismic Imager (PHI) an Bord von Solar Orbiter. Um die Daten aufzunehmen, zu speichern und zu verarbeiten, ist das Instrument mit einem Data Processing Module (DPM) ausgestattet. Dieses nutzt ein heterogenes Rechnersystem aus einem dedizierten LEON3 Prozessor, zusammen mit zwei rekonfigurierbaren Xilinx Virtex-4 Field-Programmable Gate Arrays (FPGAs). Die folgende Arbeit gibt einen Überblick über verfügbare Komponenten zur Datenverarbeitung (Prozessoren und FPGAs), die den Anforderungen von Raumfahrtmissionen gerecht werden, und stellt einige existierende Plattformen vor, die auf einem heterogenen System aus Prozessor und FPGA basieren. Hierzu gehört auch das Data Processing Module des PHI Instrumentes, dessen Architektur im Verlauf dieser Arbeit beschrieben wird. Als Kernelement der Dissertation wird ein Framework vorgestellt, das sowohl eine performante, als auch eine flexible Bilddatenverarbeitung auf einem solchen System ermöglicht. Dieses Framework besteht aus verschiedenen Modulen zur Hardwarebeschleunigung und bindet diese nahtlos in den Datenfluss der On-Board Software ein. Dabei wird außerdem die Möglichkeit genutzt, die eingesetzten Virtex-4 FPGAs dynamisch zur Laufzeit zu rekonfigurieren. Die Flexibilität des vorgestellten Frameworks wird anhand mehrerer Fallbeispiele aus der Bildverarbeitung von PHI dargestellt. Das Framework wird bezüglich der Verarbeitungsgeschwindigkeit und Energieeffizienz analysiert

Embedded middleware for smart camera networks and sensor fusion

Abstract Smart cameras are an interesting research field that has evolved over the last decade. In this chapter we focus on the integration of multiple, potentially heterogeneous, smart cameras into a distributed system for computer vision and sensor fusion. An important aspect for every distributed system is the system-level software, also called middleware. Hence, we discuss the requirements on middleware for distributed smart cameras and the services such a middleware has to provide. In our opinion a middleware following the agent-oriented paradigm allows to build flexible and self-organizing applications that encourage a modular design

Evaluation of a SoC for Real-time 3D SLAM

SLAM, or Simultaneous Localization and Mapping, is the combined problem of constructing a map of an agent’s environment while localizing, or tracking that same agent’s pose in tandem. It is among the most challenging and fundamental tasks in computer vision, with applications ranging from augmented reality to robotic navigation. With the increasing capability and ubiquity of mobile computers such as cell phones, portable 3D SLAM systems are becoming feasible for widespread use. The Microsoft Hololens, Google Project Tango, and other 3D aware devices are modern day examples of the potential of SLAM and the challenges it has yet to face. The ICP, or Iterative Closest Point Algorithm, is a popular solution for retrieving the relative transformation between two scans of the same object. It has gained a resurgence in popularity due to the rise of affordable depth sensors such as the Kinect in robotics and augmented reality research. ICP, while providing a high certainty of correctness given similar point clouds, is challenging to implement in real time due to its computational complexity. In this thesis, a basic 3D SLAM algorithm is implemented and evaluated, and two proposed FPGA architectures to accelerate the Nearest Neighbor component of ICP for use in a mobile ARM-based System-on-Chip (SoC) are presented. These architectures are predicted to achieve speedups of up to 7.89x and 17.22x over a naive embedded software implementation

Automatisiertes Kamera-Tracking zur Verbesserung der Lesbarkeit von Tafelanschrieben im Rahmen der Vorlesungsaufzeichnung an der Universität Stuttgart

Diese Arbeit beschäftigt sich mit dem Test, Programmanpassungen und -erweiterungen sowie der erstmaligen Installation eines automatisierten Kamera-Tracking-Systems an der Universität Stuttgart. Ziel dabei war es, die Aufzeichnung von Vorlesungen so zu ver- bessern, dass Tafelanschriebe auf Videoaufnahmen wesentlich besser lesbar werden. Im Laufe der Arbeit wird das Projekt „Hörsäle 2020“ beschrieben, bei dem die Hörsäle an der Universität Stuttgart mit neuer Medientechnik ausgestattet werden und durch das diese Arbeit möglich wurde. Außerdem wird näher auf die Kamerasteuerung über VISCA und VISCA over IP eingegangen und der Aufbau sowie die Funktionsweise der Software Lec- tureSight erklärt. Mit Hilfe dieser Software entstand ein Testaufbau, um zu prüfen, wie das Tracking funktioniert und welche Probleme dabei entstehen. Dieser Test wurde spä- ter auf einen Hörsaal ausgeweitet, um die Konfiguration abzuschließen und ein voll funk- tionsfähiges System zu erhalten. Durch mehrere Maßnahmen konnte das Tracking wei- ter verbessert werden, bis abschließend ein Test zur Lesbarkeit durchgeführt wurde, der bestätigt, dass mit einer bewegten Kamera und höherem Zoomfaktor die Deutlichkeit von Tafelanschrieben in den Videoaufzeichnungen drastisch zunimmt.This Bachelor thesis deals with the test, program modifications and extensions as well as the first installation of an automated camera tracking system at the University of Stuttgart. The aim was to improve the lecture recordings in such a way that the writing on the blackboard is much more readable on video recordings. In the course of this thesis, the “Hörsäle 2020” project will be described, in which the lecture halls at the University of Stuttgart will be equipped with new media technology and which made this thesis pos- sible. In addition, the camera control via VISCA and VISCA over IP is discussed and the structure and functionality of the software LectureSight explained. With the help of this software, a test set-up was developed to test how the tracking works and to identify what problems arise. Later on this test was extended to a lecture hall to complete the config- uration and to obtain a fully functional system. By several measures, the tracking was further improved and a test for readability was carried out, which confirms that the clarity of the writing on the blackboard increases drastically with a moving camera and a higher zoom factor

NASA Tech Briefs, January 1990

Topics include: Electronic Components and Circuits. Electronic Systems, Physical Sciences, Materials, Computer Programs, Mechanics, Machinery, Fabrication Technology, Mathematics and Information Sciences, and Life Science

Evaluation of single photon avalanche diode arrays for imaging fluorescence correlation spectroscopy : FPGA-based data readout and fast correlation analysis on CPUs, GPUs and FPGAs

The metabolism of all living organisms, and specifically also of their smallest constituents, the cell, is based on chemical reactions. A key factor determining the speed of these processes is transport of reactants, energy, and information within the and between the cells of an organism. It has been shown that the relevant transport processes also depend on the spatial organization of the cells. Such transport processes are typically investigated using fluorescence correlation spectroscopy (FCS) in combination with fluorescent labeling of the molecules of interest. In FCS, one observes the fluctuating fluorescence signal from a femtoliter-sized sub-volume within the sample (e.g. a cell). The variations in the intensity arise from the particles moving in and out of this sub-volume. By means of an autocorrelation analysis of the intensity signal, conclusion can be drawn regarding the concentration and the mobility parameters, such as the diffusion coefficient. Typically, one uses the laser focus of a confocal microscope for FCS measurements. But with this microscopy technique, FCS is limited to a single spot a every time. In order to conduct parallel multi-spot measurements, i.e. to create diffusion maps, FCS can be combined with the lightsheet based selective plane illumination microscopy (SPIM). This recent widefield microscopy technique allows observing a small plane of a sample (1-3um thick), which can be positioned arbitrarily. Usually, FCS on a SPIM is done using fast electron-multiplying charge-coupled device (EMCCD) cameras, which offer a limited temporal resolution (500us). Such a temporal resolution only allows measuring the motion of intermediately sized particles within a cell reliably. The limited temporal resolution renders the detection of even smaller molecules impossible. In this thesis, arrays of single photon avalanche diodes (SPADs) were used as detectors. Although SPAD-based image sensors still lack in sensitivity, they provide a significantly better temporal resolution (1-10us for full frames) that is not achievable with sensitive cameras and seem to be the ideal sensors for SPIM-FCS. In the course of this work, two recent SPAD arrays (developed in the groups of Prof. Edoardo Charbon, TU Delft, the Netherlands, and EPFL, Switzerland) were extensively characterized with regards to their suitability for SPIM-FCS. The evaluated SPAD arrays comprise 32x32 and 512x128 pixels and allow for frame rates of up to 300000 or 150000 frames per second, respectively. With these specifications, the latter array is one of the largest and fastest sensors that is currently available. During full-frame readout, it delivers a data rate of up to 1.2 GiB/s. For both arrays, suitable readout-hardware-based on field programmable gate arrays (FPGAs) was designed. To cope with the high data rate and to allow real-time correlation analysis, correlation algorithms were implemented and characterized on the three major high performance computing platforms, namely FPGAs, CPUs, and graphics processing units (GPUs). Of all three platforms, the GPU performed best in terms of correlation analysis, and a speed of 2.6 over real time was achieved for the larger SPAD array. Beside the lack in sensitivity, which could be accounted for by microlenses, a major drawback of the evaluated SPAD arrays was their afterpulsing. It appeared that the temporal structure superimposed the signal of the diffusion. Thus, extracting diffusion properties from the autocorrelation analysis only proved impossible. By additionally performing a spatial cross-correlation analysis such influences could be significantly minimized. Furthermore, this approach allowed for the determination of absolute diffusion coefficients without prior calibration. With that, spatially resolved measurements of fluorescent proteins in living cells could be conducted successfully