Image Enhancement and Parameter Estimation for Time-of-Flight Cameras

The determination of the spatial depth of a scene, which supplies important information for a number of applications, was a very complex matter with previous technologies like stereo cameras or laser scanners and sometimes problematic. With Time-of-Flight cameras there is a technology at hand today, that facilitates the acquisition of these data tremendously. With the Microsoft Kinect 2 Sensor ToF cameras have recently made the jump into the consumer market. Unfortunately these cameras still exhibit different statistical and systematic errors. This thesis shows how the statistical errors can be fixed with the help of methods from 2D image processing and gives for the first time explanations of previously not understood systematic errors by means of an exact analysis of the sensor data. With newly introduced approaches and methods these errors can be reduced notably with little effort and lead to a significant improvement in the accuracy of the measurements

Depth Super-Resolution with Hybrid Camera System

An important field of research in computer vision is the 3D analysis and reconstruction of objects and scenes. Currently, among all the the techniques for 3D acquisition, stereo vision systems are the most common. More recently, Time-of-Flight (ToF) range cameras have been introduced. The focus of this thesis is to combine the information from the ToF with one or two standard cameras, in order to obtain a high- resolution depth imageopenEmbargo per motivi di segretezza e/o di proprietà dei risultati e informazioni di enti esterni o aziende private che hanno partecipato alla realizzazione del lavoro di ricerca relativo alla tes

Compressive Sensing for Low Power Sensor Design

Recent sensor System-on-Chips (SoC) have enabled significant advances in energy-efficiency by incorporating various micro-powered building blocks. Unfortunately, most of these sensor systems do not address the high power cost associated with data storage and transmission, which in some cases vastly exceeds the power consumed by the rest of the SoC. In recent years, Compressive-Sensing (CS) has been proposed as a method to accomplish significant sensor data compression, achieving compression rates up to 10x depending on the signal sparsity. This work addresses conventional CS issues including non-adaptive compression rate and offers a solution. First, a feasibility study is conducted to investigate the sparsity variance of some biomedical signals. Then an adaptive CS framework is proposed, to adjust the compression rate based upon the input signal’s sparsity on-the-fly. Thirdly, a CS framework is proposed, the reconstruction of which is aided by statistics collection. It is demonstrated how to fuse sensor data and statistics information together to improve the signal-to-error ratio (SER) of reconstruction. A test chip fabricated in TSMC 65-nm technology to implement the algorithm in a SoC incorporating statistics collection block in order to improve performance of the CS algorithm. The final portion of this research devoted to study emerging application of time-of-flight cameras for depth measurement. These devices generate a 3 Dimensional (3D) point cloud that basically includes 3D details of objects in front of them. A framework to apply CS to 3D point cloud data is presented. Finally it demonstrates how the idea of adaptive CS can be used for 3D point cloud data compression

Acquisition and Processing of ToF and Stereo data

Providing a computer the capability to estimate the three-dimensional geometry of a scene is a fundamental problem in computer vision. A classical systems that has been adopted for solving this problem is the so-called stereo vision system (stereo system). Such a system is constituted by a couple of cameras and it exploits the principle of triangulation in order to provide an estimate of the framed scene. In the last ten years, new devices based on the time-of-flight principle have been proposed in order to solve the same problem, i.e., matricial Time-of-Flight range cameras (ToF cameras). This thesis focuses on the analysis of the two systems (ToF and stereo cam- eras) from a theoretical and an experimental point of view. ToF cameras are introduced in Chapter 2 and stereo systems in Chapter 3. In particular, for the case of the ToF cameras, a new formal model that describes the acquisition process is derived and presented. In order to understand strengths and weaknesses of such different systems, a comparison methodology is introduced and explained in Chapter 4. From the analysis of ToF cameras and stereo systems it is possible to understand the complementarity of the two systems and it is intuitive to figure that a synergic fusion of their data might provide an improvement in the quality of the measurements preformed by the two devices. In Chapter 5 a method for fusing ToF and stereo data based on a probability approach is presented. In Chapter 6 a method that exploits color and three-dimensional geometry information for solving the classical problem of scene segmentation is explaine

Photoemission electron microscopy for nanoscale imaging and attosecond control of light-matter interaction at metal surfaces

Electron dynamics at solid surfaces unfold on the nanometer length and attosecond timescale when driven by electromagnetic fields at optical frequencies, enabling vast scientific and technological applications in the field of nano-optics and nanoplasmonics. Direct imaging of the electrons upon interaction with light is a highly desirable tool for understanding and control of the dynamics, which requires ultrahigh spatiotemporal resolution. This thesis explores the combination of photoemission electron microscopy (PEEM) with few-cycle femtosecond laser pulses and attosecond extreme ultraviolet (XUV) pulses for studying ultrafast electron dynamics from metallic surfaces and nanosystems. The work involves development and implementation of new experimental tools including detection, data acquisition and analysis techniques for PEEM measurements. The first approach is using a combination of PEEM with attosecond streaking spectroscopy (atto-PEEM) for direct, non-invasive probing of nanoplasmonic fields from supported nanostructures. As a first step towards the implementation of the atto-PEEM concept, PEEM imaging on lithographically fabricated gold structures employing 93 eV attosecond XUV pulses from a 1 kHz high-harmonic generation (HHG) source is performed. The spatial resolution is limited to ~200 nm due to space charge effects when working with such a low-repetition-rate HHG source and chromatic aberrations caused by the large energy bandwidth of XUV-generated photoelectrons. Nevertheless, we show that microspectroscopic imaging of core-level and valence band electrons is achievable using our energy-resolved PEEM despite the aforementioned issues. Most importantly, we find that the fast photoelectrons from the valence band, which carry the attosecond temporal structure of the plasmonic field, are not affected by space charge effects. The currently developed megahertz-repetition-rate attosecond XUV sources are therefore expected to enable the experimental realization of nanoplasmonic streaking with ultrahigh spatiotemporal resolution in the near future. Second, PEEM is coupled with a single-shot stereographic above-threshold ionization phase meter, which allows carrier-envelope phase (CEP) tagging for studying attosecond control of photoemission. First experiments performed on gold nanospheres on a gold plane and on a random rough gold surface using few-cycle near-infrared pulses show a CEP artefact with a modulation period of π. The artefact is found to be caused by a laser intensity dependence of both the photoelectron spectra and the CEP measurement. Intensity tagging is therefore added to the current CEP tagging technique to correct this intensity-dependent artefact. As a result, a very weak CEP modulation (~1% amplitude) of the photoemission yield from a bulk tungsten surface with a 2π modulation period (as expected from solids) is successfully detected in the above-threshold photoemission regime after applying appropriate corrections based on the intensity tagging. Entering the tunneling regime, the CEP modulation increases to ~7% despite the presence of space charge effects due to high laser peak intensity. We also apply this technique to investigate the CEP dependence on gold nanotriangles and find no apparent CEP modulation within an accuracy of ~0.6% as given by our experimental conditions, which constitutes an upper limit for a possible CEP modulation from this nanostructure.Elektronendynamik an Festkörperoberflächen, die von elektromagnetischen Feldern mit optischen Frequenzen getrieben wird, findet auf einer Längen- und Zeitskala im Bereich von Nanometern bzw. Attosekunden statt und ermöglicht eine Vielzahl wissenschaftlicher und technischer Anwendungen auf dem Gebiet der Nanooptik und Nanoplasmonik. Die direkte Visualisierung der Elektronen in Folge ihrer Wechselwirkung mit Licht, was eine ultrahohe räumlich-zeitliche Auflösung erfordert, ist ein sehr nützliches Instrument zum Verständnis dieser Dynamik und ihrer Kontrolle. In dieser Dissertation wird eine Kombination aus Photoemissionselektronenmikroskopie (PEEM) mit Femtosekundenlaserpulsen von wenigen Zyklen Dauer sowie extrem ultravioletten (XUV) Attosekundenpulsen erforscht, um ultraschnelle Elektronendynamik an Metalloberflächen und in Nanosystemen zu untersuchen. Diese Arbeit beinhaltet die Entwicklung und Implementierung neuer Messinstrumente und Methoden für PEEM-Experimente, insbesondere Detektion, Datenerfassung und Datenanalyse. Der erste Ansatz für eine direkte, nichtinvasive Untersuchung nanoplasmonischer Felder an ortsfesten Nanostrukturen ist eine Kombination von PEEM mit Attosekunden-Streaking (Atto-PEEM). Als eine Voraussetzung für die Implementierung des Atto-PEEM-Konzepts wird eine PEEM-Abbildung von lithographisch hergestellten Goldstrukturen mittels 93 eV XUV Attosekundenpulsen aus einer 1 kHz Quelle für die Erzeugung hoher Harmonischer realisiert. Wegen Raumladungseffekten, die durch die niedrige Repetitionsrate der hohen Harmonischen zustande kommen, sowie chromatischer Aberrationen aufgrund der hohen Energiebandbreite der durch die XUV-Strahlung erzeugten Photoelektronen, ist die räumliche Auflösung auf ~200 nm begrenzt. Dennoch wird gezeigt, dass trotz dieser Schwierigkeiten eine mikrospektroskopische Abbildung von inneren Elektronen und Valenzelektronen mittels unserer energieaufgelöster PEEM möglich ist. Unsere wichtigste Erkenntnis ist, dass die schnellen Photoelektronen aus dem Valenzband, die die zeitliche Struktur der plasmonischen Felder auf der Attosekundenskala abtasten, nicht durch Raumladungseffekte beeinträchtigt werden. Die sich derzeit in Entwicklung befindenden Quellen für Attosekunden-XUV-Pulse mit Megahertz Repetitionsraten sind daher vielversprechend für die experimentelle Realisierung von nanoplasmonischem Streaking mit ultrahoher räumlicher und zeitlicher Auflösung in naher Zukunft. Zweitens wird PEEM mit einem stereographischen, auf Above-Threshold-Ionisation basierenden Einzelschuss-Phasenmessgerät verbunden, was eine Zuordnung (Tagging) der Träger-Einhüllenden-Phase (carrier-envelope phase, CEP) erlaubt und dadurch ermöglicht, die Kontrolle der Photoemission auf der Attosekundenskala zu erforschen. Erste Experimente an Goldnanosphären auf einer Goldebene sowie an einer rauen Goldoberfläche mit wenige Zyklen kurzen Laserpulsen im Nah-Infraroten weisen ein CEP-Artefakt mit einer Modulationsperiode von π auf. Es wird gezeigt, dass dieses Artefakt durch eine Abhängigkeit sowohl der Photoelektronenspektra als auch der CEP-Messung von der Laserintensität hervorgerufen wird. Die bisherige CEP-Tagging-Technik wird deshalb um Intensitäts-Tagging erweitert, um dieses intensitätsabhängige Artefakt zu korrigieren. Als Resultat wird nach angemessenen Korrekturen basierend auf dem Intensitäts-Tagging eine schwache CEP-Modulation (~1% Amplitude) der Photoemissionsergiebigkeit von einer unstrukturierten Wolframoberfläche mit einer Modulationsperiode von 2π (wie bei Festkörpern erwartet) im Above-Threshold-Photoemissionsregime erfolgreich nachgewiesen. Im Tunnelregime wächst die CEP-Modulation auf ~7% trotz aufkommender Raumladungseffekte aufgrund der starken Spitzenintensität der Laserpulse. Es werden ebenfalls Goldnanodreiecke mit dieser Technik untersucht, jedoch kann keine CEP-Modulation innerhalb der experimentellen Genauigkeit von ~0.6% gefunden werden. Dies stellt eine Obergrenze für eine mögliche CEP-Modulation an dieser Nanostruktur dar

Advanced Image Acquisition, Processing Techniques and Applications

"Advanced Image Acquisition, Processing Techniques and Applications" is the first book of a series that provides image processing principles and practical software implementation on a broad range of applications. The book integrates material from leading researchers on Applied Digital Image Acquisition and Processing. An important feature of the book is its emphasis on software tools and scientific computing in order to enhance results and arrive at problem solution

Development of an augmented reality guided computer assisted orthopaedic surgery system

Previously held under moratorium from 1st December 2016 until 1st December 2021.This body of work documents the developed of a proof of concept augmented reality guided computer assisted orthopaedic surgery system – ARgCAOS. After initial investigation a visible-spectrum single camera tool-mounted tracking system based upon fiducial planar markers was implemented. The use of visible-spectrum cameras, as opposed to the infra-red cameras typically used by surgical tracking systems, allowed the captured image to be streamed to a display in an intelligible fashion. The tracking information defined the location of physical objects relative to the camera. Therefore, this information allowed virtual models to be overlaid onto the camera image. This produced a convincing augmented experience, whereby the virtual objects appeared to be within the physical world, moving with both the camera and markers as expected of physical objects. Analysis of the first generation system identified both accuracy and graphical inadequacies, prompting the development of a second generation system. This too was based upon a tool-mounted fiducial marker system, and improved performance to near-millimetre probing accuracy. A resection system was incorporated into the system, and utilising the tracking information controlled resection was performed, producing sub-millimetre accuracies. Several complications resulted from the tool-mounted approach. Therefore, a third generation system was developed. This final generation deployed a stereoscopic visible-spectrum camera system affixed to a head-mounted display worn by the user. The system allowed the augmentation of the natural view of the user, providing convincing and immersive three dimensional augmented guidance, with probing and resection accuracies of 0.55±0.04 and 0.34±0.04 mm, respectively.This body of work documents the developed of a proof of concept augmented reality guided computer assisted orthopaedic surgery system – ARgCAOS. After initial investigation a visible-spectrum single camera tool-mounted tracking system based upon fiducial planar markers was implemented. The use of visible-spectrum cameras, as opposed to the infra-red cameras typically used by surgical tracking systems, allowed the captured image to be streamed to a display in an intelligible fashion. The tracking information defined the location of physical objects relative to the camera. Therefore, this information allowed virtual models to be overlaid onto the camera image. This produced a convincing augmented experience, whereby the virtual objects appeared to be within the physical world, moving with both the camera and markers as expected of physical objects. Analysis of the first generation system identified both accuracy and graphical inadequacies, prompting the development of a second generation system. This too was based upon a tool-mounted fiducial marker system, and improved performance to near-millimetre probing accuracy. A resection system was incorporated into the system, and utilising the tracking information controlled resection was performed, producing sub-millimetre accuracies. Several complications resulted from the tool-mounted approach. Therefore, a third generation system was developed. This final generation deployed a stereoscopic visible-spectrum camera system affixed to a head-mounted display worn by the user. The system allowed the augmentation of the natural view of the user, providing convincing and immersive three dimensional augmented guidance, with probing and resection accuracies of 0.55±0.04 and 0.34±0.04 mm, respectively

First results on ProtoDUNE-SP liquid argon time projection chamber performance from a beam test at the CERN Neutrino Platform

The ProtoDUNE-SP detector is a single-phase liquid argon time projection chamber with an active volume of 7.2× 6.1× 7.0 m3. It is installed at the CERN Neutrino Platform in a specially-constructed beam that delivers charged pions, kaons, protons, muons and electrons with momenta in the range 0.3 GeV/c to 7 GeV/c. Beam line instrumentation provides accurate momentum measurements and particle identification. The ProtoDUNE-SP detector is a prototype for the first far detector module of the Deep Underground Neutrino Experiment, and it incorporates full-size components as designed for that module. This paper describes the beam line, the time projection chamber, the photon detectors, the cosmic-ray tagger, the signal processing and particle reconstruction. It presents the first results on ProtoDUNE-SP\u27s performance, including noise and gain measurements, dE/dx calibration for muons, protons, pions and electrons, drift electron lifetime measurements, and photon detector noise, signal sensitivity and time resolution measurements. The measured values meet or exceed the specifications for the DUNE far detector, in several cases by large margins. ProtoDUNE-SP\u27s successful operation starting in 2018 and its production of large samples of high-quality data demonstrate the effectiveness of the single-phase far detector design