Optical Synchronization of Time-of-Flight Cameras

Time-of-Flight (ToF)-Kameras erzeugen Tiefenbilder (3D-Bilder), indem sie Infrarotlicht aussenden und die Zeit messen, bis die Reflexion des Lichtes wieder empfangen wird. Durch den Einsatz mehrerer ToF-Kameras können ihre vergleichsweise geringere Auflösungen überwunden, das Sichtfeld vergrößert und Verdeckungen reduziert werden. Der gleichzeitige Betrieb birgt jedoch die Möglichkeit von Störungen, die zu fehlerhaften Tiefenmessungen führen. Das Problem der gegenseitigen Störungen tritt nicht nur bei Mehrkamerasystemen auf, sondern auch wenn mehrere unabhängige ToF-Kameras eingesetzt werden. In dieser Arbeit wird eine neue optische Synchronisation vorgestellt, die keine zusätzliche Hardware oder Infrastruktur erfordert, um ein Zeitmultiplexverfahren (engl. Time-Division Multiple Access, TDMA) für die Anwendung mit ToF-Kameras zu nutzen, um so die Störungen zu vermeiden. Dies ermöglicht es einer Kamera, den Aufnahmeprozess anderer ToF-Kameras zu erkennen und ihre Aufnahmezeiten schnell zu synchronisieren, um störungsfrei zu arbeiten. Anstatt Kabel zur Synchronisation zu benötigen, wird nur die vorhandene Hardware genutzt, um eine optische Synchronisation zu erreichen. Dazu wird die Firmware der Kamera um das Synchronisationsverfahren erweitert. Die optische Synchronisation wurde konzipiert, implementiert und in einem Versuchsaufbau mit drei ToF-Kameras verifiziert. Die Messungen zeigen die Wirksamkeit der vorgeschlagenen optischen Synchronisation. Während der Experimente wurde die Bildrate durch das zusätzliche Synchronisationsverfahren lediglich um etwa 1 Prozent reduziert.Time-of-Flight (ToF) cameras produce depth images (three-dimensional images) by measuring the time between the emission of infrared light and the reception of its reflection. A setup of multiple ToF cameras may be used to overcome their comparatively low resolution, increase the field of view, and reduce occlusion. However, the simultaneous operation of multiple ToF cameras introduces the possibility of interference resulting in erroneous depth measurements. The problem of interference is not only related to a collaborative multicamera setup but also to multiple ToF cameras operating independently. In this work, a new optical synchronization for ToF cameras is presented, requiring no additional hardware or infrastructure to utilize a time-division multiple access (TDMA) scheme to mitigate interference. It effectively enables a camera to sense the acquisition process of other ToF cameras and rapidly synchronizes its acquisition times to operate without interference. Instead of requiring cables to synchronize, only the existing hardware is utilized to enable an optical synchronization. To achieve this, the camera’s firmware is extended with the synchronization procedure. The optical synchronization has been conceptualized, implemented, and verified with an experimental setup deploying three ToF cameras. The measurements show the efficacy of the proposed optical synchronization. During the experiments, the frame rate was reduced by only about 1% due to the synchronization procedure

Enhanced single-photon time-of-flight 3D ranging

We developed a system for acquiring 3D depth-resolved maps by measuring the Time-of-Flight (TOF) of single photons. It is based on a CMOS 32 × 32 array of Single-Photon Avalanche Diodes (SPADs) and 350 ps resolution Time-to-Digital Converters (TDCs) into each pixel, able to provide photon-counting or photon-timing frames every 10 μs. We show how such a system can be used to scan large scenes in just hundreds of milliseconds. Moreover, we show how to exploit TDC unwarping and refolding for improving signal-to-noise ratio and extending the full-scale depth range. Additionally, we merged 2D and 3D information in a single image, for easing object recognition and tracking

On-chip fully reconfigurable Artificial Neural Network in 16 nm FinFET for Positron Emission Tomography

Smarty is a fully-reconfigurable on-chip feed-forward artificial neural network (ANN) with ten integrated time-to-digital converters (TDCs) designed in a 16 nm FinFET CMOS technology node. The integration of TDCs together with an ANN aims to reduce system complexity and minimize data throughput requirements in positron emission tomography (PET) applications. The TDCs have an average LSB of 53.5 ps. The ANN is fully reconfigurable, the user being able to change its topology as desired within a set of constraints. The chip can execute 363 MOPS with a maximum power consumption of 1.9 mW, for an efficiency of 190 GOPS/W. The system performance was tested in a coincidence measurement setup interfacing Smarty with two groups of five 4 mm x 4 mm analog silicon photomultipliers (A-SiPMs) used as inputs for the TDCs. The ANN successfully distinguished between six different positions of a radioactive source placed between the two photodetector arrays by solely using the TDC timestamps.Comment: 13 pages, 24 Figure

Hybrid non-destructive technique for volumetric defect analysis and reconstruction by remote laser induced ultrasound

This PhD thesis is devoted to the design, development and implementation of a non-contact hybrid non-destructive testing (NDT) method applied to the analysis of metallic objects that contain embedded defects or fractures. We propose a hybrid opto-acoustic technique that combines laser generated ultrasound as exciter and ultrasound transducers as receivers. This work envisages a detailed study of the detection and one, two or three-dimensional reconstruction of defects, using the proposed hybrid technique and its application as a remotely controlled non-contact NDT. Our device combines several advantages of both photonic and ultrasonic techniques, while reduces some of the drawbacks of both individual methods. Our method relay on the combination of experimental results with high-resolution signal processing procedures based on different mathematical algorithms. Our basic experimental setup uses a nanosecond pulsed laser at 532nm wavelength that impacts onto the surface of the object under study. The laser pulse is rapidly absorbed into a shallow volume of material and creates a localized thermo-elastic expansion inducing a broadband ultrasound pulse that propagate inside the material. The laser beam scans a selected area of the object surface, being remotely controlled by means of a programmable XY scanner. For each excitation point, the ultrasound waves propagate through the object are reflected or scattered by material 3D defects. They are detected by ultrasound transducers and recorded with a PC data-acquisition system for a further process and analysis. As a first step, the time of flight analysis provides enough data for the location and size of the defect in 1D view. The detection capabilities of internal defects in a metallic sample are studied by means of wavelet transform, chosen due to its multi-resolution time-frequency characteristics. A novel algorithm using a density-based spatial clustering is applied to the resulting time frequency maps to estimate the defect’s position. For the 2D visualization and reconstruction of the defects we extended the signal analysis using the synthetic aperture focusing technique (SAFT). We implement a novel 2D apodization window filtering applied along with the SAFT, and we show it removes undesired effects of the side lobes and wide-angle reflections of ultrasound waves, enhancing the reconstructed image of the defect. We move then towards the 3D analysis and reconstruction of defects and in this case we achieve and implement a fully non-contact and automatized experimental configuration allowing the scan areas on different object’s faces. The defect details are recorded from different angles/perspectives and a complete 3D reconstruction is achieved. Finally, we show our results on a complementary topic related to a particular case of the ultrasound propagation in solids. We were concerned on the physical understanding of the propagation and diffraction of ultrasound waves in solid materials from the first moment. The control of the diffraction pattern in solids, using an ultrasonic lens, would help focus/collimate the ultrasound reducing echoes and boundary reflections, resulting in a further improve NDT process. Phononic crystals have been used to regulate the diffraction and frequency response of ultrasonic waves traveling in fluids. However, they were much less studied in solid materials due to the difficulty of building the crystal and to high coupling losses. We perform detailed numerical simulations of the ultrasound propagation in a solid phononic crystal and we show focusing and the self-collimation effects. We further extend our analysis and couple our phononic crystal lens to a solid under study, showing that the diffraction control is preserved inside the target solid object trough the coupling material.Esta tesis doctoral versa sobre el diseño, estudio e implementación de un método híbrido, sin contacto, de ensayos no destructivos (NDT, non-destructive testing) para el análisis de objetos metálicos que contienen defectos o fracturas internas. Proponemos una técnica híbrida opto-acústica que combina ultrasonidos generados por impacto láser como excitador y transductores de ultrasonidos como receptores. El trabajo plantea un estudio detallado de la detección y reconstrucción en 1D, 2D y 3D de defectos presentes en un objeto metálico, usando la técnica híbrida de NDT sin contacto y controlado remotamente. Nuestro dispositivo presenta varias ventajas de las técnicas fotónicas y de ultrasonidos, reduciendo al mismo tiempo algunos inconvenientes de dichos métodos tomados por separado. Nuestro método combina resultados experimentales con simulaciones numéricas basadas en el procesado de señal de alta resolución. El montaje experimental consiste en un láser pulsado de ns a una longitud de onda de 532 nm, que impacta sobre la superficie del objeto. El pulso láser se absorbe, creando una expansión termoelástica localizada que induce un pulso de ultrasonidos de banda ancha que se propaga en el material. El láser, controlado remotamente, realiza un barrido sobre un área seleccionada de la superficie del objeto. Por cada punto de excitación, el ultrasonido se propaga a través del objeto y se refleja o dispersa en los defectos del material. Dichas ondas se detectan mediante transductores y se registran en un sistema de adquisición de datos para su ulterior procesado. En un primer paso, mediante el análisis del tiempo de vuelo, podemos localizar y determinar el tamaño del defecto en una vista 1D. Las capacidades de detección de defectos internos en una muestra metálica se estudian también mediante transformación wavelet debido a sus características de multi-resolución en tiempo y frecuencia. Se aplica un algoritmo novedoso de agrupamiento (clustering) espacial y se usan los mapas resultantes de tiempo y frecuencia para estimar la posición del defecto. Para la visualización 2D de los defectos ampliamos el análisis de la señal utilizando la técnica de focalización por apertura sintética (SAFT, synthetic aperture focusing technique). Implementamos un novedoso filtro de apodización 2D, juntamente con la técnica SAFT, y demostramos que elimina efectos no deseados, mejorando la resolución de la imagen reconstruida del defecto. El siguiente paso es un análisis y reconstrucción 3D. En este caso conseguimos una configuración experimental totalmente automatizada y sin contacto, permitiendo áreas de barrido sobre diferentes caras de un objeto. Los detalles de los defectos se registran desde diferentes ángulos, consiguiéndose una completa reconstrucción 3D. Finalmente, mostramos nuestros resultados en un tema complementario, relacionado con un caso particular de propagación de ultrasonidos en sólidos. Desde un primer momento, quisimos tener una comprensión física de la propagación y difracción de ondas de ultrasonidos en materiales sólidos. El control de los patrones de difracción en sólidos, mediante el uso de lentes ultrasónicas, ayudaría a la focalización/colimación del ultrasonido, reduciendo ecos y reflexiones en la superficie de contorno, mejorando del proceso de análisis NDT. Los cristales fonónicos se usan para regular la difracción y la respuesta en frecuencia de ondas de ultrasonido que se propagan en fluidos. No obstante, dichas estructuras se han estudiado mucho menos en materiales sólidos. Hemos realizado detalladas simulaciones numéricas de la propagación de ultrasonidos en un cristal fonónico sólido y hemos demostrado efectos de focalización y autocolimación. Finalmente hemos acoplado nuestra lente de cristal fonónico al sólido objeto de estudio, demostrando que el control de la difracción se conserva en el interior de dicho objeto a través del material de acoplamiento. Finalmente, proporcionamos una conclusión general sobre el trabajo declarado en esta tesis y un plan de trabajo futuro donde esta investigación puede extenderse y expandirse aún más a aplicaciones industriales en colaboración con el mercado de producciónPostprint (published version

Discovering user mobility and activity in smart lighting environments

"Smart lighting" environments seek to improve energy efficiency, human productivity and health by combining sensors, controls, and Internet-enabled lights with emerging “Internet-of-Things” technology. Interesting and potentially impactful applications involve adaptive lighting that responds to individual occupants' location, mobility and activity. In this dissertation, we focus on the recognition of user mobility and activity using sensing modalities and analytical techniques. This dissertation encompasses prior work using body-worn inertial sensors in one study, followed by smart-lighting inspired infrastructure sensors deployed with lights. The first approach employs wearable inertial sensors and body area networks that monitor human activities with a user's smart devices. Real-time algorithms are developed to (1) estimate angles of excess forward lean to prevent risk of falls, (2) identify functional activities, including postures, locomotion, and transitions, and (3) capture gait parameters. Two human activity datasets are collected from 10 healthy young adults and 297 elder subjects, respectively, for laboratory validation and real-world evaluation. Results show that these algorithms can identify all functional activities accurately with a sensitivity of 98.96% on the 10-subject dataset, and can detect walking activities and gait parameters consistently with high test-retest reliability (p-value < 0.001) on the 297-subject dataset. The second approach leverages pervasive "smart lighting" infrastructure to track human location and predict activities. A use case oriented design methodology is considered to guide the design of sensor operation parameters for localization performance metrics from a system perspective. Integrating a network of low-resolution time-of-flight sensors in ceiling fixtures, a recursive 3D location estimation formulation is established that links a physical indoor space to an analytical simulation framework. Based on indoor location information, a label-free clustering-based method is developed to learn user behaviors and activity patterns. Location datasets are collected when users are performing unconstrained and uninstructed activities in the smart lighting testbed under different layout configurations. Results show that the activity recognition performance measured in terms of CCR ranges from approximately 90% to 100% throughout a wide range of spatio-temporal resolutions on these location datasets, insensitive to the reconfiguration of environment layout and the presence of multiple users.2017-02-17T00:00:00

Development of an IoT solution for detergent supervision in industrial washing machines

Dupla diplomação dom a UTFPR - Universidade Tecnológica Federal do ParanáAutomation of industrial activities aim to improve the efficiency of the productive processes while reducing costs and increasing safety. In industrial laundries, the detergent level measurement is a key element for asset management, mainly due to the necessity of maintaining a continuous flow of washing processes. Therefore, this work presents a solution implemented in the industrial laundry reservoirs of Santa Casa da Misericórdia de Bragança, in Portugal, using an Internet of Things (IoT) approach, which integrates a Wi-Fi based measurement system, capable of monitoring and recording the detergent liquid level from reservoirs in real-time. Thereby, a microcontrolled system was developed to perform level measurements using an ultrasonic sensor, in which data is sent to a database and, through a web based platform, the client can remotely access the measurement results. In order to facilitate the physical installation of the developed hardware in the existent setup, a custom-made enclosure was designed and 3D printed.A automação das atividades industriais tem como objetivo melhorar a eficiência de processos produtivos, reduzindo custos e aumentando a segurança. Em lavanderias industriais, a medição de nível de detergente líquido é um elemento fundamental para o gerenciamento de ativos, principalmente devido à necessidade de manter um fluxo contínuo dos processos de lavagem. Dessa forma, o trabalho apresenta uma solução implementada nos reservatórios da lavanderia industrial da Santa Casa da Misericórdia de Bragança, em Portugal, usando uma abordagem de internet das coisas, na qual integra um sistema de medição com conexão Wi-Fi, capaz de monitorar e registrar o nível de detergente líquido dos reservatórios em tempo real. Com isso, foi desenvolvido um sistema microcontrolado responsável por realizar as medições de nível ulilizando sensor ultrasônico, na qual os dados são enviados para um banco de dados e, através de uma plataforma web, o cliente consiga acessar de forma remota o resultado das medições. Para facilitar a instalação do sistema nos reservatórios, um bujão foi desenhado sob medida e impresso em 3D

The BrightEyes-TTM: an open-source time-tagging module for fluorescence lifetime imaging microscopy applications

The aim of this Ph.D. work is to reason and show how an open-source multi-channel and standalone time-tagging device was developed, validated and used in combination with a new generation of single-photon array detectors to pursue super-resolved time-resolved fluorescence lifetime imaging measurements. Within the compound of time-resolved fluorescence laser scanning microscopy (LSM) techniques, fluorescence lifetime imaging microscopy (FLIM) plays a relevant role in the life-sciences field, thanks to its ability of detecting functional changes within the cellular micro-environment. The recent advancements in photon detection technologies, such as the introduction of asynchronous read-out single-photon avalanche diode (SPAD) array detectors, allow to image a fluorescent sample with spatial resolution below the diffraction limit, at the same time, yield the possibility of accessing the single-photon information content allowing for time-resolved FLIM measurements. Thus, super-resolved FLIM experiments can be accomplished using SPAD array detectors in combination with pulsed laser sources and special data acquisition systems (DAQs), capable of handling a multiplicity of inputs and dealing with the single-photons readouts generated by SPAD array detectors. Nowadays, the commercial market lacks a true standalone, multi-channel, single-board, time-tagging and affordable DAQ device specifically designed for super-resolved FLIM experiments. Moreover, in the scientific community, no-efforts have been placed yet in building a device that can compensate such absence. That is why, within this Ph.D. project, an open-source and low-cost device, the so-called BrightEyes-TTM (time tagging module), was developed and validated both for fluorescence lifetime and time-resolved measurements in general. The BrightEyes-TTM belongs to a niche of DAQ devices called time-to-digital converters (TDCs). The field-gate programmable array (FPGA) technology was chosen for implementing the BrightEyes-TTM thanks to its reprogrammability and low cost features. The literature reports several different FPGA-based TDC architectures. Particularly, the differential delay-line TDC architecture turned out to be the most suitable for this Ph.D. project as it offers an optimal trade-off between temporal precision, temporal range, temporal resolution, dead-time, linearity, and FPGA resources, which are all crucial characteristics for a TDC device. The goal of the project of pursuing a cost-effective and further-upgradable open-source time-tagging device was achieved as the BrigthEyes-TTM was developed and assembled using low-cost commercially available electronic development kits, thus allowing for the architecture to be easily reproduced. BrightEyes-TTM was deployed on a FPGA development board which was equipped with a USB 3.0 chip for communicating with a host-processing unit and a multi-input/output custom-built interface card for interconnecting the TTM with the outside world. Licence-free softwares were used for acquiring, reconstructing and analyzing the BrightEyes-TTM time-resolved data. In order to characterize the BrightEyes-TTM performances and, at the same time, validate the developed multi-channel TDC architecture, the TTM was firstly tested on a bench and then integrated into a fluorescent LSM system. Yielding a 30 ps single-shot precision and linearity performances that allows to be employed for actual FLIM measurements, the BrightEyes-TTM, which also proved to acquire data from many channels in parallel, was ultimately used with a SPAD array detector to perform fluorescence imaging and spectroscopy on biological systems. As output of the Ph.D. work, the BrightEyes-TTM was released on GitHub as a fully open-source project with two aims. The principal aim is to give to any microscopy and life science laboratory the possibility to implement and further develop single-photon-based time-resolved microscopy techniques. The second aim is to trigger the interest of the microscopy community, and establish the BrigthEyes-TTM as a new standard for single-photon FLSM and FLIM experiments

A High Rate Pixelated Neutron Detector for Neutron Reflectometry at the Spallation Neutron Source

This work presents the development of a high-rate 6Li-based pixelated neutron detector for neutron reflectometry instruments at the Spallation Neutron Source (SNS), Oak Ridge National Laboratory. The current detector technology falls short on the instrument requirements, particularly on the counting rate capability. This detector was designed specifically to overcome the limitation in counting rate by having a fully pixelated design from neutron conversion layer to photodetector and readout system. For the neutron converting layer, a 6Li-based neutron scintillator was used. Each scintillator element was coupled to a photodetector, in this case, a silicon photomultiplier (SiPM). The output of each SiPM was read out independently. This design has been demonstrated to have a significantly higher rate capability compared to other charge-sharing designs due to the fact that each pixel is fully functional on its own, without relying on adjacent pixels. An experimental prototype was constructed and characterized using various standardized tests such as counting rate, neutron detection efficiency, spatial resolution, γ sensitivity, crosstalk, and uniformity. Lastly, neutron reflectivity experiments were conducted using the detector prototype to demonstrate the performance and capability of the detector in practice

3D photogrammetric data modeling and optimization for multipurpose analysis and representation of Cultural Heritage assets

This research deals with the issues concerning the processing, managing, representation for further dissemination of the big amount of 3D data today achievable and storable with the modern geomatic techniques of 3D metric survey. In particular, this thesis is focused on the optimization process applied to 3D photogrammetric data of Cultural Heritage assets. Modern Geomatic techniques enable the acquisition and storage of a big amount of data, with high metric and radiometric accuracy and precision, also in the very close range field, and to process very detailed 3D textured models. Nowadays, the photogrammetric pipeline has well-established potentialities and it is considered one of the principal technique to produce, at low cost, detailed 3D textured models. The potentialities offered by high resolution and textured 3D models is today well-known and such representations are a powerful tool for many multidisciplinary purposes, at different scales and resolutions, from documentation, conservation and restoration to visualization and education. For example, their sub-millimetric precision makes them suitable for scientific studies applied to the geometry and materials (i.e. for structural and static tests, for planning restoration activities or for historical sources); their high fidelity to the real object and their navigability makes them optimal for web-based visualization and dissemination applications. Thanks to the improvement made in new visualization standard, they can be easily used as visualization interface linking different kinds of information in a highly intuitive way. Furthermore, many museums look today for more interactive exhibitions that may increase the visitors’ emotions and many recent applications make use of 3D contents (i.e. in virtual or augmented reality applications and through virtual museums). What all of these applications have to deal with concerns the issue deriving from the difficult of managing the big amount of data that have to be represented and navigated. Indeed, reality based models have very heavy file sizes (also tens of GB) that makes them difficult to be handled by common and portable devices, published on the internet or managed in real time applications. Even though recent advances produce more and more sophisticated and capable hardware and internet standards, empowering the ability to easily handle, visualize and share such contents, other researches aim at define a common pipeline for the generation and optimization of 3D models with a reduced number of polygons, however able to satisfy detailed radiometric and geometric requests. iii This thesis is inserted in this scenario and focuses on the 3D modeling process of photogrammetric data aimed at their easy sharing and visualization. In particular, this research tested a 3D models optimization, a process which aims at the generation of Low Polygons models, with very low byte file size, processed starting from the data of High Poly ones, that nevertheless offer a level of detail comparable to the original models. To do this, several tools borrowed from the game industry and game engine have been used. For this test, three case studies have been chosen, a modern sculpture of a contemporary Italian artist, a roman marble statue, preserved in the Civic Archaeological Museum of Torino, and the frieze of the Augustus arch preserved in the city of Susa (Piedmont- Italy). All the test cases have been surveyed by means of a close range photogrammetric acquisition and three high detailed 3D models have been generated by means of a Structure from Motion and image matching pipeline. On the final High Poly models generated, different optimization and decimation tools have been tested with the final aim to evaluate the quality of the information that can be extracted by the final optimized models, in comparison to those of the original High Polygon one. This study showed how tools borrowed from the Computer Graphic offer great potentialities also in the Cultural Heritage field. This application, in fact, may meet the needs of multipurpose and multiscale studies, using different levels of optimization, and this procedure could be applied to different kind of objects, with a variety of different sizes and shapes, also on multiscale and multisensor data, such as buildings, architectural complexes, data from UAV surveys and so on