Review of Recent Trends

This work was partially supported by the European Regional Development Fund (FEDER), through the Regional Operational Programme of Centre (CENTRO 2020) of the Portugal 2020 framework, through projects SOCA (CENTRO-01-0145-FEDER-000010) and ORCIP (CENTRO-01-0145-FEDER-022141). Fernando P. Guiomar acknowledges a fellowship from “la Caixa” Foundation (ID100010434), code LCF/BQ/PR20/11770015. Houda Harkat acknowledges the financial support of the Programmatic Financing of the CTS R&D Unit (UIDP/00066/2020).MIMO-OFDM is a key technology and a strong candidate for 5G telecommunication systems. In the literature, there is no convenient survey study that rounds up all the necessary points to be investigated concerning such systems. The current deeper review paper inspects and interprets the state of the art and addresses several research axes related to MIMO-OFDM systems. Two topics have received special attention: MIMO waveforms and MIMO-OFDM channel estimation. The existing MIMO hardware and software innovations, in addition to the MIMO-OFDM equalization techniques, are discussed concisely. In the literature, only a few authors have discussed the MIMO channel estimation and modeling problems for a variety of MIMO systems. However, to the best of our knowledge, there has been until now no review paper specifically discussing the recent works concerning channel estimation and the equalization process for MIMO-OFDM systems. Hence, the current work focuses on analyzing the recently used algorithms in the field, which could be a rich reference for researchers. Moreover, some research perspectives are identified.publishersversionpublishe

Hyperspectral Imaging for Real-Time Unmanned Aerial Vehicle Maritime Target Detection

The hyperspectral cameras use has been increasing over the past years, driven by the exponential growth of the computational systems power. The capability of acquiring multiple spectre wavelengths benefits the increase of the hyperspectral systems range of applications. However, until now, most hyperspectral systems are used in posprocessing and do not allow to take full advantage of the system capabilities. There is a recent trend to be able to use hyperspectral systems in real-time. Given the recent problems in European Union borders due to irregular immigration and drug smuggling, there is the need to develop novel autonomous surveillance systems that can work on these scenarios. This thesis addresses the scenario of using hyperspectral imaging systems for maritime target detection using unmanned aerial vehicles. Specifically, by working in the creation of a hyperspectral real-time data processing system pipeline. In our work, we develop a boresight calibration method that allows to calibrate the position of the navigation sensor related to the camera imaging sensor, and improve substantially the accuracy of the target geo-reference. We also develop a novel method of distinguish targets (boats) from their dominant background. With this application our system is able to only select relevant information to send to a remote station on the ground, thus making it suitable to be installed in an actual unmanned maritime surveillance system.A utilização de câmaras hiperespectrais tem vindo a aumentar nos últimos anos, motivada pelo crescimento exponencial da capacidade de processamento dos mais recentes sistemas computacionais. A sua aptidão para observar múltiplos comprimentos de onda beneficia aplicações em diferentes campos de atividade. No entanto, a maior parte das aplicações com câmaras hiperespectrais são realizadas em pós-processamento, não aproveitando totalmente as capacidades destes sistemas. Existe uma necessidade emergente de detetar mais características sobre o cenário que está a ser observado, incentivando o desenvolvimento de sistemas hiperespectrais capazes de adquirir e processar informação em tempo-real. Face aos mais recentes problemas de emigração e contrabando ilegal na União Europeia, surge a necessidade da realização de vigilância autónoma capaz de adquirir o máximo de informação possível sobre os meios envolventes presentes num dado percurso. E neste contexto que se insere a dissertação que visa a criação é implementação de um sistema hiperespectral em tempo-real. Para construir o sistema, foi necessário dividir o problema em diferentes etapas. Iniciou-se por um estudo detalhado dos sistemas hiperespectrais, desenvolvendo um método de calibração dos ângulos de boresight, que permitiu calibrar a relação entre o sistema de posicionamento e navegação da câmara hiperespectral e o sensor imagem. Esta calibração, permite numa fase posterior geo-referenciar os alvos com maior precisão. Posteriormente, foi criada uma pipeline de processamento, que permite analisar os espectros obtidos, distinguindo os alvos do cenário onde estão inseridos. Após a deteção dos alvos, procede-se `a sua geo-referenciação, de forma a obter as coordenadas UTM do alvo. Toda a informação obtida sobre o alvo e a sua posição é enviada para uma estacão em terra, de forma a ser validada por um humano. Para tal, foi também desenvolvida a metodologia de envio, para selecionar a informação a enviar apenas à mais relevante

Portable Ultrasound Imaging

This PhD project investigates hardware strategies and imaging methods for hand-held ultrasound systems. The overall idea is to use a wireless ultrasound probe linked to general-purpose mobile devices for the processing and visualization. The approach has the potential to reduce the upfront costs of the ultrasound system and, consequently, to allow for a wide-scale utilization of diagnostic ultrasound in any medical specialties and out of the radiology department. The first part of the contribution deals with the study of hardware solutions for the reduction of the system complexity. Analog and digital beamforming strategies are simulated from a system-level perspective. The quality of the B-mode image is evaluated and the minimum specifications are derived for the design of a portable probe with integrated electronics in-handle. The system is based on a synthetic aperture sequential beamforming approach that allows to significantly reduce the data rate between the probe and processing unit. The second part investigates the feasibility of vector flow imaging in a hand-held ultrasound system. Vector flow imaging overcomes the limitations of conventional imaging methods in terms of flow angle compensation. Furthermore, high frame rate can be obtained by using synthetic aperture focusing techniques. A method is developed combining synthetic aperture sequential beamforming and directional transverse oscillation to achieve the wireless transmission of the data along with a relatively inexpensive 2-D velocity estimation. The performance of the method is thoroughly assessed through simulations and measurements, and in vivo investigations are carried out to show its potential in presence of complex flow dynamics. A sufficient frame rate is achieved to allow for the visualization of vortices in the carotid bifurcation. Furthermore, the method is implemented on a commercially available tablet to evaluate the real-time processing performance in the built-in GPU with concurrent wireless transmission of the data. Based on the demonstrations in this thesis, a flexible framework can be implemented with performance that can be scaled to the needs of the user and according to the computing resources available. The integration of high-frame-rate vector flow imaging in a hand-held ultrasound scanner, in addition, has the potential to improve the operator’s workflow and opens the way to new possibilities in the clinical practice

A contribution to unobtrusive video-based measurement of respiratory signals

Due to the growing popularity of video-based methods for physiological signal measurement, and taking into account the technological advancements of these type of devices, this work proposes a series of new novel methods to obtain the respiratory signal from a distance, based on video analysis. This thesis aims to improve the state of the art video methods for respiratory measurement, more specifically, by presenting methods that can be used to obtain respiratory variability or perform respiratory rhythm measurements. Moreover, this thesis also aims to present a new implementation of a time-frequency signal processing technique, to improve its computational efficiency when applied to the respiratory signals. In this document a first approach to video-based methods for respiratory signal measurement is performed, to assert the feasibility of using a consumer-grade camera, not only to measure the mean respiratory rate or frequency, but to assert if this hardware could be used to acquire the raw respiratory signal and the respiratory rhythm as well. In this regard a new video-based method was introduced that measures the respiratory signal of a subject at a distance, with the aid of a custom pattern placed on the thorax of the subject. Given the results from the first video-based method, a more broad approach was taken by comparing three different types of video hardware, with the aim to characterise if they could be used for respiratory signal acquisition and respiratory variability measurements. The comparative analysis was performed in terms of instantaneous frequency, as it allowed to characterise the methods in terms of respiratory variability and to compare them in the same terms with the reference method. Subsequently, and due to the previous obtained results, a new method was proposed using a stereo depth camera with the aim to tackle the limitations of the previous study. The proposed method uses an hybrid architecture were the synchronized infrared frame and depth point-cloud from the same camera are acquired. The infrared frame is used to detect the movements of the subject inside the scene, and to recompute on demand a region of interest to obtain the respiratory signal from the depth point-cloud. Furthermore, in this study an opportunistic approach is taken in order to process all the obtained data, as it is also the aim of this study to verify if using a more realistic approach to respiratory signal analysis in real-life conditions, would influence the respiratory rhythm measurement. Even though the depth camera method proved reliable in terms of respiratory rhythm measurement, the opportunistic approach relied on visual inspection of the obtained respiratory signal to properly define each piece. For this reason, a quality indicator had to be proposed that could objectively identify whenever a respiratory signal contained errors. Furthermore, from the idea to characterise the movements of a subject, and by changing the measuring point from a frontal to a lateral perspective to avoid most of the occlusions, a new method based on obtaining the movement of the thoraco-abdominal region using dense optical flow was proposed. This method makes us of the phase of the optical flow to obtain the respiratory signal of the subject, while using the modulus to compute a quality index. Finally, regarding the different signal processing methods used in this thesis to obtain the instantaneous frequency, there were none that could perform in real-time, making the analysis of the respiratory variability not possible in real-life systems where the signals have to be processed in a sample by sample basis. For this reason, as a final chapter a new implementation of the synchrosqueezing transform for time-frequency analysis in real-time is proposed, with the aim to provide a new tool for non-contact methods to obtain the variability of the respiratory signal in real-time.A causa de la creixent popularitat en la mesura de senyals fisiològics amb mètodes de vídeo, i tenint en compte els avenços tecnològics d'aquests dispositius, aquesta tesi proposa una sèrie de nous mètodes per tal d'obtenir la respiració a distància mitjançant l'anàlisi de vídeo. Aquesta tesi té com a objectiu millorar l'estat de l'art referent a la mesura de senyal respiratòria mitjançant els mètodes que en ella es descriuen, així com presentar mètodes que puguin ser usats per obtenir la variabilitat o el ritme respiratori. A més, aquesta tesi té com a objectiu presentar una nova implementació d'un mètode de processat de senyal temps-freqüencial, per tal de millorar-ne l'eficiència computacional quant s’aplica a senyals respiratoris. En aquest document, es realitza una primera aproximació a la mesura de senyal respiratòria mitjançant mètodes de vídeo per tal de verificar si és factible utilitzar una càmera de consum, no només per mesurar el senyal respiratori, sinó verificar si aquest tipus de hardware també pot ser emprat per obtenir el ritme respiratori. En aquest sentit, es presenta en aquest document un nou mètode d'adquisició de senyal respiratòria a distància basat en vídeo, el qual fa ús d'un patró ubicat al tòrax del subjecte per tal d'obtenir-ne la respiració. Un cop obtinguts els resultats del primers resultats, s'han analitzat tres tipus diferents de càmeres, amb la finalitat de caracteritzar-ne la viabilitat d'obtenir el senyal respiratori i la seva variabilitat. L'estudi comparatiu s'ha realitzat en termes de freqüència instantània, donat que permet caracteritzar els mètodes en termes de variabilitat respiratòria i comparar-los, en les mateixes condicions, amb el mètode de referencia. A continuació, s'ha presentat un nou mètode basat en una càmera de profunditat estèreo amb la finalitat de millorar i corregir les limitacions anteriors. El nou mètode proposat es basa en una arquitectura hibrida la qual utilitza els canals de vídeo infraroig i de profunditat de forma sincronitzada. El canal infraroig s'utilitza per detectar els moviments del subjecte dins l'escena i calcular, sota demanda, una regió d'interès que s'utilitza posteriorment en el canal de profunditat per extreure el senyal respiratori. A més a més, en aquest estudi s'ha utilitzat una aproximació oportunista en el processat del senyal respiratori, donat que també és un dels objectius d'aquest estudi, verificar si el fet d'utilitzar una aproximació més realista en l'adquisició de senyal, pot influir en la mesura del ritme respiratori. Tot i que el mètode anterior es mostra fiable en termes de mesura del ritme respiratori, la selecció oportunista del senyal necessita d’inspecció visual per tal de definir correctament cada fragment. Per aquest motiu, era necessari definir un índex de qualitat el qual permetés identificar de forma objectiva cada tram de senyal, així com detectar si el senyal conté errors. Partint de la idea de caracteritzar el moviment del subjecte de l'estudi anterior, i modificant el punt de mesura frontal cap a un de lateral per tal d'evitar oclusions, es proposa un nou mètode basat en l'obtenció del moviment toràcic-abdominal a partir del flux òptic del senyal de vídeo. Aquest mètode recupera el senyal respiratori del subjecte a partir de la fase del flux òptic, tot calculant un índex de qualitat a partir del mòdul. Finalment, i tenint en compte els diferents mètodes de processat utilitzats en aquesta tesi per tal de obtenir la freqüència instantània, es pot apreciar que cap d'ells és capaç de funcionar en temps real, fent inviable l'anàlisi de la variabilitat respiratòria en sistemes reals amb processat mostra a mostra. Per aquest motiu, en el capítol final d'aquesta tesi, s'ha proposat una nova implementació de la transformació "synchrosqueezing" per tal de realitzar l’anàlisi temporal-freqüencial en temps real, i proveir d'una nova eina per tal d'obtenir la variabilitat respiratòria en temps real, amb mètodes sense contacte

Towards Intelligent Data Acquisition Systems with Embedded Deep Learning on MPSoC

Large-scale scientific experiments rely on dedicated high-performance data-acquisition systems to sample, readout, analyse, and store experimental data. However, with the rapid development in detector technology in various fields, the number of channels and the data rate are increasing. For trigger and control tasks data acquisition systems needs to satisfy real-time constraints, enable short-time latency and provide the possibility to integrate intelligent data processing. During recent years machine learning approaches have been used successfully in many applications. This dissertation will study how machine learning techniques can be integrated already in the data acquisition of large-scale experiments. A universal data acquisition platform for multiple data channels has been developed. Different machine learning implementation methods and application have been realized using this system. On the hardware side, recent FPGAs do not only provide high-performance parallel logic but more and more additional features, like ultra-fast transceivers and embedded ARM processors. TSMC\u27s 16nm FinFET Plus (16FF+) 3D transistor technology enables Xilinx in the Zynq UltraScale+ FPGA devices to increase the performance/watt ratio by 2 to 5 times compared to their previous generation. The selected main processor ZU11EG owns 32 GTH transceivers where each one could operate up to $16.3$ Gb/s and 16 GTY transceivers where each of them could operate up to $32.75$ Gb/s. These transceivers are routed to x16 lanes Gen $3$/$4$ PCIe, $12$ lanes full-duplex FireFly electrical/optical data link and VITA 57.4 FMC+ connector. The new Zynq UltraScale+ device provides at least three major advantages for advanced data acquisition systems: First, the 16nm FinFET+ programmable logic (PL) provides high-speed readout capabilities by high-speed transceivers; second, built-in quad-core 64-bit ARM Cortex-A53 processor enable host embedded Linux system. Thus, webservers, slow control and monitoring application could be realized in a embedded processor environment; third, the Zynq Multiprocessor System-on-Chip technology connects programmable logic and microprocessors. In this thesis, the benefits of such architectures for the integration of machine learning algorithms in data acquisition systems and control application are demonstrated. On the algorithm side, there have been many achievements in the field of machine learning over the last decades. Existing machine learning algorithms split into several categories depending on how the learning phase is organized: Supervised Learning, Unsupervised Learning, Semi-Supervised Learning and Reinforcement Learning. Most commonly used in scientific applications are supervised learning and reinforcement learning. Supervised learning learns from the labelled input and output, and generates a function that could predict the future different input to the appropriate output. A common application instance is a classification. They have a wide difference in basic math theory, training, inference, and their implementation. One of the natural solutions is Application Specific Integrated Circuit (ASIC) Artificial Intelligence (AI) chips. A typical example is the Google Tensor Processing Unit (TPU), it could cover the training and inference for both supervised learning and reinforcement learning. One of the major issues is that such chip could not provide high data transferring bandwidth other than high compute power. As a comparison, the Xilinx UltraScale+ FPGA could also provide raw compute power and efficiency for all different data types down to a single bit. From a deployment point of view, the training part of supervised learning is typically performed by CPU/GPU/TPU on a fixed dataset. For reinforcement learning, the training phase is more complex. The algorithm needs to periodically interact with the controlled system and execute a Markov Decision Process (MDP). There is no static training dataset, but it is obtained in real-time. The time slot between each step depends on the dynamics of the controlled system. The inference is also bound to this sampling time because the algorithm needs to interact with the environment and decide the appropriate action for a response, then a higher demand on time is proposed. This thesis gives solutions for both training and inference of reinforcement learning. At first, the requirements are analyzed, then the algorithm is deduced from scratch, and training on the PS part of Zynq device is implemented, meanwhile the inference at FPGA side is proposed which is similar solution compared with supervised learning. The results for Policy Gradient show a lot of improvement over a CPU/GPU-based machine learning framework. The Deep Deterministic Policy Gradient also has improvement regarding both training latency and stability. This implementation method provides a low-latency approach for reinforcement learning on-field training process

Computational Methods for Photon-Counting and Photon- Processing Detectors

We present computational methods for attribute estimation of photon-counting and photon-processing detectors. We define a photon-processing detector as any imaging device that uses maximum-likelihood methods to estimate photon attributes, such as position, direction of propagation and energy. Estimated attributes are then stored at full precision in the memory of a computer. Accurate estimation of a large number of attributes for each collected photon does require considerable computational power. We show how mass-produced graphics processing units (GPUs) are viable parallel computing solutions capable of meeting the required computing needs of photon-counting and photon-processing detectors, while keeping overall costs affordable