A Single Chip System for Sensor Data Fusion Based on a Drift-diffusion Model

Current multisensory system face data communication overhead in integrating disparate sensor data to build a coherent and accurate global phenomenon. We present here a novel hardware and software co-design platform for a heterogeneous data fusion solution based on a perceptual decision making approach (the drift-diffusion model). It provides a convenient infrastructure for sensor data acquisition and data integration and only uses a single chip Xilinx ZYNQ-7000 XC7Z020 AP SOC. A case study of controlling the moving speed of a single ground-based robot, according to physiological states of the operator based on heart rates, is conducted and demonstrates the possibility of integrated sensor data fusion architecture. The results of our DDM-based data integration shows a better correlation coefficient with the raw ECG signal compare with a simply piecewise approach

CABE : a cloud-based acoustic beamforming emulator for FPGA-based sound source localization

Microphone arrays are gaining in popularity thanks to the availability of low-cost microphones. Applications including sonar, binaural hearing aid devices, acoustic indoor localization techniques and speech recognition are proposed by several research groups and companies. In most of the available implementations, the microphones utilized are assumed to offer an ideal response in a given frequency domain. Several toolboxes and software can be used to obtain a theoretical response of a microphone array with a given beamforming algorithm. However, a tool facilitating the design of a microphone array taking into account the non-ideal characteristics could not be found. Moreover, generating packages facilitating the implementation on Field Programmable Gate Arrays has, to our knowledge, not been carried out yet. Visualizing the responses in 2D and 3D also poses an engineering challenge. To alleviate these shortcomings, a scalable Cloud-based Acoustic Beamforming Emulator (CABE) is proposed. The non-ideal characteristics of microphones are considered during the computations and results are validated with acoustic data captured from microphones. It is also possible to generate hardware description language packages containing delay tables facilitating the implementation of Delay-and-Sum beamformers in embedded hardware. Truncation error analysis can also be carried out for fixed-point signal processing. The effects of disabling a given group of microphones within the microphone array can also be calculated. Results and packages can be visualized with a dedicated client application. Users can create and configure several parameters of an emulation, including sound source placement, the shape of the microphone array and the required signal processing flow. Depending on the user configuration, 2D and 3D graphs showing the beamforming results, waterfall diagrams and performance metrics can be generated by the client application. The emulations are also validated with captured data from existing microphone arrays.</jats:p

Autonomous Pedestrian Detection in Transit Buses

This project created a proof of concept for an automated pedestrian detection and avoidance system designed for transit buses. The system detects objects up to 12 meters away, calculates the distance from the system using a solid-state LIDAR, and determines if that object is human by passive infrared. This triggers a visual and sound warning. A Xilinx Zynq-SoC utilizing programmable logic and an ARM-based processing system drive data fusion, and an external power unit makes it configurable for transit-buses

HW/SW Co-design and Prototyping Approach for Embedded Smart Camera: ADAS Case Study

In 1968, Volkswagen integrated an electronic circuit as a new control fuel injection system, called the “Little Black Box”, it is considered as the first embedded system in the automotive industry. Currently, automobile constructors integrate several embedded systems into any of their new model vehicles. Behind these automobile’s electronics systems, a sophisticated Hardware/Software (HW/SW) architecture, which is based on heterogeneous components, and multiple CPUs is built. At present, they are more oriented toward visionbased systems using tiny embedded smart camera. This visionbased system in real time aspects represents one of the most challenging issues, especially in the domain of automobile’s applications. On the design side, one of the optimal solutions adopted by embedded systems designer for system performance, is to associate CPUs and hardware accelerators in the same design, in order to reduce the computational burden on the CPU and to speed-up the data processing. In this paper, we present a hardware platform-based design approach for fast embedded smart Advanced Driver Assistant System (ADAS) design and prototyping, as an alternative for the pure time-consuming simulation technique. Based on a Multi-CPU/FPGA platform, we introduced a new methodology/flow to design the different HW and SW parts of the ADAS system. Then, we shared our experience in designing and prototyping a HW/SW vision based on smart embedded system as an ADAS that helps to increase the safety of car’s drivers. We presented a real HW/SW prototype of the vision ADAS based on a Zynq FPGA. The system detects the fatigue/drowsiness state of the driver by monitoring the eyes closure and generates a real time alert. A new HW Skin Segmentation step to locate the eyes/face is proposed. Our new approach migrates the skin segmentation step from processing system (SW) to programmable logic (HW) taking the advantage of High-Level Synthesis (HLS) tool flow to accelerate the implementation, and the prototyping of the Vision based ADAS on a hardware platform

Embedded landmark acquisition system for visual slam using star identification based stereo correspondence descriptor

Orientador : Prof. Dr. Eduardo TodtDissertação (mestrado) - Universidade Federal do Paraná, Setor de Ciências Exatas, Programa de Pós-Graduação em Informática. Defesa: Curitiba, 13/04/2015Inclui referênciasResumo: O uso de câmeras como sensores principais em Localização e Mapeamento Simultâneos (Simultaneous Localization and Mapping), o que é denominado SLAM Visual (Visual SLAM), tem crescido recentemente devido à queda nos preços das câmeras. Ao mesmo tempo em que imagens trazem informações mais ricas do que outros sensores típicos empregados em aplicações SLAM, como lasers e sonares, há um custo adicional de processamento significativo quando elas são utilizadas. A informação de profundidade adicional proveniente de configurações estéreo de câmeras às fazem mais interessantes para aplicações SLAM. Nesta abordagem em especial, grande parte do custo de processamento adicional vem da extração de pontos únicos ou pedaços em ambas as imagens em estéreo e da solução do problema de correspondência entre eles. Com posse dessa informação, a disparidade horizontal entre o par de imagens pode ser utilizada para recuperar a informação de profundidade. Esse trabalho explora a utilização de uma plataforma embarcada do tipo system-ona- chip (SoC) que integra um processador ARM multinúcleo com lógica FPGA como um módulo de processamento para visão estéreo. O detector de cantos Harris e Stephens (Harris & Stephens, 1988) é usado para encontrar pontos de interesse (Points of Interest, POIs) em imagens estéreo em um coprocessador soft sintetizado no FPGA para acelerar a extração de características e livrar o processador principal deste processo altamente paralelizável. As tarefas restantes tais como correção das imagens pela calibração de câmeras, encontrar um descritor único para as características detectadas e a correspondência entre os POIs no par de imagens estéreo são solucionadas em software executando no processador principal. A arquitetura proposta para o coprocessador permite que a tarefa de extração de cantos seja executada em aproximadamente metade do tempo necessário pelo processador principal sem auxílio algum. Após encontrar os POIs, para cada um dos pontos um descritor único é necessário para que seja possível encontrar o POI correspondente na outra imagem. Esse trabalho também propõe um descritor inovador que considera o relacionamento espacial bidimensional global entre os pontos detectados para descrevê-los individualmente. Para cada imagem, cada ponto da nuvem de pontos detectada pelo algoritmo de Harris e Stephens é descrito considerando-se apenas as posições relativas entre ele e seus vizinhos. Quando somente a posição é considerada, um padrão de céu estrelado noturno é formado pelos POIs. Com o padrão de POIs sendo considerado como estrelas, descritores já utilizados em problemas de identificação de estrelas podem ser reaplicados para identificar unicamente POIs. Um protótipo do descritor baseado do algoritmo de grade de Padgett e KreutzDelgado (Padgett & KreutzDelgado, 1997) é escrito e seus resultados comparados com os descritores normalmente utilizados para este propósito, mostrando que a informação espacial bidimensional pode ser utilizada por si só para resolver o problema de correspondência. O número de correspondências úteis é comparável ao atingido com o SIFT, o descritor com melhor desempenho neste quesito, enquanto a velocidade foi superior ao BRIEF, o descritor mais rápido utilizado na comparação, na plataforma ARM, com um speedup de 1,64 e 1,40 nas bases de dados dos testes. Palavras-chave: Harris; FPGA; SLAM; Hardware Reconfigurável; VHDL; Processamento de Imagem; Visão Estéreo; Computer Vision; Arquitetura Híbrida; Sistemas Embarcados; Pontos de Interesse; Keypoints; Correspondência; Correspondência Estéreo; Identificação de Estrelas; Descrição de Características; Percepção de Profundidade.Abstract: The use of cameras as the main sensors in Simultaneous Localization and Mapping, what is called Visual SLAM, has risen recently due to the fall in camera prices. While images bring richer information than other typical SLAM sensors, such as lasers and sonars, there is significant extra processing cost when they are used. The extra depth information available from stereo camera setups makes them preferable for SLAM applications. In this particular approach, great part of the added processing cost comes from extracting unique points or image patches in both stereo images and solving the correspondence problem between them. With this information, the horizontal disparity between the pair can be used to retrieve depth information. This work explores the use of an embedded system-on-a-chip (SoC) platform that integrates a multicore ARM processor with FPGA fabric as a stereo vision processing module. The Harris and Stephens corner detector (Harris & Stephens, 1988) is used to find Point of Interests (POIs) in stereo images in a hardware soft co-processor synthesized in the FPGA to speed up feature extraction and relieve this highly parallelizable process from the main embedded processor. Remaining tasks such as image correction from camera calibration, finding unique descriptor for the detected features and the correspondence between POIs in the stereo pair are solved in software running on the main processor. The proposed architecture for the co-processor enabled the corner extraction task to be performed in about half the time taken by the main processor without aid. After finding the POIs, for each point a unique descriptor is needed for finding the correspondent POI in the other image. This work also proposes an innovative descriptor that considers a global two-dimensional spatial relationship between the detected points to describe them individually. In each image, every point in the cloud of points detected by the Harris and Stephens algorithm is described by considering only the relative position between it and its neighbors. When position alone is considered, a starry night pattern is formed by the POIs. With the POI pattern being considered as stars, the descriptors already used in star identification problems can be reapplied to uniquely identify POIs. A prototype of the descriptor based on the Padgett and KreutzDelgado's grid algorithm (Padgett & KreutzDelgado, 1997) is written and the results compared with common descriptors used for this purpose, showing that two-dimensional spatial information alone can be used to solve the correspondence problem. The number of useful matches was comparable to what was obtained with SIFT, the best performing descriptor in this matter, while the speed was superior to BRIEF, the fastest descriptor used in the comparison, on the ARM platform, with a speedup of 1.64 and 1.40 on the tested datasets. Keywords: Harris; FPGA; SLAM; Reconfigurable Hardware; VHDL; Image Processing; Stereo Vision; Computer Vision; Hybrid Architecture; Embedded Systems; Point Of Interest; Keypoints; Matching; Stereo Correspondence; Star Identification; Feature Description; Depth Perception

Computer-Assisted Algorithms for Ultrasound Imaging Systems

Ultrasound imaging works on the principle of transmitting ultrasound waves into the body and reconstructs the images of internal organs based on the strength of the echoes. Ultrasound imaging is considered to be safer, economical and can image the organs in real-time, which makes it widely used diagnostic imaging modality in health-care. Ultrasound imaging covers the broad spectrum of medical diagnostics; these include diagnosis of kidney, liver, pancreas, fetal monitoring, etc. Currently, the diagnosis through ultrasound scanning is clinic-centered, and the patients who are in need of ultrasound scanning has to visit the hospitals for getting the diagnosis. The services of an ultrasound system are constrained to hospitals and did not translate to its potential in remote health-care and point-of-care diagnostics due to its high form factor, shortage of sonographers, low signal to noise ratio, high diagnostic subjectivity, etc. In this thesis, we address these issues with an objective of making ultrasound imaging more reliable to use in point-of-care and remote health-care applications. To achieve the goal, we propose (i) computer-assisted algorithms to improve diagnostic accuracy and assist semi-skilled persons in scanning, (ii) speckle suppression algorithms to improve the diagnostic quality of ultrasound image, (iii) a reliable telesonography framework to address the shortage of sonographers, and (iv) a programmable portable ultrasound scanner to operate in point-of-care and remote health-care applications

Front-ends para LiDAR baseados em ADC e TDC

Autonomous vehicles are a promising technology to save over a million lives each year that are lost in road accidents. However, bringing safe autonomous vehicles to market requires massive development, starting with vision sensors. LiDAR is a fundamental vision sensor for autonomous vehicles, as it enables high resolution 3D vision. However, automotive LiDAR is not yet a mature technology, and, also requires massive development in many aspects. This thesis aims to contribute to the maturity of LiDAR, focusing on sampling architectures for LiDAR front-ends. Two architectures were developed. The first is based on a pipelined ADC, available from an AD-FMCDAQ2-EBZ board. The ADC is synchronized with the emitted pulse and able to sample at 1 Gsample/s. The second architecture is based on a TDC that is directly implemented in an FPGA. It relies on a tapped delay line topology comprising 45 delay elements and on a mux-based decoder, resulting in a resolution of 50 ps. Preliminary test results show that both implementations operate correctly, and are both suitable for sampling short pulses typically used by LiDARs. When comparing both architectures, we conclude that an ADC consumes a significant amount of power, and uses many FPGA resources. However, it samples the LiDAR waveform without any loss of information, therefore enabling maximum range and precision. The TDC is just the opposite: it consumes little power, and uses less FPGA resources. However, it only captures one sample per pulse.Os veículos autónomos são uma tecnologia promissora para salvar mais de um milhão de vidas por ano, colhidas por acidentes rodoviários. Contudo, colocar veículos autónomos seguros no mercado requer inúmeros desenvolvimentos, a começar por sensores de visão. O LiDAR é um sensor de visão fundamental para veículos autónomos, pois permite uma visão 3D de alta resolução. Contudo, o LiDAR automotivo não é uma tecnologia madura, e portanto requer também desenvolvimento em vários aspectos. Esta dissertação visa contribuir para a maturidade do LiDAR, com foco em arquiteturas de amostragem para front-ends de LiDAR. Foram desenvolvidas duas arquiteturas. A primeira assenta numa ADC pipelined, por sua vez implementada numa placa de teste AD-FMCDAQ2-EBZ. A ADC opera em sincronismo com o pulso emitido, e permite capturar amostras a 1 Gsample/s. A segunda arquitetura assenta num TDC implementado diretamente numa FPGA. O TDC baseia-se numa topologia tapped delay line com 45 linhas de atraso, e num descodificador à base de multiplexers, permitindo uma resolução temporal de 50 ps. Resultados preliminares mostram que ambas as implementações operam corretamente, e são adequadas para amostrar pulsos curtos tipicamente associados a LiDAR. Em termos comparativos, a arquitectura com base numa ADC tem um consumo de potência considerável e requer uma quantidade significativa de recursos da FPGA. Contudo, esta permite amostrar a forma de onda de LiDAR sem nenhuma perda de informação, permitindo assim alcance e precisão máximos. A arquitectura com base num TDC é exatamente o oposto: tem um baixo consumo de potência e requer poucos recursos da FPGA. Contudo, permite capturar apenas uma amostra por pulso.Mestrado em Engenharia Eletrónica e Telecomunicaçõe

Advanced photonic and electronic systems - WILGA 2017

WILGA annual symposium on advanced photonic and electronic systems has been organized by young scientist for young scientists since two decades. It traditionally gathers more than 350 young researchers and their tutors. Ph.D students and graduates present their recent achievements during well attended oral sessions. Wilga is a very good digest of Ph.D. works carried out at technical universities in electronics and photonics, as well as information sciences throughout Poland and some neighboring countries. Publishing patronage over Wilga keep Elektronika technical journal by SEP, IJET by PAN and Proceedings of SPIE. The latter world editorial series publishes annually more than 200 papers from Wilga. Wilga 2017 was the XL edition of this meeting. The following topical tracks were distinguished: photonics, electronics, information technologies and system research. The article is a digest of some chosen works presented during Wilga 2017 symposium. WILGA 2017 works were published in Proc. SPIE vol.10445

Adaptive Noise Cancellation Algorithms Implemented onto FPGA-Based Electrical Impedance Tomography System

Electrical Impedance Tomography (EIT) as a non-invasive of electrical conductivity imaging method commonly employs the stationary-coefficient based filters (such as FFT) in order to remove the noise signal. In the practical applications, the stationary-coefficient based filters fail to remove the time-varying random noise which leads to the lack of impedance measurement sensitivity. In this paper, the implementation of adaptive noise cancellation (ANC) algorithms which are Least Mean Square (LMS) and Normalized Least Mean Square (NLMS) filters onto Field Programmable Gate Array (FPGA)-based EIT system is proposed in order to eliminate the time-varying random noise signal. The proposed method was evaluated through experimental studies with biomaterial phantom. The reconstructed EIT images with NLMS is better than the images with LMS by amplitude response AR = 12.5%, position error PE = 200%, resolution RES = 33%, and shape deformation SD = 66%. Moreover, the Analog-to-Digital Converter (ADC) performances of power spectral density (PSD) and the effective number of bit ENOB with NLMS is higher than the performances with LMS by SI = 5.7 % and ENOB = 15.4 %. The results showed that implementing ANC algorithms onto FPGA-based EIT system shows significantly more accurate image reconstruction as compared without ANC algorithms implementation

Co-design hardware/software of real time vision system on FPGA for obstacle detection

La détection, localisation d'obstacles et la reconstruction de carte d'occupation 2D sont des fonctions de base pour un robot navigant dans un environnement intérieure lorsque l'intervention avec les objets se fait dans un environnement encombré. Les solutions fondées sur la vision artificielle et couramment utilisées comme SLAM (simultaneous localization and mapping) ou le flux optique ont tendance a être des calculs intensifs. Ces solutions nécessitent des ressources de calcul puissantes pour répondre à faible vitesse en temps réel aux contraintes. Nous présentons une architecture matérielle pour la détection, localisation d'obstacles et la reconstruction de cartes d'occupation 2D en temps réel. Le système proposé est réalisé en utilisant une architecture de vision sur FPGA (field programmable gates array) et des capteurs d'odométrie pour la détection, localisation des obstacles et la cartographie. De la fusion de ces deux sources d'information complémentaires résulte un modèle amelioré de l'environnement autour des robots. L'architecture proposé est un système à faible coût avec un temps de calcul réduit, un débit d'images élevé, et une faible consommation d'énergieObstacle detection, localization and occupancy map reconstruction are essential abilities for a mobile robot to navigate in an environment. Solutions based on passive monocular vision such as simultaneous localization and mapping (SLAM) or optical flow (OF) require intensive computation. Systems based on these methods often rely on over-sized computation resources to meet real-time constraints. Inverse perspective mapping allows for obstacles detection at a low computational cost under the hypothesis of a flat ground observed during motion. It is thus possible to build an occupancy grid map by integrating obstacle detection over the course of the sensor. In this work we propose hardware/software system for obstacle detection, localization and 2D occupancy map reconstruction in real-time. The proposed system uses a FPGA-based design for vision and proprioceptive sensors for localization. Fusing this information allows for the construction of a simple environment model of the sensor surrounding. The resulting architecture is a low-cost, low-latency, high-throughput and low-power system