A Hardware-Oriented Dynamically Adaptive Disparity Estimation Algorithm and its Real-Time Hardware

The computational complexity of disparity estimation algorithms and the need of large size and bandwidth for the external and internal memory make the real-time processing of disparity estimation challenging, especially for High Resolution (HR) images. This paper proposes a hardware-oriented adaptive window size disparity estimation (AWDE) algorithm and its real time reconfigurable hardware implementation that targets HR video with high quality disparity results. The proposed algorithm is a hybrid solution involving the Sum of Absolute Differences and the Census cost computation methods to vote and select the best suitable disparity candidates. It utilizes a pixel intensity based refinement step to remove faulty disparity computations. The AWDE algorithm dynamically adapts the window size considering the local texture of the image to increase the disparity estimation quality. The proposed reconfigurable hardware of the AWDE algorithm enables handling 60 frames per second on Virtex-5 FPGA at a 1024×768 XGA video resolution for a 120 pixel disparity range

Compressed look-up-table based real-time rectification hardware

Stereo image rectification is a pre-processing step of disparity estimation intended to remove image distortions and to enable stereo matching along an epipolar line. A real-time disparity estimation system needs to perform real-time rectification which requires solving the models of lens distortions, image translations and rotations. Look-up-table based rectification algorithms allow image rectification without demanding high complexity operations. However, they require an external memory to store large size look-up-tables. In this work, we present an intermediate solution that compresses the rectification information to fit the look-up-table into the onchip memory of a Virtex-5 FPGA. The low-complexity decompression process requires a negligible amount of hardware resources for its real-time implementation. The proposed image rectification hardware consumes 0.28% of the DFF and 0.32% of the LUT resources of the Virtex-5 XCUVP-110T FPGA, it can process 347 frames per second for a 1024×768 pixels image resolution, and it does not need the availability of an external memory

3D image acquisition and processing with high continuous data throughput for human-machine-interaction and adaptive manufacturing

Many applications in industrial environment are able to detect structures in measurement volumes from macroscopic to microscopic range. One way to process the resulting image data and to calculate three-dimensional (3D) images is the use of active stereo vision technology. In this context, one of the main challenges is to deal with the permanently increasing amount of data. This paper aims to describes methods for handling the required data throughput for 3D image acquisition in active stereo vision systems. Thus, the main focus is on implementing the steps of the image processing chain on re-configurable hardware. Among other things, this includes the pre-processing step with the correction of distortion and rectification of incoming image data. Therefore, the approach uses the offline pre-calculation of rectification maps. Furthermore, with the aid of the rectified maps, each image is directly rectified during the image acquisition. Afterwards, an FPGA and GPU-based approach is selected for optimal performance of stereo matching and 3D point calculation

Semi-dense SLAM on an FPGA SoC

Deploying advanced Simultaneous Localisation and Mapping, or SLAM, algorithms in autonomous low-power robotics will enable emerging new applications which require an accurate and information rich reconstruction of the environment. This has not been achieved so far because accuracy and dense 3D reconstruction come with a high computational complexity. This paper discusses custom hardware design on a novel platform for embedded SLAM, an FPGA-SoC, combining an embedded CPU and programmable logic on the same chip. The use of programmable logic, tightly integrated with an efficient multicore embedded CPU stands to provide an effective solution to this problem. In this work an average framerate of more than 4 frames/second for a resolution of 320×240 has been achieved with an estimated power of less than 1 Watt for the custom hardware. In comparison to the software-only version, running on a dual-core ARM processor, an acceleration of 2× has been achieved for LSD-SLAM, without any compromise in the quality of the result

An Architecture for Configuring an Efficient Scan Path for a Subset of Elements

LaTeX4Web 1.4 OUTPUT Field Programmable Gate Arrays (FPGAs) have many modern applications. A feature of FPGAs is that they can be reconfigured to suit the computation. One such form of reconfiguration, called partial reconfiguration (PR), allows part of the chip to be altered. The smallest part that can be reconfigured is called a frame. To reconfigure a frame, a fixed number of configuration bits are input (typically from outside) to the frame. Thus PR involves (a) selecting a subset C Í S of k out of n frames to configure and (b) inputting the configuration bits for these k frames. The, recently proposed, MU-Decoder has made it possible to select the subset C quickly. This thesis involves mechanisms to input the configuration bits to the selected frames. Specifically, we propose a class of architectures that, for any subset C Í S (set of frames), constructs a path connecting only the k frames of C through which the configuration bits can be scanned in. We introduce a Basic Network that runs in Q (k log n) time, where k is the number of frames selected out of the total number n of available frames; we assume the number of configuration bits per frame is constant. The Basic Network does not exploit any locality or other structure in the subset of frames selected. We show that for certain structures (such as frames that are relatively close to each other) the speed of reconfiguration can be improved. We introduce an addition to the Basic Network that suggests the fastest clock speed that can be employed for a given set of frames. This enhancement decreases configuration time to O(k log k) for certain cases. We then introduce a second enhancement, called shortcuts, that for certain cases reduces the time to an optimal O(k). All the proposed architectures require an optimal Q(n) number of gates. We implement our networks on the CAD tools and show that the theoretical predictions are a good reflection of the network¢s performance. Our work, although directed to FPGAs, may also apply to other applications; for example hardware testing and novel memory accesses

High quality framegrabber for an IR imaging camera

Diseño electrónico de un digitalizador de video para una camara de infrarrojos con potenciales aplicaciones en el campo de "Gas Sensing

Novi koncept arhitekture sistema za obradu velike količine podataka u senzorskim uređajima sa ograničenim računarskim resursima

Državna infrastruktura, kako vojna, tako i civilna, u današnje vreme se ne može ni zamisliti bez upošljavanja računarskih, telekomunikacionih i senzorskih sistema koji su odgovorni za nesmetano funkcionisanje života u savremenom svetu. Posmatrajući senzorske sisteme, zajednička potreba za sve njih jeste određeni procesorski sistem, odnosno platforma za prihvat i obradu signala sa senzora. U današnje vreme, mnogi proizvođači pristupaju pomenutim zadacima kreiranjem procesorskih platformi za specifične namene kao što su platforme za obradu video signala namenjene elektro-optičkim sistemima, platforme za digitalnu obradu signala namenjene radarskim sistemima, zatim kombinacijom pomenutih tehnologija namenjenim automobilskoj industriji, itd. U okviru ove disertacije dat je predlog nove arhitekture univerzalne platforme za obradu signala u senzorskim uređajima prilagođene za primenu u modernim senzorsikm sistemima. Arhitektura predsavlja kombinaciju više savremenih procesorskih tehnologija za obradu signala kao štu su programabilna logika (eng. field programable gate array – FPGA), više-jezgarni mikroprocesori i jedinice za obradu grafičkih podataka (eng. graphics processin unit - GPU), kao i integracijom prateće mrežne infrastrukture, naročito vodeći računa o masi i dimenzijama, pokazano je i dokazano da razvijena plaforma ima sve odlike univerzalnosti i da se može koristiti u različitim senzorskim sistemima. Poseban akcenat je dat modularnosti i prilagodljivosti čitavog sistema turbulencijama na svetskom tržištu elektronskih komponenata izazvanih raznim krizama na globalnom nivou. Koncept univerzalne platforme je verifikovan kroz njenu praktičnu implementaciju. U disertaciji, tok razvoja je detaljno opisan u narednim poglavljima, prateći razvoj hardvera, implementaciju i testiranje složenih softverskih algoritama, planiranje mrežne infrastrukture, integraciju u moderan elektro-optički sistem i testiranje na sistemskom nivou. Pokazano je da implementirana platforma u potpunosti ispunjava pretpostavke nove predložene arhitekture. Dati su pravci daljeg razvoja i posebno naznačen doprinos disertacije u odnosu na tradicionalne načine rešavanja pomenutih zadataka

An FPGA-based processing pipeline for high-definition stereo video

This paper presents a real-time processing platform for high-definition stereo video. The system is capable toprocess stereo video streams at resolutions up to 1, 920 × 1, 080 at 30 frames per second (1080p30). In the hybridFPGA-GPU-CPU system, a high-density FPGA is used not only to perform the low-level image processing tasks suchas color interpolation and cross-image color correction, but also to carry out radial undistortion, image rectification,and disparity estimation. We show how the corresponding algorithms can be implemented very efficiently inprogrammable hardware, relieving the GPU from the burden of these tasks. Our FPGA implementation results arecompared with corresponding GPU implementations and with other implementations reported in the literature.ISSN:1687-5176ISSN:1687-528

Sincronização de múltiplas câmaras e controlo de iluminação sobre uma plataforma FPGA

O objetivo deste projeto foi o de realizar a sincronização de pelo menos quatro câmaras individuais, ajustando dinamicamente o frame rate de operação de cada câmara, tendo por base a família de sensores de imagem CMOS NanEye da empresa Awaiba, numa plataforma FPGA com interface USB3. Durante o projeto analisou-se, com a assistência de um supervisor da Awaiba, o sistema core de captura de imagem existente, baseado em VHDL. Foi estudado e compreendido o princípio do ajuste dinâmico do frame rate das câmaras. Tendo sido então desenvolvido o módulo de controlo da câmara, em VHDL, e um algoritmo de ajuste dinâmico do frame rate, sendo este implementado junto com a plataforma de processamento e interface da FPGA. Foi criado um módulo para efetuar a monitorização da frequência de operação de cada câmara, medindo o período de cada linha numa frame, tendo por base um sinal de relógio de valor conhecido. A frequência é ajustada variando o nível de tensão aplicado ao sensor com base no erro entre o período da linha medido e o período pretendido. Para garantir o funcionamento conjunto de múltiplas câmaras em modo síncrono foi implementada uma interface Master-Slave entre estas. Paralelamente ao módulo anteriormente descrito, implementou-se um sistema de controlo automático de iluminação com base na análise de regiões de interesse em cada frame captada por uma câmara NanEye. A intensidade de corrente aplicada às fontes de iluminação acopladas à câmara é controlada dinamicamente com base no nível de saturação dos pixéis analisados em cada frame. Foram desenvolvidas e implementadas variantes do algoritmo de controlo e o seu desempenho foi avaliado em laboratório. Os resultados obtidos na prática evidenciam que a solução implementada cumpre os requisitos de controlo e ajuste da frequência de operação de múltiplas câmaras. Mostrou ser um método de controlo capaz de manter um erro de sincronização médio de 3,77 μs mesmo na presença de variações de temperatura de aproximadamente 50 °C. Foi também demonstrado que o sistema de controlo de iluminação é capaz de proporcionar uma experiência de visualização adequada, alcançando erros menores que 3% e uma velocidade de ajuste máxima inferior a 1 s