7 research outputs found
Gaussian filtering for FPGA based image processing with High-Level Synthesis tools
With the gradual improvement and uprising interest from the industry to High-Level
Synthesis tools, like Vivado HLS form Xilinx, Field Programmable Gate Arrays are becoming an
attractive option for accelerator architecture in image processing domain. However, an efficient
high-level design still requires knowledge of hardware specifics. A great amount of image
processing operations falls into a group of convolution-based operators - operators which result
depends only on a particular pixel and its neighborhood and obtained by performing a
convolution between a kernel and a part of an image. This paper investigates the impact of
factors, such as kernel size, target frequency, convolution implementation specifics, floating-point
vs. fixed-point filter kernel, on resulting register-transfer level design of convolution-based
operators and FPGA resources utilization. The Gaussian filter was analyzed as an example of a
convolution-based operator. It is shown experimentally that floating-point operators require a
noticeably larger amount of resources, rather fixed-point once. Resulting clock frequency
independence from kernel size is demonstrated as well as the number of used flip-flops grows
with the increasing target clock frequency is investigated in this work.Although using of HLS can simplify and accelerates the development of FPGA-based
applications, it is still requires careful design space exploration. It is crucial to remember
that existing HLS tools do not provide full abstraction and the result of the development is not
software but hardware. The efficiency of resulting FPGA solution and its resources utilization
depends heavily on many factors which have to be taken into account on the programming
stage. Floating-point operations implemented on FPGA are usually inefficient and consume a
tremendous amount of resources, therefore should be avoided. Kernel size doesnât affect clock
frequency and just increases the number of resources required for storing bigger kernel and
temporary image areas. A number of used flip-flops grows rapidly with the increasing target
clock frequency and generally bigger for bigger kernels. Therefore a trade-off between target
speed and resources utilization should be considered by a developer. A benefit achieved with
the use of vendor-provided libraries has to be noted. They provide convenient abstractions
usually at no additional resources cost. Thus, for instance, window and line buffers from Vivado
Video Library might be used as an alternative to hand-programmed data structures. Results
obtained in this work might be extended to any convolution-based image processing operator
implemented on FPGA with HLS
Efficient Smart CMOS Camera Based on FPGAs Oriented to Embedded Image Processing
This article describes an image processing system based on an intelligent ad-hoc camera, whose two principle elements are a high speed 1.2 megapixel Complementary Metal Oxide Semiconductor (CMOS) sensor and a Field Programmable Gate Array (FPGA). The latter is used to control the various sensor parameter configurations and, where desired, to receive and process the images captured by the CMOS sensor. The flexibility and versatility offered by the new FPGA families makes it possible to incorporate microprocessors into these reconfigurable devices, and these are normally used for highly sequential tasks unsuitable for parallelization in hardware. For the present study, we used a Xilinx XC4VFX12 FPGA, which contains an internal Power PC (PPC) microprocessor. In turn, this contains a standalone system which manages the FPGA image processing hardware and endows the system with multiple software options for processing the images captured by the CMOS sensor. The system also incorporates an Ethernet channel for sending processed and unprocessed images from the FPGA to a remote node. Consequently, it is possible to visualize and configure system operation and captured and/or processed images remotely
Computer vision and optimization methods applied to the measurements of in-plane deformations
fi=vertaisarvioitu|en=peerReviewed
Détection automatique de chutes de personnes basée sur des descripteurs spatio-temporels (définition de la méthode, évaluation des performances et implantation temps-réel)
Nous proposons une méthode supervisée de détection de chutes de personnes en temps réel, robusteaux changements de point de vue et d environnement. La premiÚre partie consiste à rendredisponible en ligne une base de vidéos DSFD enregistrées dans quatre lieux différents et qui comporteun grand nombre d annotations manuelles propices aux comparaisons de méthodes. Nousavons aussi défini une métrique d évaluation qui permet d évaluer la méthode en s adaptant à la naturedu flux vidéo et la durée d une chute, et en tenant compte des contraintes temps réel. Dans unsecond temps, nous avons procédé à la construction et l évaluation des descripteurs spatio-temporelsSTHF, calculés à partir des attributs géométriques de la forme en mouvement dans la scÚne ainsique leurs transformations, pour définir le descripteur optimisé de chute aprÚs une méthode de sélectiond attributs. La robustesse aux changements d environnement a été évaluée en utilisant les SVMet le Boosting. On parvient à améliorer les performances par la mise à jour de l apprentissage parl intégration des vidéos sans chutes enregistrées dans l environnement définitif. Enfin, nous avonsréalisé, une implantation de ce détecteur sur un systÚme embarqué assimilable à une caméra intelligentebasée sur un composant SoC de type Zynq. Une démarche de type Adéquation AlgorithmeArchitecture a permis d obtenir un bon compromis performance de classification/temps de traitementWe propose a supervised approach to detect falls in home environment adapted to location andpoint of view changes. First, we maid publicly available a realistic dataset, acquired in four differentlocations, containing a large number of manual annotation suitable for methods comparison. We alsodefined a new metric, adapted to real-time tasks, allowing to evaluate fall detection performance ina continuous video stream. Then, we build the initial spatio-temporal descriptor named STHF usingseveral combinations of transformations of geometrical features and an automatically optimised setof spatio-temporal descriptors thanks to an automatic feature selection step. We propose a realisticand pragmatic protocol which enables performance to be improved by updating the training in thecurrent location with normal activities records. Finally, we implemented the fall detection in Zynqbasedhardware platform similar to smart camera. An Algorithm-Architecture Adequacy step allowsa good trade-off between performance of classification and processing timeDIJON-BU Doc.électronique (212319901) / SudocSudocFranceF
Suivi visuel d'objets dans un réseau de caméras intelligentes embarquées
Multi-object tracking constitutes a major step in several computer vision applications. The requirements of these applications in terms of performance, processing time, energy consumption and the ease of deployment of a visual tracking system, make the use of low power embedded platforms essential. In this thesis, we designed a multi-object tracking system that achieves real time processing on a low cost and a low power embedded smart camera. The tracking pipeline was extended to work in a network of cameras with nonoverlapping field of views. The tracking pipeline is composed of a detection module based on a background subtraction method and on a tracker using the probabilistic Gaussian Mixture Probability Hypothesis Density (GMPHD) filter. The background subtraction, we developed, is a combination of the segmentation resulted from the Zipfian Sigma-Delta method with the gradient of the input image. This combination allows reliable detection with low computing complexity. The output of the background subtraction is processed using a connected components analysis algorithm to extract the features of moving objects. The features are used as input to an improved version of GMPHD filter. Indeed, the original GMPHD do not manage occlusion problems. We integrated two new modules in GMPHD filter to handle occlusions between objects. If there are no occlusions, the motion feature of objects is used for tracking. When an occlusion is detected, the appearance features of the objects are saved to be used for re-identification at the end of the occlusion. The proposed tracking pipeline was optimized and implemented on an embedded smart camera composed of the Raspberry Pi version 1 board and the camera module RaspiCam. The results show that besides the low complexity of the pipeline, the tracking quality of our method is close to the stat of the art methods. A frame rate of 15 â 30 was achieved on the smart camera depending on the image resolution. In the second part of the thesis, we designed a distributed approach for multi-object tracking in a network of non-overlapping cameras. The approach was developed based on the fact that each camera in the network runs a GMPHD filter as a tracker. Our approach is based on a probabilistic formulation that models the correspondences between objects as an appearance probability and space-time probability. The appearance of an object is represented by a vector of m dimension, which can be considered as a histogram. The space-time features are represented by the transition time between two input-output regions in the network and the transition probability from a region to another. Transition time is modeled as a Gaussian distribution with known mean and covariance. The distributed aspect of the proposed approach allows a tracking over the network with few communications between the cameras. Several simulations were performed to validate the approach. The obtained results are promising for the use of this approach in a real network of smart cameras.Le suivi dâobjets est de plus en plus utilisĂ© dans les applications de vision par ordinateur. Compte tenu des exigences des applications en termes de performance, du temps de traitement, de la consommation dâĂ©nergie et de la facilitĂ© du dĂ©ploiement des systĂšmes de suivi, lâutilisation des architectures embarquĂ©es de calcul devient primordiale. Dans cette thĂšse, nous avons conçu un systĂšme de suivi dâobjets pouvant fonctionner en temps rĂ©el sur une camĂ©ra intelligente de faible coĂ»t et de faible consommation Ă©quipĂ©e dâun processeur embarquĂ© ayant une architecture lĂ©gĂšre en ressources de calcul. Le systĂšme a Ă©tĂ© Ă©tendu pour le suivi dâobjets dans un rĂ©seau de camĂ©ras avec des champs de vision non-recouvrant. La chaĂźne algorithmique est composĂ©e dâun Ă©tage de dĂ©tection basĂ© sur la soustraction de fond et dâun Ă©tage de suivi utilisant un algorithme probabiliste Gaussian Mixture Probability Hypothesis Density (GMPHD). La mĂ©thode de soustraction de fond que nous avons proposĂ©e combine le rĂ©sultat fournie par la mĂ©thode Zipfian Sigma-Delta avec lâinformation du gradient de lâimage dâentrĂ©e dans le but dâassurer une bonne dĂ©tection avec une faible complexitĂ©. Le rĂ©sultat de soustraction est traitĂ© par un algorithme dâanalyse des composantes connectĂ©es afin dâextraire les caractĂ©ristiques des objets en mouvement. Les caractĂ©ristiques constituent les observations dâune version amĂ©liorĂ©e du filtre GMPHD. En effet, le filtre GMPHD original ne traite pas les occultations se produisant entre les objets. Nous avons donc intĂ©grĂ© deux modules dans le filtre GMPHD pour la gestion des occultations. Quand aucune occultation nâest dĂ©tectĂ©e, les caractĂ©ristiques de mouvement des objets sont utilisĂ©es pour le suivi. Dans le cas dâune occultation, les caractĂ©ristiques dâapparence des objets, reprĂ©sentĂ©es par des histogrammes en niveau de gris sont sauvegardĂ©es et utilisĂ©es pour la rĂ©-identification Ă la fin de lâoccultation. Par la suite, la chaĂźne de suivi dĂ©veloppĂ©e a Ă©tĂ© optimisĂ©e et implĂ©mentĂ©e sur une camĂ©ra intelligente embarquĂ©e composĂ©e de la carte Raspberry Pi version 1 et du module camĂ©ra RaspiCam. Les rĂ©sultats obtenus montrent une qualitĂ© de suivi proche des mĂ©thodes de lâĂ©tat de lâart et une cadence dâimages de 15 â 30 fps sur la camĂ©ra intelligente selon la rĂ©solution des images. Dans la deuxiĂšme partie de la thĂšse, nous avons conçu un systĂšme distribuĂ© de suivi multi-objet pour un rĂ©seau de camĂ©ras avec des champs non recouvrants. Le systĂšme prend en considĂ©ration que chaque camĂ©ra exĂ©cute un filtre GMPHD. Le systĂšme est basĂ© sur une approche probabiliste qui modĂ©lise la correspondance entre les objets par une probabilitĂ© dâapparence et une probabilitĂ© spatio-temporelle. Lâapparence dâun objet est reprĂ©sentĂ©e par un vecteur de m Ă©lĂ©ments qui peut ĂȘtre considĂ©rĂ© comme un histogramme. La caractĂ©ristique spatio-temporelle est reprĂ©sentĂ©e par le temps de transition des objets et la probabilitĂ© de transition dâun objet dâune rĂ©gion dâentrĂ©e-sortie Ă une autre. Le temps de transition est modĂ©lisĂ© par une loi normale dont la moyenne et la variance sont supposĂ©es ĂȘtre connues. Lâaspect distribuĂ© de lâapproche proposĂ©e assure un suivi avec peu de communication entre les noeuds du rĂ©seau. Lâapproche a Ă©tĂ© testĂ©e en simulation et sa complexitĂ© a Ă©tĂ© analysĂ©e. Les rĂ©sultats obtenus sont prometteurs pour le fonctionnement de lâapproche dans un rĂ©seau de camĂ©ras intelligentes rĂ©el