FPGA in image processing supported by IOPT-Flow

Image processing is widely used in the most diverse industries. One of the tools widely used to perform image processing is the OpenCV library. Although the implementation of image processing algorithms can be made in software, it is also possible to implement image processing algorithms in hardware. In some cases, the execution time can be smaller than the execution time achieved in software. This work main goal is to evaluate the use of VHDL, DS-Pnets, and IOPT-Flow to develop image processing systems in hardware, in FPGA-based platforms. To enable it, a validation platform was developed. A set of image processing algorithms were specified, during this work, in VHDL and/or in DS-Pnets. These were validated using the IOPT-Flow validation tool and/or the Xilinx ISE Simulator. The automatic VHDL code generator from IOPT-Flow framework was used to translate DS-Pnet models into the implementation code. The FPGA-based implementations were compared with software implementations, supported by the OpenCV library. The created DS-Pnet models were added into a folder of the IOPT-Flow editor, to create an image processing library. It was possible to conclude that the DS-Pnets and their associated tools, IOPT-Flow tools, support the development of image processing systems. These tools, which simplify the development of image processing systems, are available online at http://gres.uninova.pt/iopt-flow/

Reconfigurable Vision Processing System for Player Tracking in Indoor Sports

Ibraheem OW, Irwansyah A, Hagemeyer J, Porrmann M, Rückert U. Reconfigurable Vision Processing System for Player Tracking in Indoor Sports. In: Conference on Design and Architectures for Signal and Image Processing (DASIP 2017). Piscataway, NJ: IEEE; 2017: 1-6.In recent years, there has been an increasing growth of using vision-based systems for tracking the players in team sports to evaluate and enhance their performance. Vision-based player tracking has high computational demands since it requires processing of a huge amount of video data based on the utilization of multiple cameras with high resolution and high frame rates. In this paper, we present a reconfigurable system to track the players in indoor sports automatically without user interaction. The proposed system can process live video data streams from multiple cameras as well as offline data from recorded video files. FPGA technology is used to accelerate this player tracking system by implementing the video acquisition, video preprocessing, player segmentation, and team identification & player detection modules in hardware, realizing a real-time system. The teams are identified and the players' positions are detected based on the colors of their jerseys. The detection results are sent from the FPGA to the host-PC where the players are tracked. Our results show that the achieved average player detection rate is up to 95.5%. The proposed system can process live video data using two GigE Vision cameras with a resolution of 1392×1040 pixels and 30 fps for each camera. A speed-up of 20 is achieved compared to an OpenCV-based software implementation on a host-PC equipped with a 2.93 GHz Intel i7 CPU

Reconfigurable Vision Processing for Player Tracking in Indoor Sports

Ibraheem OW. Reconfigurable Vision Processing for Player Tracking in Indoor Sports. Bielefeld: Universität Bielefeld; 2018.Over the past decade, there has been an increasing growth of using vision-based systems for tracking players in sports. The tracking results are used to evaluate and enhance the performance of the players as well as to provide detailed information (e.g., on the players and team performance) to viewers. Player tracking using vision systems is a very challenging task due to the nature of sports games, which includes severe and frequent interactions (e.g., occlusions) between the players. Additionally, these vision systems have high computational demands since they require processing of a huge amount of video data based on the utilization of multiple cameras with high resolution and high frame rate. As a result, most of the existing systems based on general-purpose computers are not able to perform online real-time player tracking, but track the players offline using pre-recorded video files, limiting, e.g., direct feedback on the player performance during the game. In this thesis, a reconfigurable vision-based system for automatically tracking the players in indoor sports is presented. The proposed system targets player tracking for basketball and handball games. It processes the incoming video streams from GigE Vision cameras, achieving online real-time player tracking. The teams are identified and the players are detected based on the colors of their jerseys, using background subtraction, color thresholding, and graph clustering techniques. Moreover, the trackingby-detection approach is used to realize player tracking. FPGA technology is used to handle the compute-intensive vision processing tasks by implementing the video acquisition, video preprocessing, player segmentation, and team identification & player detection in hardware, while the less compute-intensive player tracking is performed on the CPU of a host-PC. Player detection and tracking are evaluated using basketball and handball datasets. The results of this work show that the maximum achieved frame rate for the FPGA implementation is 96.7 fps using a Xilinx Virtex-4 FPGA and 136.4 fps using a Virtex-7 device. The player tracking requires an average processing time of 2.53 ms per frame in a host-PC equipped with a 2.93 GHz Intel i7-870 CPU. As a result, the proposed reconfigurable system supports a maximum frame rate of 77.6 fps using two GigE Vision cameras with a resolution of 1392x1040 pixels each. Using the FPGA implementation, a speedup by a factor of 15.5 is achieved compared to an OpenCV-based software implementation in a host-PC. Additionally, the results show a high accuracy for player tracking. In particular, the achieved average precision and recall for player detection are up to 84.02% and 96.6%, respectively. For player tracking, the achieved average precision and recall are up to 94.85% and 94.72%, respectively. Furthermore, the proposed reconfigurable system achieves a 2.4 times higher performance per Watt than a software-based implementation (without FPGA support) for player tracking in a host-PC.Acknowledgments: I (Omar W. Ibraheem) would like to thank the German Academic Exchange Service (DAAD), the Congnitronics and Sensor Systems research group, and the Cluster of Excellence Cognitive Interaction Technology ‘CITEC’ (EXC 277) (Bielefeld University) not only for funding the work in this thesis, but also for all the help and support they gave to successfully finish my thesis