Harris Corner Detector (HCD) algorithm is widely used in many applications of image processing. Its performance with noisy images exceeds many other methods, in terms of accuracy and stability. Various methods are used to compare images and detect moving objects such as block matching but these methods are slow and have less accuracy. Moreover, the implementation of HCD has been proven to be computationally intensive, therefore, real-time streaming is difficult to achieve with sequential software implementation. This report presents the hardware implementation of HCD using Field-Programmable Gate Array (FPGA). The targeted board for the design is DE2-115 FPGA development board with an Altera Cyclone IV device. The architecture was tested using a SystemVerilog test-bench, enveloped by a MATLAB test-bench. The accuracy of the results obtained was tested visually and compared with the results of the same algorithm implemented in MATLAB. A maximum operational frequency of 170 MHz was achieved. The system uses 40% of the board’s logic elements. Resource utilization and timing performance are considerably balanced compared to recent works

Al-Shatari, Mohammed Omar Awadh

Universiti Teknologi Malaysia Institutional Repository

i  IMPLEMENTATION OF HARRIS CORNER DETECTOR ON FPGA         MOHAMMED OMAR AWADH AL-SHATARI         A project report submitted in partial fulfilment of the requirements for the award of the degree of Master of Engineering (Electronics & Telecommunication)       Faculty of Electrical Engineering Universiti Teknologi Malaysia        JUNE 2016 iii                An appreciation to all who made this possible  especially my parents, wife, supervisor and friends.           iv  ACKNOWLEDGEMENT First, I wish to thank Allah for the strength He has given me to complete this project. It was a valuable experience and great opportunity for me to work on such project. I would like to thank the people who offered me their help and advice during my project time. I would like to express my deepest gratitude to my supervisor, Dr. Nasir Shaikh Husin for giving me the opportunity to work with him and for his guidance, motivations and advice. Next, I wish to convey my sincere appreciation to my family for their love, blessing and support. Last but not least, I would like to thank all my friends who were involved directly or indirectly for their knowledge, support, kindness and the time we spent together.              v  ABSTRACT Harris Corner Detector (HCD) algorithm is widely used in many applications of image processing.  Its performance with noisy images exceeds many other methods, in terms of accuracy and stability. Various methods are used to compare images and detect moving objects such as block matching but these methods are slow and have less accuracy. Moreover, the implementation of HCD has been proven to be computationally intensive, therefore, real-time streaming is difficult to achieve with sequential software implementation. This report presents the hardware implementation of HCD using Field-Programmable Gate Array (FPGA). The targeted board for the design is DE2-115 FPGA development board with an Altera Cyclone IV device. The architecture was tested using a SystemVerilog test-bench, enveloped by a MATLAB test-bench. The accuracy of the results obtained was tested visually and compared with the results of the same algorithm implemented in MATLAB. A maximum operational frequency of 170 MHz was achieved. The system uses 40% of the board’s logic elements. Resource utilization and timing performance are considerably balanced compared to recent works.       vi  ABSTRAK Algoritma Harris Corner Detector (HCD) digunakan secara meluas untuk aplikasi pemprosesan imej. Prestasinya untuk imej hingar melebihi kebanyakan kaedah yang lain, dari segi ketepatan dan kestabilan. Pelbagai kaedah digunakan untuk membandingkan imej dan mengesan pergerakan objek seperti  pemadanan blok tetapi  kaedah-kaedah tersebut adalah perlahan dan kurang tepat. Selain itu, pelaksanaan HCD terbukti memerlukan proses pengkomputeran yang intensif, maka strim dalam masa nyata adalah sukar untuk dicapai dengan pelaksanaan perisian secara berjujukan. Laporan ini membentangkan pelaksanaan perkakasan HCD menggunakan Field-Programmable Gate Array (FPGA). Papan litar yang disasarkan untuk reka bentuk HCD adalah papan pembangunan FPGA DE2-115 dengan peranti Cyclone IV Altera. Senibina ini telah diuji menggunakan penanda aras SystemVerilog, dengan dikelubungi oleh penanda aras MATLAB. Kejituan dalam keputusan yang diperolehi telah diuji secara visual dan dibandingkan dengan keputusan algoritma yang sama yang dilaksanakan dalam MATLAB. Frekuensi pengendalian maksimum 170 MHz telah dicapai. Sistem ini menggunakan 40% unsur logic papan . Penggunaan sumber dan prestasi pemasaan adalah lebih jauh seimbang berbanding pelaksanaan yang terkini.      vii  TABLE OF CONTENTS CHAPTER TITLE PAGE    DECLARATION  ii  DEDICATION iii  ACKNOWLEDGEMENT iv  ABSTRACT v  ABSTRAK vi  TABLE OF CONTENTS vii  LIST OF TABLES x  LIST OF FIGURES xi  LIST OF ABBREVIATIONS xiv  LIST OF APPENDICES xv    1 INTRODUCTION 1  1.1 Background 1  1.2 Motivation 2  1.3 Objectives 2  1.4 Scopes 3  1.5 Report Outline 3    2 LITERATURE REVIEW 4  2.1 Introduction 4  2.2 Corner Detection 4  2.3 Qualitative Description of Corner Detection 4  2.4 Mathematical Description of Corner Detection 5  2.5 Harris Corner Detector 11  2.6 Related Work 17 viii  3 METHODOLOGY 19  3.1 Introduction   3.2 Harris Corner Detector 20      3.2.1 Spatial Derivatives 21          3.2.1.1 Algorithm of Spatial Derivatives 22          3.2.1.2 Dataflow Graph of Spatial Derivatives 23          3.2.1.3 Functional Block Diagram of Spatial  24                      Derivatives       3.2.2 Gaussian Filtering 25          3.2.2.1 Algorithm of Gaussian Filter 26          3.2.2.2 Dataflow Graph of Gaussian Kernel 27          3.2.2.3 Functional Block Diagram of Gaussian Filter 28          3.2.2.4 Pixel Buffer 30          3.2.2.5 Block RAM (BRAM) of Gaussian Filter 30      3.2.3 Harris Response 31          3.2.3.1 Algorithm of Harris Response 31          3.2.3.2 Dataflow Graph of Harris Response 32          3.2.3.3 Functional Block Diagram of Harris Response 33      3.2.4 Non-Maximum Suppression 33          3.2.4.1 Dataflow Graph of Module Max7 34          3.2.4.2 Dataflow Graph of Module Max6 35          3.2.4.3 Non-Maximum Suppression Top-level Module 36          3.2.4.4 Block RAM (BRAM) of Non-Maximum  37                      Suppression     4 RESULT & DISCUSSION 38  4.1 Introduction 38  4.2 Spatial Derivatives 38  4.3 Gaussian Filtering 42  4.4 Harris Response 44  4.5 Non-Maximum Suppression 45  4.6 Performance Analysis 50    ix  5 CONCLUSION AND RECOMMENDATION 52  5.1 Conclusion 52  5.2 Recommendations and Future Improvements 53    REFERENCES 54 Appendices A-B 56-95               x  LIST OF TABLES TABLE NO. TITLE PAGE  2.1  Previous work related to Harris corner detector on FPGA  18 4.1 Performance analysis 51                       xi  LIST OF FIGURES FIGURE NO. TITLE PAGE  2.1  Direction of intensity change  5 2.2 Sum of squared difference when region shifted horizontally 6 2.3 Sum of squared difference when region shifted vertically 6 2.4 Sum of squared difference when region shifted in all directions 7 2.5 Sum of squared difference when region shifted in all directions and corner exists 8 2.6 Harris corner classification 10 2.7 Example of corner response 11 2.8 Kernel proposed in [6] for the estimation of image derivatives 12 2.9 3x3 Prewitt gradient filter mask (a) X-direction (b) Y-direction 13 2.10 1-D Gaussian distribution with μ=0 and =1 14 2.11 2-D Gaussian distribution with μ=(0,0) and =1 14 2.12 Discrete Approximation of Gaussian kernel 15 2.13 1-D Gaussian kernel 15 2.14 Corner response (a) Unsuppressed (b) Suppressed 16 2.15 Center pixel example (a) Discarded (b) Unchanged 17 3.1 Working flow of the system 20 3.2 Functional block diagram of Harris system 21 3.3 Derivative kernel (Prewitt operator) (a) X-direction (b) Y-direction 22 xii  3.4 Derivative Kernel (a) X-direction (b) Y-direction 23 3.5 Dataflow graph of the spatial derivatives 23 3.6 Derivatives top-level module 24 3.7 Functional block diagram of the derivatives module 25 3.8 2-D 5x5 Separable Gaussian filter 26 3.9 1-D Gaussian kernel (a) 1x5 (b) 5x1 27 3.10 Dataflow graph of Gaussian kernel 27 3.11 Top-level module of Gaussian filter 28 3.12 Vertical Gaussian module 29 3.13 Horizontal Gaussian module 29 3.14 Functional block diagram of the Pixel Buffer module 30 3.15 Functional block diagram of BRAM module 31 3.16 Dataflow graph of the Harris response 32 3.17 Functional block diagram of Harris Response module 33 3.18 Dataflow graph of Max7 module in NMS 34 3.19 Dataflow graph of Max6 module in NMS 35 3.20 Top-level module of Non-Maximum Suppression 36 3.21 Functional block diagram of NMS BRAM module 37 4.1 Hardware result of first derivatives (a) Ix (b) Iy 39 4.2 MATLAB result of first derivatives (a) Ix (b) Iy 39 4.3 Hardware result of spatial derivatives (a) Ix2 (b) Iy2 (c) IxIy 40 4.4 MATLAB result of spatial derivatives (a) Ix2 (b) Iy2 (c) IxIy 41 4.5 Hardware result of Gaussian smoothing 42 4.6 MATLAB result of Gaussian smoothing 43 4.7 Hardware result of Harris response 44 4.8 MATLAB result of Harris response 45 4.9 Hardware result of non-maximum suppression 46 4.10 MATLAB result of non-maximum suppression 46 4.11 Hardware result of Harris corner detector 47 4.12 MATLAB result of Harris corner detector 47 4.13 Corners of “Lena” image resulted from Hardware 48 xiii  4.14 Corners of "Lena" image resulted from MATLAB 48 4.15 Corners of "House" image resulted from Hardware 49 4.16 Corners of "House" image resulted from MATLAB 49 4.17 Simulation report of Harris corner detector system 50   xiv  LIST OF ABBREVIATIONS HCD  - Harris Corner Detector NMS  - Non-Maximum Suppression ROI  - Region of Interest DFG  - Dataflow Graph HDL  - Hardware Description Language RTL  - Register Transfer Level RAM  - Random Access Memory FPGA  - Field Programmable Gate Array ANMS  - Adaptive Non-Maximum Suppression   xv  LIST OF APPENDICES APPENDIX TITLE PAGE  A  SystemVerilog Code  56 B MATLAB Code 91                 1   CHAPTER 1 INTRODUCTION 1.1 Background The main overall goal of computer vision is to model and imitate the visual system of the human through computer software and hardware at different levels. The replication of human visual system on computational platform has proven to be problematic and challenging. Computer vision is the field of reconstructing the 3D world from 2D images and computer graphics is pursuing the opposite direction by designing 2D images to simulate the 3D world. However, 3D details are lost during image transformation from 3D world to 2D images which lead to difficulties in analysis of image processing. Image processing is in the middle connecting the computer vision and computer graphics [1]. High-level computer vision tasks and systems like motion estimation rely on the extraction of low-level processes such as image features, interest point and corner detection which represent a small portion of image pixels [2, 3]. Interest points retain similar characteristics even after image transformation, which has to be robustly detected [4]. Image features are used in many applications such as object recognition. The performance of such applications relies on robustness and efficiency on the low-level processes. Corners are intuitively distinguishable features; they represent sudden change of intensity levels in more than one direction on the image [5]. A corner detector should satisfy the performance requirements of real-time video streaming applications. 2  Harris corner detector algorithm in [6] is widely used in many applications of image processing. Its performance with noisy images exceeds many other methods, in terms of accuracy and stability, such as SUSAN and Minimum Intensity Change (MIC) [5]. Despite their inherent differences, the computation is similar for most interest point detectors where window-based image processing operators are applied locally on every image position. This makes the extraction process computationally intensive. High-speed corner detection is in high demand for computer vision systems in applications such as motion detection and object recognition [7]. This report presents the hardware implementation of Harris corner detector algorithm on Field Programmable Gate Array (FPGA). 1.2 Motivation There are some methods used for corner detection to find corners in images. However, the performance of these methods is low especially when implementing noisy images. The implementation of Harris corner detector has been proven to be computationally intensive. Therefore, real-time streaming is difficult to achieve with sequential software implementation. This work proposes a Harris corner detector hardware implementation with high throughput and accuracy to find the corners in an input image using FPGA. 1.3 Objectives The aim of this work is to present the design and implementation of Harris corner detector on FPGA. To achieve this aim, some objectives must be accomplished: 1. To design hardware blocks of the system. 2. To apply pipelining to increase throughput. 3. To optimize for resource utilization, throughput and latency. 4. To verify and visually compare the result with MATLAB. 3  1.4 Scopes The scope of this research is limited to the design and implementation of Harris corner detector on FPGA. The targeted FPGA board is Terasic DE2-115 Development Kit and the input image is limited to 256x256 gray scale image. Software used in this project are MATLAB, Quartus II and ModelSim. 1.5 Report Outline Chapter 1 introduces the background of Harris corner detector. Chapter 2 explains the literature review and the related work of Harris corner detector which has been done prior to this work. Chapter 3 describes the methodology and design of the proposed hardware architecture of Harris corner detector. Each stage of the hardware implementation is explained. Chapter 4 discusses the result of the hardware implementation compared to MATLAB and the analysis of the result. Chapter 5 summarizes the project report and proposes recommendations for future work.      54  REFERENCES 1. Pinoli, J.-C. and J.-C. Pinoli, Gray-Tone Images, in Mathematical Foundations of Image Processing and Analysis 1. 2014, John Wiley & Sons, Inc. p. 1-11. 2. Schmid, C., R. Mohr, and C. Bauckhage, Evaluation of interest point detectors. International Journal of computer vision, 2000. 37(2): p. 151-172. 3. Szeliski, R., Computer vision: algorithms and applications. 2010: Springer Science & Business Media. 4. Hernandez-Lopez, A., C. Torres-Huitzil, and J.J. Garcia-Hernandez, FPGA-based flexible hardware architecture for image interest point detection. International Journal of Advanced Robotic Systems, 2015. 12. 5. Trajković, M. and M. Hedley, Fast corner detection. Image and vision computing, 1998. 16(2): p. 75-87. 6. Harris, C. and M. Stephens. A combined corner and edge detector. in Alvey vision conference. 1988. Citeseer. 7. Mainali, P., et al. Lococo: Low complexity corner detector. in Acoustics Speech and Signal Processing (ICASSP), 2010 IEEE International Conference on. 2010. IEEE. 8. Loundagin, J., Optimizing Harris Corner Detection on GPGPUs Using CUDA. 2015. 9. Gonzalez, R. and R. Woods, Digital image processing: Pearson prentice hall. Upper Saddle River, NJ, 2008. 10. Tomasi, C. and T. Kanade, Detection and tracking of point features. 1991: School of Computer Science, Carnegie Mellon Univ. Pittsburgh. 11. Ruzon, M.A. and C. Tomasi, Edge, junction, and corner detection using color distributions. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 2001. 23(11): p. 1281-1295. 12. Shi, J. and C. Tomasi. Good features to track. in Computer Vision and Pattern Recognition, 1994. Proceedings CVPR'94., 1994 IEEE Computer Society Conference on. 1994. IEEE. 13. Derpanis, K.G., The harris corner detector. York University, 2004. 14. Dey, N., et al., A Comparative Study between Moravec and Harris Corner Detection of Noisy Images Using Adaptive Wavelet Thresholding Technique. arXiv preprint arXiv:1209.1558, 2012. 15. Ryu, J.-B., H.-H. Park, and J. Park, Corner classification using Harris algorithm. Electronics letters, 2011. 47(9): p. 536-538. 16. Moravec, H.P., Obstacle avoidance and navigation in the real world by a seeing robot rover. 1980, DTIC Document. 17. Chao, T.L. and K.H. Wong. An efficient FPGA implementation of the Harris corner feature detector. in Machine Vision Applications (MVA), 2015 14th IAPR International Conference on. 2015. IEEE. 55  18. Possa, P.R., et al., A multi-resolution fpga-based architecture for real-time edge and corner detection. Computers, IEEE Transactions on, 2014. 63(10): p. 2376-2388. 19. Amaricai, A., C.-E. Gavriliu, and O. Boncalo. An FPGA sliding window-based architecture harris corner detector. in Field Programmable Logic and Applications (FPL), 2014 24th International Conference on. 2014. IEEE. 20. Hsiao, P.-Y., C.-L. Lu, and L.-C. Fu, Multilayered image processing for multiscale Harris corner detection in digital realization. Industrial Electronics, IEEE Transactions on, 2010. 57(5): p. 1799-1805. 21. Orabi, H., N. Shaikh-Husin, and U.U. Sheikh. Low cost pipelined FPGA architecture of Harris Corner Detector for real-time applications. in Digital Information Management (ICDIM), 2015 Tenth International Conference on. 2015. 22. Yates, R., Fixed-point arithmetic: An introduction. Digital Signal Labs, 2009. 81(83): p. 198. 23. Bailey, D.G., Design for embedded image processing on FPGAs. 2011: John Wiley & Sons.           

Implementation of harris corner detector on fpga

http://eprints.utm.my/id/eprint/81604/1/MohammedOmarAwadhMFKE2016.pdf

Implementation of harris corner detector on fpga

Abstract

Similar works

Full text

Available Versions

Universiti Teknologi Malaysia Institutional Repository