النظام المقترح مفيد للغاية للمكفوفين، حيث يعطي النظام فكرة عن الحاضرين في محيط المكفوفين، لذا فإن النظام المساعدالمقترح يعتمد على الرازبيري pi3 لإبلاغ المكفوفين عن عدد الأشخاص الموجودين حولهم. يركز هذا العمل على اكتشاف مميزات الوجه بناءً على خوارزمية Haar المتتالية وOpenCV باستخدام لغة البرمجة بايثون. تم الحصول على نتائج مرضية جدا عند التعامل مع حالات مختلفة بدقة عالية ووقت معالجة قصير، وبالتالي تقلل OpenCV من تأخير الحصول على النتائج، وتزيد خوارزمية سلسلة Haar من الدقة.The proposed system is very helpful for blind people, in term of taking an idea about the attendance person/s in their environment simultaneously, so an assistive system based on raspberry pi3 is developed to inform the blind how many persons are around. This work focuses on face detection based on Haar cascade algorithm and OpenCV using python. Satisfied results with different cases are obtained with high accuracy and short processing time, hence OpenCV reduce the delay for getting output, and Haar cascade algorithm increases the accuracy, the system is portable and very light to carry

F. Alkhalid, Farah

Kadhim Oleiwi  , Bashra

Muhsin, M. Abdul

English

Journals of University of Babylon

Journal of University of Babylon for Engineering Sciences, Vol. (28), No. (2): 2020. Journal of University of Babylon for Engineering Sciences by University of Babylon is licensed under a Creative Commons Attribution 4.0 International License. 25 Real Time Blind People Assistive System Based on OpenCV Farah F. Alkhalid*           Bashra Kadhim Oleiwi*          M. Abdul Muhsin* *Control and Systems Engineering, University of Technology-Iraq 10352@uptechnology.edu.iq      60010@uotechnology.edu.iq    60100@uotechnology.edu.iq    Abstract The proposed system is very helpful for blind people, in term of taking an idea about the attendance person/s in their environment simultaneously, so an assistive system based on raspberry pi3 is developed to inform the blind how many persons are around. This work focuses on face detection based on Haar cascade algorithm and OpenCV using python. Satisfied results with different cases are obtained with high accuracy and short processing time, hence OpenCV reduce the delay for getting output, and Haar cascade algorithm increases the accuracy, the system is portable and very light to carry. Keywords: Raspberry pi3, OpenCV, Haar cascade algorithm. I. Introduction Face detection technology represents the computer ability of recognizing faces of the people in digital images. The applications of face detection using feature detection algorithms within large scales images that may include different objects. Accordingly, machine learning algorithms are used for human face detection in face detection applications within images of different sizes and different objects Many numerous objects might include large image several objects, landscapes, buildings, animals and some parts may belong to humans (e.g. body, shoulders legs, and clothes) [1]. Face detection/recognition technology is previously used for security application only, but recently there is an active extension to different applications such as retail, marketing, healthcare and so on. In the last decades, face detection and recognition received a great interest. This section will focus on the works regarding assistive technology to help blind and visually impaired persons. In [2] the authors proposed a text detection and reading algorithm within images for using by blind that walking in natural scenes of the city. The proposed algorithm succeeded with high rate of around 90% on the test set of images within small unread text typically. [3] presented textual areas extraction technique of images based on matched wavelet filters globally. A clustering method is devised to estimate matched wavelet filters globally based on a set of ground truth images. In [4] the researchers proposed a support vector machine to analyze the textual features of texts. The integration of CAMshift and support vector machine produced both efficient text detection and people to help them live independently. In [5] the author introduced the navigational systems using by blind and visually impaired for helping independent travel and navigation on large scale area. In [6] the author proposed an automatic algorithm for detection and recognition of the signs within city scenes based on effectively dealing with different degrees of lighting followed by a Gabor transform for obtaining local properties. In [7] the researchers presented a comparative study of wearable obstacle and avoidance systems to introduce the progress in assistive systems for blinds. The proposed system is very helpful for blinds, so assistive system based on raspberry pi3 to inform the blind how many persons are around, In [8] the authors proposed an assistive system for blind people to make object detection using Yolo (You only look once) which is important to tell the blind what are around him. In general, all systems based on deep learning need to train, and as known, the training needs long time and high processing, but using OpenCV with Haar cascade classifier do not need high processing nor time for training, where the detection can be done with seconds. The rest of the paper is organized as follows: Section II gives an overall system and its operation. Simulation results and discussion are presented in Section III. Finally, conclusion with future work scopes are presented in Section IV. II.System Overview The objective of this work is to help blind people to take an idea about how many persons are around in their place, as can be seen in figure (1).  Submission date:- 8/12/2019 Acceptance date:- 17/5/2020 Publication date:- 22/9/2020 Journal of University of Babylon for Engineering Sciences, Vol. (28), No. (2): 2020. 26           Figure (1): System overview. The main objective of this work is to build a low-cost system for helping visually impaired people and blind people. The camera has been connected through USB port within hardware for capturing the images. The image is then fed internally within the raspberry pi board to inform the blind and visually impaired people how many persons are around within the image. Using OpenCV library and Haar cascade classifier, the total number of people around will be identified and then the identified number will be announced by speech using earphone which is connected to audio port within hardware.  III. Face Detection Algorithm This work concentrates on face detection using Haar cascade algorithm and OpenCV using python. Basically, the face detection is sophisticated process, and the algorithms used for face detection are always beginning by searching for eyes. Eyes are creating what is called as a “valley region” which represent the easiest characteristics to be detected. When eyes detected, the algorithm can go to detect facial regions and find eyebrows, nose, mouth, iris and the nostrils.  A. OpenCV OpenCV represents processing library for video and image used in C++, C, Python, and Java. OpenCV library can be used for analysis of video and image, like facial detection and recognition, reading bar code, editing of photo, robotic vision, recognition of optical character, and a lot more. OpenCV uses three build in classifier for faces (Eigen Faces, Haar Cascade, Fisher Faces and Local Binary Patterns Histograms (LBPH))   which are open source coding, anybody can use by simply varying in code [9]. B. Haar Cascade Classifier Haar Cascade Classifier is a very popular classifier which is introduced by Paul Viola and Michael Jones in 2001 in, "Rapid Object Detection using a Boosted Cascade of Simple Features” [9]. This classifier represents an algorithm based on machine learning, begins with training a cascade function of a lot of images in term of positive and negative. Then it will be used to make objects detection in other images [10]. Haar is more accurate, but at the same time it is much slower than others. The OpenCV package has all information which will be required to apply Haar effectively. Basically, the need for an XML file with the right face information within it and it can be also created by the user if he has knows what he was doing or can just use what comes with OpenCV.  Initially, this algorithm needs set of positive images with faces and many negative images without faces to train the classifier. After that features extraction from them will be made. So, Haar features as illustrated in Figure(2) below. They are similar to the convolutional kernel. Each property represents a one value that got by subtracting summation of pixels in white rectangle from black one.  Journal of University of Babylon for Engineering Sciences, Vol. (28), No. (2): 2020. 27  Figure (2): Haar Features [10]. The calculation of  Haar features effectively, the integral image calculation should consider as in [10]. Integral image at location ii (x, y) contains the sum of pixel values above and left of (x, y) (inclusive) as illustrated in Figure (3).  Integral image for an original gray-scale image i (x, y) can be calculated as follows: 𝑖𝑖(𝑥, 𝑦) = ∑ 𝑖 (𝑥′, 𝑦′)𝑥′≤𝑥,𝑦′≤𝑦𝑖   Figure (3): Integral image computation [10]. Using the expressions below, an integral image can be calculated cumulatively and effectively over a whole input image: s(x, y) =s(x, y−1) +i(x, y) ii(x, y) =ii(x−1, y) +s(x, y) where s(x, y) is  the  cumulative  sum  of  rows, s(x,−1) = 0,and ii(−1, y) = 0. The face detection algorithm will be explained by the following steps and flowchart in Figure (4): Step1: Import all important libraries Step2: Read any type of image/s that is/are stored in predefined directory and save features of image/s in text file. Step3: Read tree types for each image (Color image, Grayscale, Alpha channel (RGBA)). Step4: Convert the color image into gray scale as face detector of OpenCV. Journal of University of Babylon for Engineering Sciences, Vol. (28), No. (2): 2020. 28 Step5: Use Haar Classifiers for eyes and faces detection, which represents input training file of the Haar Classifier. Step6: Detect a face within an image based on the Cascade Classifier. Step7: Calculate the number of face/s detected. Step8: Output with face detected bounded by square.  Figure (4): Flowchart of face detection algorithm. IV. Results  The proposed blind people assistive system utilizes camera for reading the images is designed as can be seen in figure (5). The connected hardware with monitor and raspberry based Raspbian OS is used to obtain the simulation results. In SD memory card of the hardware part the Raspbian OS is installed.  The software part which is installed on it has been employed for recognizing the number from the objects in he captured image. After that the identified number converted and announced by voice based the earphone which is connected to the audio port in board of the raspberry pi. In order to evaluate the efficiency and effectiveness of the proposed blind people assistive system, different cases have been used, as can be seen in the experiment results of the following figures.   Journal of University of Babylon for Engineering Sciences, Vol. (28), No. (2): 2020. 29  Figure (5): Hardware wiring of the proposed system. The main components of the proposed system are, Raspberry pi3 connected with portable rechargable power bank to ensure suitable voltage to raspberry pi, also, web camera connected to USB port of raspberry pi, and headphone . Different cases are used to examin the accuracy of proposed system, and each case bounded the face and isolated it, however, the system also bounded the eyes, but this boundary is no longer imortant in this system, because the region of interset is the face and only face, but, also the eyes are the region can be considered and used to detecet. The size of image is very important, hence, the large size of image is not detected well, because many features in large size image with these simple algorithms may work revers and lose detection, where image with 200x200, 225x200 and  120x120 pixels can detect face and eyes as in Figure (6-8) respectively, while the image (9) 250x350 has detect one eye from three faces, and Figure (10) 2000x3000 pixels can not recognize eyes, just faces, anyway, face is the region of interest in this system not eyes, code is available in [11] can be downloaded and tested for any cases.    Figure (6): case1- Wearning Glass. Journal of University of Babylon for Engineering Sciences, Vol. (28), No. (2): 2020. 30  Figure (7): Case2- side face view.     Figure (8): Case3-Front face view.   Figure (9): Case4-More than one face. Journal of University of Babylon for Engineering Sciences, Vol. (28), No. (2): 2020. 31  Figure (10): Case5-Different ages. Conclusion  A computer vision-based face detection and recognition assistive system can improve blind and visually impaired people mobility to navigate independently. This work focuses on face detection using Haar cascade algorithm and OpenCV using python to help blind and visually impaired persons. The results of the detection and recognition based on the collected data show the effectiveness and efficiency of the proposed system. Further this project can be implemented in hardware to detect and recognize objects and vehicles on the road. So that it will help person not to cross the road during vehicle movement. it is also portable and compatible as well. Acknowledgments The authors would like to thank Prof. Jawad K. Ali, Department of Electrical Engineering, University of Technology, Baghdad- Iraq for helping and supporting us a lot in successfully completing this research project. References  [1] K. Goyal, K. Agarwal and R. Kumar, Face detection and tracking: Using OpenCV, 2017 International conference of Electronics, Communication and Aerospace Technology (ICECA), Coimbatore, 2017, pp. 474-478. [2] Chen X, AL Yuille, "Detecting and reading text in natural scenes, in Proc. Computer. Vision Pattern Recognition, 2004. [3] II-366–II-373. 2. Kumar S, R Gupta, et al. Text extraction and document image seg mentation using matched wavelets and MRF model, IEEE Trans Image Process, August 2007; 16:2117– 2128. [4] K Kim, K Jung, et al. Texture-based approach for text detection in images using support vector machines and continuously adaptive mean shift algorithm, IEEE Trans. Pattern Anal. Mach. Intell, December 2003; 25: 1631–1639. [5] N Giudice, G Legge, Blind navigation and the role of technology, in The Engineering Handbook of Smart Technology for Aging, Disability and Independence, AA Helal, M Mokhtari, B Abdulrazak, Eds. Hoboken, NJ, USA: Wiley, 2008. [6] Chen J Y, J Zhang, et al. Automatic detection and recognition of signs from natural scenes, IEEE Trans. Image Process., January 2004 ;13: 87–99. [7] D Dakopoulos, NG Bourbakis, Wearable obstacle avoidance electronic travel aids for blind: A survey, IEEE Trans. Syst., Man, Cybern, January 2010; 40: 25–35. [8] Abdul Muhsin M, Farah F. Alkhalid, Bashra Kadhim Oleiwi, “Online Blind Assistive System using Object Recognition”, International Research Journal of Innovations in Engineering and Technology – IRJIET, Volume 3, Issue 12, December 2019 pp. 47-51. [9] Gary Bradski and Adrian Kaehler, Learning OpenCV, Published by O’Reilly Media, 2008 Journal of University of Babylon for Engineering Sciences, Vol. (28), No. (2): 2020. 32 [10] P. Viola & M. Jones (2001), “Rapid Object Detection using a Boosted Cascade of Simple Feature”, Conference on Computer Vision and Pattern Recognition. IEEE Press, Pp. 511–518. [11] https://drive.google.com/file/d/1VIjt6AxUNXifzRPqvwdur5t9oatmPJu7/view?usp=sharing.                                   Journal of University of Babylon for Engineering Sciences, Vol. (28), No. (2): 2020. 33 OpenCV  60100@uotechnology.edu.iq    60010@uotechnology.edu.iq     10352@uptechnology.edu.iq pi3HaarOpenCVOpenCVHaarHaarOpencv  

Real Time Blind People Assistive System Based on OpenCV

https://www.journalofbabylon.com/index.php/JUBES/article/download/3167/2412

Real Time Blind People Assistive System Based on OpenCV

Abstract

Similar works

Full text

Available Versions

Journals of University of Babylon