Real-time people counting from video records is a main building bloc for many applications in smart cities. In practice, this task usually encounters many problems, like the lack of real-time processing of the recorded videos or the occurrence of errors due to irrelevant people being counted. To overcome the above issues, we propose a novel real-time people counting approach dubbed YOLO-PC (YOLO based People Counting)

Djahel, SD

Peiming Ren, PR

Wei Fang, WF

E-space: Manchester Metropolitan University's Research Repository

A Novel YOLO-based Real-time People CountingApproachPeiming Ren∓, Wei Fang∓* and Soufiene Djahel±∓Department of Computer Science and Technology, School of IoT Engineering, Jiangnan University, Wuxi, Jiangsu, China±School of Computing, Mathematics and Digital Technologies, Manchester Metropolitan University, UK{peimingren@163.com, fangwei@jiangnan.edu.cn, s.djahel@mmu.ac.uk}Abstract—Real-time people counting from video records isa main building bloc for many applications in smart cities.In practice, this task usually encounters many problems, likethe lack of real-time processing of the recorded videos or theoccurrence of errors due to irrelevant people being counted. Toovercome the above issues, we propose a novel real-time peoplecounting approach dubbed YOLO-PC (YOLO based PeopleCounting).Index Terms—People-counting; Boundary-selection; YOLOI. YOLO-PC: AN OVERVIEWIn this work we modify the pioneer object detection sys-tem YOLO [1] [2] by proposing the so-called YOLO-PC(YOLO based People Counting). YOLO-PC extends the orig-inal YOLO system using a deep learning approach to achievemore accurate people counting. Due to its low computationoverhead, compared to other state-of-the-art object detectionsystems, such as R-CNN [3], Fast R-CNN [4], and Faster R-CNN [5], and its ability to detect objects in real-time, YOLOhas been chosen as the base approach in our YOLO-PC.In terms of real-time performance, YOLO-PC re-trains adeep convolution neural network to detect people at more than40 fps (frames per second) with the support of a GPU. Regard-ing people counting method, on one hand, YOLO-PC dividesthe image into a 9*9 grid and makes use of the boundary,which leads to more detected boxes and higher average confi-dence value. On the other hand, YOLO-PC chooses differentboundary areas according to the actual application scenario tocount people contrapuntally and further improves the countingaccuracy. YOLO-PC can ignore the invalid persons who maybe in the billboards or in other irrelevant areas.Experimental results show that YOLO-PC can quickly countpeople with high accuracy at the entrance or exit of someplaces, such as escalators, scenic spots etc.II. YOLO-PC: THE MAIN STEPSThe operation of YOLO-PC consists in five steps as shownin Figure 1. The first step consists in setting the detectionthreshold and adjusting the camera. The detection resultsbelow the threshold, which is usually set from 0.2 to 0.4, willnot be counted. For the sake of simplicity, we use the defaultvalue of 0.2 in this work. In the actual scene, the camerashould be adjusted to the appropriate height and angle.Fig. 1: The process of YOLO-PCIn the second step, we detect people through re-training aconvolutional neural network. YOLO divides the image intoa 7*7 grid and for each grid cell predicts two boundingboxes as well as the confidence value for those boxes. Webelieve that this division is not sufficient and we aim thatour algorithm will be more efficient in identifying people toachieve higher counting accuracy. In other words, YOLO-PCworks better with more detected boxes and higher confidencevalues. To this end, YOLO-PC uses 9*9 grid and 3 boundingboxes. We set up three sets of experiments of 4 minutesvideo each using different thresholds (i.e., 0.2, 0.3 and 0.4).The obtained results, shown in Table 1, are promising sinceYOLO-PC detects more boxes and achieves higher confidencevalues for those boxes compared to YOLO. More specifically,YOLO-PC detects more than 10 percent of the boxes, theconfidence average value is more than 50 percent higher whenthe threshold is 0.2.The third step consists in detecting and counting peoplebased on an appropriate boundary selection. YOLO-PC selectsone or more grid cells as the area boundary from 243(9*9*3)cells and chooses a different boundary according to the actualsituation. If people turn left by somewhere, the boundary ofthe left area of the video should be selected, the value of theTABLE I: The number of detected boxes and the averageconfidence value at different threshold values (T)MethodBoxes number Average confidence valueT = 0.2 T = 0.3 T = 0.4 T = 0.2 T = 0.3 T = 0.4YOLO 13400 9423 5856 0.39 0.45 0.51YOLO-PC 14664 13612 12514 0.59 0.62 0.64Fig. 2: Left boundary selection sketch mapboundary is around 113 and people through that boundary willbe counted. Figure 2 shows the sketch map of the left boundaryarea and Figure 3 shows the experiment result. Similarly, ifpeople turn right by somewhere, the value of the boundary isaround 129. If people go straight by somewhere, the value ofthe boundary is around 121. YOLO-PC can be more accuratein detecting the flow of people as some nonsense interference,such as people in the billboards and unrelated backs, can beignored because of the boundary selection.The counting information of the previous step is nowprocessed in the fourth step. In the selected boundary area,the boxes number accumulates and constantly updates, werefer to this number as S. The value of S at a moment tin a video represents the number of detected people at thatmoment, which is very accurate. The value of S in a period oftime represents the total number of detected people. Becauseit takes time for people to move in the boundary area, thevalue of S is repeated, that is to say, the same person hasbeen detected many times. According to the experiments,every person has been detected around 18 times when passingthrough the selected boundary area, so the predicted numberis S/18 at the default threshold.In the fifth and last step, we output the real-time peoplecounting information. YOLO-PC can directly show the real-time people counting information in the video images, includ-ing the current number, FPS, confidence value etc. YOLO-PCcan also save real-time information and continue to update,and then output them through some interfaces.III. CONCLUSION AND FUTURE WORKIn this paper we introduced YOLO-PC, a YOLO basedreal-time people counting approach using boundary selection.(a) The 36th person (b) The 37th person (c) The 38th personFig. 3: The experimental results of the left boundary selection:the experimental video is a continuous high-definition videoat 30 fps, YOLO-PC continuously detects people and countsthem exactly at 40 fpsYOLO-PC outperforms YOLO as it re-trains YOLO network,which enables it to detect more boxes and achieve higheraverage confidence value. The boundary selection in YOLO-PC makes the counting more targeted and its result accurateand fast. In conclusion, this method is very effective and itis also able to recognize irrelevant people and ignore themin the counting process. YOLO-PC has a wide range ofapplications as it can assist the construction of many aspectsof the smart cities. On the current basis, we look forwardto making more improvements to this method by adding, forexample, abnormal behavior detection and children counting.ACKNOWLEDGMENTThis work was partially supported by the National Nat-ural Science foundation of China (Grant Nos. 61673194,61105128Key Research and Development Program of JiangsuProvince, China (Grant No. BE2017630), the PostdoctoralScience Foundation of China (Grant No. 2014M560390), SixTalent Peaks Project of Jiangsu Province (Grant No. DZXX-025), Natural Science Foundation for College and Universitiesin Jiangsu Province (Project Number: 16KJB520051).REFERENCES[1] J. Redmon et al., ”You Only Look Once: Unified, Real-Time ObjectDetection,” [EB/OL].https://pjreddie.com/darknet/yolov1/.[2] J. Redmon, et al., ”You only look once: Unified, real-time objectdetection,” [C]//Proceedings of the IEEE Conference on ComputerVision and Pattern Recognition. 2016: 779-788.[3] R. Girshick, et al., ”Rich feature hierarchies for accurate object detectionand semantic segmentation,” [C]//Proceedings of the IEEE conferenceon computer vision and pattern recognition. 2014: 580-587.[4] R. Girshick, ”Fast r-cnn,” [C]//Proceedings of the IEEE internationalconference on computer vision. 2015: 1440-1448. fig[5] S. Ren et al., ”Faster R-CNN: Towards real-time object detectionwith region proposal networks,” [C]//Advances in neural informationprocessing systems. 2015: 91-99.0

A novel YOLO-Based real-time people counting approach

Abstract

Similar works

Full text

Available Versions

E-space: Manchester Metropolitan University's Research Repository

Crossref