Real-time on-board pedestrian detection using generic single-stage algorithms and on-road databases

[EN] Pedestrian detection is a particular case of object detection that helps to reduce accidents in advanced driver-assistance systems and autonomous vehicles. It is not an easy task because of the variability of the objects and the time constraints. A performance comparison of object detection methods, including both GPU and non-GPU implementations over a variety of on-road specific databases, is provided. Computer vision multi-class object detection can be integrated on sensor fusion modules where recall is preferred over precision. For this reason, ad hoc training with a single class for pedestrians has been performed and we achieved a significant increase in recall. Experiments have been carried out on several architectures and a special effort has been devoted to achieve a feasible computational time for a real-time system. Finally, an analysis of the input image size allows to fine-tune the model and get better results with practical costs.The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by PRYSTINE project which had received funding within the Electronic Components and Systems for European Leadership Joint Undertaking (ECSEL JU) in collaboration with the European Union's H2020 Framework Programme and National Authorities, under grant agreement no. 783190. It was also funded by Generalitat Valenciana through the Instituto Valenciano de Competitividad Empresarial (IVACE).Ortiz, V.; Del Tejo Catala, O.; Salvador Igual, I.; Perez-Cortes, J. (2020). Real-time on-board pedestrian detection using generic single-stage algorithms and on-road databases. International Journal of Advanced Robotic Systems. 17(5). https://doi.org/10.1177/1729881420929175S175Zhang, S., Benenson, R., Omran, M., Hosang, J., & Schiele, B. (2018). Towards Reaching Human Performance in Pedestrian Detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(4), 973-986. doi:10.1109/tpami.2017.2700460Viola, P., Jones, M. J., & Snow, D. (2005). Detecting Pedestrians Using Patterns of Motion and Appearance. International Journal of Computer Vision, 63(2), 153-161. doi:10.1007/s11263-005-6644-8Dollar, P., Appel, R., Belongie, S., & Perona, P. (2014). Fast Feature Pyramids for Object Detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(8), 1532-1545. doi:10.1109/tpami.2014.2300479Dollar, P., Wojek, C., Schiele, B., & Perona, P. (2012). Pedestrian Detection: An Evaluation of the State of the Art. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(4), 743-761. doi:10.1109/tpami.2011.155Munder, S., & Gavrila, D. M. (2006). An Experimental Study on Pedestrian Classification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(11), 1863-1868. doi:10.1109/tpami.2006.217Enzweiler, M., & Gavrila, D. M. (2009). Monocular Pedestrian Detection: Survey and Experiments. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(12), 2179-2195. doi:10.1109/tpami.2008.260He, K., Zhang, X., Ren, S., & Sun, J. (2015). Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(9), 1904-1916. doi:10.1109/tpami.2015.2389824McGehee, D. V., Mazzae, E. N., & Baldwin, G. H. S. (2000). Driver Reaction Time in Crash Avoidance Research: Validation of a Driving Simulator Study on a Test Track. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 44(20), 3-320-3-323. doi:10.1177/15419312000440202

Implementation and Evaluation of a Cooperative Vehicle-to-Pedestrian Safety Application

While the development of Vehicle-to-Vehicle (V2V) safety applications based on Dedicated Short-Range Communications (DSRC) has been extensively undergoing standardization for more than a decade, such applications are extremely missing for Vulnerable Road Users (VRUs). Nonexistence of collaborative systems between VRUs and vehicles was the main reason for this lack of attention. Recent developments in Wi-Fi Direct and DSRC-enabled smartphones are changing this perspective. Leveraging the existing V2V platforms, we propose a new framework using a DSRC-enabled smartphone to extend safety benefits to VRUs. The interoperability of applications between vehicles and portable DSRC enabled devices is achieved through the SAE J2735 Personal Safety Message (PSM). However, considering the fact that VRU movement dynamics, response times, and crash scenarios are fundamentally different from vehicles, a specific framework should be designed for VRU safety applications to study their performance. In this article, we first propose an end-to-end Vehicle-to-Pedestrian (V2P) framework to provide situational awareness and hazard detection based on the most common and injury-prone crash scenarios. The details of our VRU safety module, including target classification and collision detection algorithms, are explained next. Furthermore, we propose and evaluate a mitigating solution for congestion and power consumption issues in such systems. Finally, the whole system is implemented and analyzed for realistic crash scenarios

FuSSI-Net: Fusion of Spatio-temporal Skeletons for Intention Prediction Network

Pedestrian intention recognition is very important to develop robust and safe autonomous driving (AD) and advanced driver assistance systems (ADAS) functionalities for urban driving. In this work, we develop an end-to-end pedestrian intention framework that performs well on day- and night- time scenarios. Our framework relies on objection detection bounding boxes combined with skeletal features of human pose. We study early, late, and combined (early and late) fusion mechanisms to exploit the skeletal features and reduce false positives as well to improve the intention prediction performance. The early fusion mechanism results in AP of 0.89 and precision/recall of 0.79/0.89 for pedestrian intention classification. Furthermore, we propose three new metrics to properly evaluate the pedestrian intention systems. Under these new evaluation metrics for the intention prediction, the proposed end-to-end network offers accurate pedestrian intention up to half a second ahead of the actual risky maneuver.Comment: 5 pages, 6 figures, 5 tables, IEEE Asilomar SS