Reduced computational cost prototype for street theft detection based on depth decrement in Convolutional Neural Network. Application to Command and Control Information Systems (C2IS) in the National Police of Colombia

[EN] This paper shows the implementation of a prototype of street theft detector using the deep learning technique R- CNN (Region-Based Convolutional Network), applied in the Command and Control Information System (C2IS) of National Police of Colombia, the prototype is implemented using three models of CNN (Convolutional Neural Network), AlexNet, VGG16 and VGG19 comparing their computational cost measuring the image processing time, according to the complexity (depth) of each model. Finally, we conclude which model has the lowest computational cost and is more useful for the case of the National Police of Colombia.Suarez-Paez, JE.; Salcedo-González, ML.; Esteve Domingo, M.; Gomez, J.; Palau Salvador, CE.; Pérez Llopis, I. (2018). Reduced computational cost prototype for street theft detection based on depth decrement in Convolutional Neural Network. Application to Command and Control Information Systems (C2IS) in the National Police of Colombia. International Journal of Computational Intelligence Systems. 12(1):123-130. https://doi.org/10.2991/ijcis.2018.25905186S12313012

A Novel Low Processing Time System for Criminal Activities Detection Applied to Command and Control Citizen Security Centers

[EN] This paper shows a Novel Low Processing Time System focused on criminal activities detection based on real-time video analysis applied to Command and Control Citizen Security Centers. This system was applied to the detection and classification of criminal events in a real-time video surveillance subsystem in the Command and Control Citizen Security Center of the Colombian National Police. It was developed using a novel application of Deep Learning, specifically a Faster Region-Based Convolutional Network (R-CNN) for the detection of criminal activities treated as "objects" to be detected in real-time video. In order to maximize the system efficiency and reduce the processing time of each video frame, the pretrained CNN (Convolutional Neural Network) model AlexNet was used and the fine training was carried out with a dataset built for this project, formed by objects commonly used in criminal activities such as short firearms and bladed weapons. In addition, the system was trained for street theft detection. The system can generate alarms when detecting street theft, short firearms and bladed weapons, improving situational awareness and facilitating strategic decision making in the Command and Control Citizen Security Center of the Colombian National Police.This work was co-funded by the European Commission as part of H2020 call SEC-12-FCT-2016-Subtopic3 under the project VICTORIA (No. 740754). This publication reflects the views only of the authors and the Commission cannot be held responsible for any use which may be made of the information contained therein.Suarez-Paez, J.; Salcedo-Gonzalez, M.; Climente, A.; Esteve Domingo, M.; Gomez, J.; Palau Salvador, CE.; Pérez Llopis, I. (2019). A Novel Low Processing Time System for Criminal Activities Detection Applied to Command and Control Citizen Security Centers. Information. 10(12):1-19. https://doi.org/10.3390/info10120365S1191012Wang, L., Rodriguez, R. M., & Wang, Y.-M. (2018). A dynamic multi-attribute group emergency decision making method considering expertsr hesitation. International Journal of Computational Intelligence Systems, 11(1), 163. doi:10.2991/ijcis.11.1.13Esteve, M., Perez-Llopis, I., & Palau, C. E. (2013). Friendly Force Tracking COTS solution. IEEE Aerospace and Electronic Systems Magazine, 28(1), 14-21. doi:10.1109/maes.2013.6470440Senst, T., Eiselein, V., Kuhn, A., & Sikora, T. (2017). Crowd Violence Detection Using Global Motion-Compensated Lagrangian Features and Scale-Sensitive Video-Level Representation. IEEE Transactions on Information Forensics and Security, 12(12), 2945-2956. doi:10.1109/tifs.2017.2725820Shi, Y., Tian, Y., Wang, Y., & Huang, T. (2017). Sequential Deep Trajectory Descriptor for Action Recognition With Three-Stream CNN. IEEE Transactions on Multimedia, 19(7), 1510-1520. doi:10.1109/tmm.2017.2666540Arunnehru, J., Chamundeeswari, G., & Bharathi, S. P. (2018). Human Action Recognition using 3D Convolutional Neural Networks with 3D Motion Cuboids in Surveillance Videos. Procedia Computer Science, 133, 471-477. doi:10.1016/j.procs.2018.07.059Kamel, A., Sheng, B., Yang, P., Li, P., Shen, R., & Feng, D. D. (2019). Deep Convolutional Neural Networks for Human Action Recognition Using Depth Maps and Postures. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 49(9), 1806-1819. doi:10.1109/tsmc.2018.2850149Zhang, B., Wang, L., Wang, Z., Qiao, Y., & Wang, H. (2018). Real-Time Action Recognition With Deeply Transferred Motion Vector CNNs. IEEE Transactions on Image Processing, 27(5), 2326-2339. doi:10.1109/tip.2018.2791180Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2016). Region-Based Convolutional Networks for Accurate Object Detection and Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(1), 142-158. doi:10.1109/tpami.2015.2437384Suarez-Paez, J., Salcedo-Gonzalez, M., Esteve, M., Gómez, J. A., Palau, C., & Pérez-Llopis, I. (2018). Reduced computational cost prototype for street theft detection based on depth decrement in Convolutional Neural Network. Application to Command and Control Information Systems (C2IS) in the National Police of Colombia. International Journal of Computational Intelligence Systems, 12(1), 123. doi:10.2991/ijcis.2018.25905186Ren, S., He, K., Girshick, R., & Sun, J. (2017). Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6), 1137-1149. doi:10.1109/tpami.2016.2577031Hao, S., Wang, P., & Hu, Y. (2019). Haze Image Recognition Based on Brightness Optimization Feedback and Color Correction. Information, 10(2), 81. doi:10.3390/info10020081Peng, M., Wang, C., Chen, T., & Liu, G. (2016). NIRFaceNet: A Convolutional Neural Network for Near-Infrared Face Identification. Information, 7(4), 61. doi:10.3390/info7040061NVIDIA CUDA® Deep Neural Network library (cuDNN)https://developer.nvidia.com/cuda-downloadsWu, X., Lu, X., & Leung, H. (2018). A Video Based Fire Smoke Detection Using Robust AdaBoost. Sensors, 18(11), 3780. doi:10.3390/s18113780Park, J. H., Lee, S., Yun, S., Kim, H., & Kim, W.-T. (2019). Dependable Fire Detection System with Multifunctional Artificial Intelligence Framework. Sensors, 19(9), 2025. doi:10.3390/s19092025García-Retuerta, D., Bartolomé, Á., Chamoso, P., & Corchado, J. M. (2019). Counter-Terrorism Video Analysis Using Hash-Based Algorithms. Algorithms, 12(5), 110. doi:10.3390/a12050110Zhao, B., Zhao, B., Tang, L., Han, Y., & Wang, W. (2018). Deep Spatial-Temporal Joint Feature Representation for Video Object Detection. Sensors, 18(3), 774. doi:10.3390/s18030774He, Z., & He, H. (2018). Unsupervised Multi-Object Detection for Video Surveillance Using Memory-Based Recurrent Attention Networks. Symmetry, 10(9), 375. doi:10.3390/sym10090375Muhammad, K., Hamza, R., Ahmad, J., Lloret, J., Wang, H., & Baik, S. W. (2018). Secure Surveillance Framework for IoT Systems Using Probabilistic Image Encryption. IEEE Transactions on Industrial Informatics, 14(8), 3679-3689. doi:10.1109/tii.2018.2791944Barthélemy, J., Verstaevel, N., Forehead, H., & Perez, P. (2019). Edge-Computing Video Analytics for Real-Time Traffic Monitoring in a Smart City. Sensors, 19(9), 2048. doi:10.3390/s19092048Aqib, M., Mehmood, R., Alzahrani, A., Katib, I., Albeshri, A., & Altowaijri, S. M. (2019). Smarter Traffic Prediction Using Big Data, In-Memory Computing, Deep Learning and GPUs. Sensors, 19(9), 2206. doi:10.3390/s19092206Xu, S., Zou, S., Han, Y., & Qu, Y. (2018). Study on the Availability of 4T-APS as a Video Monitor and Radiation Detector in Nuclear Accidents. Sustainability, 10(7), 2172. doi:10.3390/su10072172Plageras, A. P., Psannis, K. E., Stergiou, C., Wang, H., & Gupta, B. B. (2018). Efficient IoT-based sensor BIG Data collection–processing and analysis in smart buildings. Future Generation Computer Systems, 82, 349-357. doi:10.1016/j.future.2017.09.082Jha, S., Dey, A., Kumar, R., & Kumar-Solanki, V. (2019). A Novel Approach on Visual Question Answering by Parameter Prediction using Faster Region Based Convolutional Neural Network. International Journal of Interactive Multimedia and Artificial Intelligence, 5(5), 30. doi:10.9781/ijimai.2018.08.004Cho, S., Baek, N., Kim, M., Koo, J., Kim, J., & Park, K. (2018). Face Detection in Nighttime Images Using Visible-Light Camera Sensors with Two-Step Faster Region-Based Convolutional Neural Network. Sensors, 18(9), 2995. doi:10.3390/s18092995Zhang, J., Xing, W., Xing, M., & Sun, G. (2018). Terahertz Image Detection with the Improved Faster Region-Based Convolutional Neural Network. Sensors, 18(7), 2327. doi:10.3390/s18072327Bakheet, S., & Al-Hamadi, A. (2016). A Discriminative Framework for Action Recognition Using f-HOL Features. Information, 7(4), 68. doi:10.3390/info7040068(2018). Robust Eye Blink Detection Based on Eye Landmarks and Savitzky–Golay Filtering. Information, 9(4), 93. doi:10.3390/info9040093Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017). ImageNet classification with deep convolutional neural networks. Communications of the ACM, 60(6), 84-90. doi:10.1145/3065386Jetson Embedded Development Kit|NVIDIAhttps://developer.nvidia.com/embedded-computingNVIDIA TensorRT|NVIDIA Developerhttps://developer.nvidia.com/tensorrtNVIDIA DeepStream SDK|NVIDIA Developerhttps://developer.nvidia.com/deepstream-sdkFraga-Lamas, P., Fernández-Caramés, T., Suárez-Albela, M., Castedo, L., & González-López, M. (2016). A Review on Internet of Things for Defense and Public Safety. Sensors, 16(10), 1644. doi:10.3390/s16101644Gomez, C., Shami, A., & Wang, X. (2018). Machine Learning Aided Scheme for Load Balancing in Dense IoT Networks. Sensors, 18(11), 3779. doi:10.3390/s18113779AMD Embedded RadeonTMhttps://www.amd.com/en/products/embedded-graphic

Arquitectura de detección de actividades criminales basada en análisis de vídeo en tiempo real

[ES] Esta tesis doctoral propone el desarrollo de una arquitectura para sistema de detección de actividades criminales en vídeo aplicado a sistemas de mando y control para seguridad ciudadana. Este sistema está basado en la técnica de Deep Learning Faster R-CNN y tiene el novedoso enfoque de tratar las acciones criminales como los hurtos callejeros, en donde pueden ser identificados objetos como evidencia en una escena de vídeo. Esta tesis muestra el desarrollo de dicha aplicación, que demuestra ser efectiva, identificando la manera de reducir el costo computacional del análisis de vídeo cuadro a cuadro obteniendo rendimientos congruentes con las tasas de cuadros por segundo generados por cámaras de sistema de vídeo vigilancia ciudadana. También es objeto de estudio una posible implementación en el sistema de seguridad ciudadana de la Policía Nacional de Colombia.[EN] This doctoral thesis proposes the development of a system to detect criminal activities in video applied to command and control systems for citizen security. This system is based on the Deep Learning technique called Faster R-CNN and has the novel approach of treating criminal actions like street thefts as objects that can be identified in a video scene. This thesis shows the development of this application and the way to reduce the computational cost of the video analysis frame by frame, obtaining performances congruent with the frame rate generated by citizen video surveillance system cameras. There is also a possible implementation in the citizen security system of the National Police of Colombia is being studied.[CA] Esta tesi doctoral proposa el desenrotllament d'una arquitectura per a sistema de detecció d'activitats criminals en vídeo aplicat a sistemes de comandament i control per a seguretat ciutadana. Este sistema està basat en la tècnica de Deep Learning Faster R-CNN i té el nou enfocament de tractar les accions criminals com les afanades guies de carrers com a objectes que poden ser identificats en una escena de vídeo. Esta tesi mostra el desenrotllament de la dita aplicació, que demostra ser efectiva, identificant la manera de reduir el cost computacional de l'anàlisi de vídeo quadro a quadro obtenint rendiments congruents amb les taxes de cuados per segon generats per cambres de sistema de vídeo vigilància ciutadana. També s'estudia una possible implementació en el sistema de seguretat ciutadana de la Policia Nacional de Colòmbia.Suárez Páez, JE. (2020). Arquitectura de detección de actividades criminales basada en análisis de vídeo en tiempo real [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/153162TESI

A Novel Method of Spatiotemporal Dynamic Geo-Visualization of Criminal Data, Applied to Command and Control Centers for Public Safety

[EN] This article shows a novel geo-visualization method of dynamic spatiotemporal data that allows mobility and concentration of criminal activity to be study. The method was developed using, only and significantly, real data of Santiago de Cali (Colombia), collected by the Colombian National Police (PONAL). This method constitutes a tool that allows criminal influx to be analyzed by concentration, zone, time slot and date. In addition to the field experience of police commanders, it allows patterns of criminal activity to be detected, thereby enabling a better distribution and management of police resources allocated to crime deterrence, prevention and control. Additionally, it may be applied to the concepts of safe city and smart city of the PONAL within the architecture of Command and Control System (C2S) of Command and Control Centers for Public Safety. Furthermore, it contributes to a better situational awareness and improves the future projection, agility, efficiency and decision-making processes of police officers, which are all essential for fulfillment of police missions against crime. Finally, this was developed using an open source software, it can be adapted to any other city, be used with real-time data and be implemented, if necessary, with the geographic software of any other C2S.This work was co-funded by the European Commission as part of H2020 call SEC-12-FCT-2016-thrtopic3 under the project VICTORIA (No. 740754). This publication reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein. The authors would like to thank Colombian National Police and its Office of Telematics for their support on development of this project.Salcedo-González, ML.; Suarez-Paez, JE.; Esteve Domingo, M.; Gomez, J.; Palau Salvador, CE. (2020). A Novel Method of Spatiotemporal Dynamic Geo-Visualization of Criminal Data, Applied to Command and Control Centers for Public Safety. ISPRS International Journal of Geo-Information. 9(3):1-17. https://doi.org/10.3390/ijgi9030160S11793Lacinák, M., & Ristvej, J. (2017). Smart City, Safety and Security. Procedia Engineering, 192, 522-527. doi:10.1016/j.proeng.2017.06.090Neumann, M., & Elsenbroich, C. (2016). Introduction: the societal dimensions of organized crime. Trends in Organized Crime, 20(1-2), 1-15. doi:10.1007/s12117-016-9294-zPhillips, P., & Lee, I. (2012). Mining co-distribution patterns for large crime datasets. Expert Systems with Applications, 39(14), 11556-11563. doi:10.1016/j.eswa.2012.03.071Linning, S. J. (2015). Crime seasonality and the micro-spatial patterns of property crime in Vancouver, BC and Ottawa, ON. Journal of Criminal Justice, 43(6), 544-555. doi:10.1016/j.jcrimjus.2015.05.007Spicer, V., & Song, J. (2017). The impact of transit growth on the perception of crime. Journal of Environmental Psychology, 54, 151-159. doi:10.1016/j.jenvp.2017.09.002Beland, L.-P., & Brent, D. A. (2018). Traffic and crime. Journal of Public Economics, 160, 96-116. doi:10.1016/j.jpubeco.2018.03.002Newspaper of National Circulation in Colombia, E.T. Robos en Trancones en El Tintal—Bogotá—.ELTIEMPO.COM https://www.eltiempo.com/bogota/robos-en-trancones-en-el-tintal-168226Nueva Modalidad de Atraco a Conductores en Los Trancones de Bogotá|ELESPECTADOR.COM http://www.elespectador.com/noticias/bogota/nueva-modalidad-de-atraco-conductores-en-los-trancones-de-bogota-articulo-697716Carrillo, P. E., Lopez-Luzuriaga, A., & Malik, A. S. (2018). Pollution or crime: The effect of driving restrictions on criminal activity. Journal of Public Economics, 164, 50-69. doi:10.1016/j.jpubeco.2018.05.007Twinam, T. (2017). Danger zone: Land use and the geography of neighborhood crime. Journal of Urban Economics, 100, 104-119. doi:10.1016/j.jue.2017.05.006Sadler, R. C., Pizarro, J., Turchan, B., Gasteyer, S. P., & McGarrell, E. F. (2017). Exploring the spatial-temporal relationships between a community greening program and neighborhood rates of crime. Applied Geography, 83, 13-26. doi:10.1016/j.apgeog.2017.03.017Roth, R. E., Ross, K. S., Finch, B. G., Luo, W., & MacEachren, A. M. (2013). Spatiotemporal crime analysis in U.S. law enforcement agencies: Current practices and unmet needs. Government Information Quarterly, 30(3), 226-240. doi:10.1016/j.giq.2013.02.001Sustainable Development Goals|UNDP https://www.undp.org/content/undp/en/home/sustainable-development-goals.htmlGiménez-Santana, A., Caplan, J. M., & Drawve, G. (2018). Risk Terrain Modeling and Socio-Economic Stratification: Identifying Risky Places for Violent Crime Victimization in Bogotá, Colombia. European Journal on Criminal Policy and Research, 24(4), 417-431. doi:10.1007/s10610-018-9374-5Kim, S., Jeong, S., Woo, I., Jang, Y., Maciejewski, R., & Ebert, D. S. (2018). Data Flow Analysis and Visualization for Spatiotemporal Statistical Data without Trajectory Information. IEEE Transactions on Visualization and Computer Graphics, 24(3), 1287-1300. doi:10.1109/tvcg.2017.2666146Kounadi, O., & Leitner, M. (2014). Spatial Information Divergence: Using Global and Local Indices to Compare Geographical Masks Applied to Crime Data. Transactions in GIS, 19(5), 737-757. doi:10.1111/tgis.12125Khalid, S., Shoaib, F., Qian, T., Rui, Y., Bari, A. I., Sajjad, M., … Wang, J. (2017). Network Constrained Spatio-Temporal Hotspot Mapping of Crimes in Faisalabad. Applied Spatial Analysis and Policy, 11(3), 599-622. doi:10.1007/s12061-017-9230-xLopez-Cuevas, A., Medina-Perez, M. A., Monroy, R., Ramirez-Marquez, J. E., & Trejo, L. A. (2018). FiToViz: A Visualisation Approach for Real-Time Risk Situation Awareness. IEEE Transactions on Affective Computing, 9(3), 372-382. doi:10.1109/taffc.2017.2741478Xue, Y., & Brown, D. E. (2006). Spatial analysis with preference specification of latent decision makers for criminal event prediction. Decision Support Systems, 41(3), 560-573. doi:10.1016/j.dss.2004.06.007Nakaya, T., & Yano, K. (2010). Visualising Crime Clusters in a Space-time Cube: An Exploratory Data-analysis Approach Using Space-time Kernel Density Estimation and Scan Statistics. Transactions in GIS, 14(3), 223-239. doi:10.1111/j.1467-9671.2010.01194.xAnuar, N. B., & Yap, B. W. (2018). Data Visualization of Violent Crime Hotspots in Malaysia. Soft Computing in Data Science, 350-363. doi:10.1007/978-981-13-3441-2_27Malik, A., Maciejewski, R., Towers, S., McCullough, S., & Ebert, D. S. (2014). Proactive Spatiotemporal Resource Allocation and Predictive Visual Analytics for Community Policing and Law Enforcement. IEEE Transactions on Visualization and Computer Graphics, 20(12), 1863-1872. doi:10.1109/tvcg.2014.2346926Arietta, S. M., Efros, A. A., Ramamoorthi, R., & Agrawala, M. (2014). City Forensics: Using Visual Elements to Predict Non-Visual City Attributes. IEEE Transactions on Visualization and Computer Graphics, 20(12), 2624-2633. doi:10.1109/tvcg.2014.2346446Hu, Y., Wang, F., Guin, C., & Zhu, H. (2018). A spatio-temporal kernel density estimation framework for predictive crime hotspot mapping and evaluation. Applied Geography, 99, 89-97. doi:10.1016/j.apgeog.2018.08.001Yang, D., Heaney, T., Tonon, A., Wang, L., & Cudré-Mauroux, P. (2017). CrimeTelescope: crime hotspot prediction based on urban and social media data fusion. World Wide Web, 21(5), 1323-1347. doi:10.1007/s11280-017-0515-4ToppiReddy, H. K. R., Saini, B., & Mahajan, G. (2018). Crime Prediction & Monitoring Framework Based on Spatial Analysis. Procedia Computer Science, 132, 696-705. doi:10.1016/j.procs.2018.05.075Devia, N., & Weber, R. (2013). Generating crime data using agent-based simulation. Computers, Environment and Urban Systems, 42, 26-41. doi:10.1016/j.compenvurbsys.2013.09.001Kuo, P.-F., Lord, D., & Walden, T. D. (2013). Using geographical information systems to organize police patrol routes effectively by grouping hotspots of crash and crime data. Journal of Transport Geography, 30, 138-148. doi:10.1016/j.jtrangeo.2013.04.006Camacho-Collados, M., & Liberatore, F. (2015). A Decision Support System for predictive police patrolling. Decision Support Systems, 75, 25-37. doi:10.1016/j.dss.2015.04.012Kagawa, T., Saiki, S., & Nakamura, M. (2019). Analyzing street crimes in Kobe city using PRISM. International Journal of Web Information Systems, 15(2), 183-200. doi:10.1108/ijwis-04-2018-0032Jentner, W., Sacha, D., Stoffel, F., Ellis, G., Zhang, L., & Keim, D. A. (2018). Making machine intelligence less scary for criminal analysts: reflections on designing a visual comparative case analysis tool. The Visual Computer, 34(9), 1225-1241. doi:10.1007/s00371-018-1483-0Suarez-Paez, J., Salcedo-Gonzalez, M., Esteve, M., Gómez, J. A., Palau, C., & Pérez-Llopis, I. (2018). Reduced computational cost prototype for street theft detection based on depth decrement in Convolutional Neural Network. Application to Command and Control Information Systems (C2IS) in the National Police of Colombia. International Journal of Computational Intelligence Systems, 12(1), 123. doi:10.2991/ijcis.2018.25905186Suarez-Paez, J., Salcedo-Gonzalez, M., Climente, A., Esteve, M., Gómez, J. A., Palau, C. E., & Pérez-Llopis, I. (2019). A Novel Low Processing Time System for Criminal Activities Detection Applied to Command and Control Citizen Security Centers. Information, 10(12), 365. doi:10.3390/info10120365Esteve, M., Perez-Llopis, I., & Palau, C. E. (2013). Friendly Force Tracking COTS solution. IEEE Aerospace and Electronic Systems Magazine, 28(1), 14-21. doi:10.1109/maes.2013.6470440Esteve, M., Perez-Llopis, I., Hernandez-Blanco, L. E., Palau, C. E., & Carvajal, F. (2007). SIMACOP: Small Units Management C4ISR System. Multimedia and Expo, 2007 IEEE International Conference on. doi:10.1109/icme.2007.4284862OpenStreetMap http://www.openstreetmap.or