Deep learning-based anomalous object detection system powered by microcontroller for PTZ cameras

Automatic video surveillance systems are usually designed to detect anomalous objects being present in a scene or behaving dangerously. In order to perform adequately, they must incorporate models able to achieve accurate pattern recognition in an image, and deep learning neural networks excel at this task. However, exhaustive scan of the full image results in multiple image blocks or windows to analyze, which could make the time performance of the system very poor when implemented on low cost devices. This paper presents a system which attempts to detect abnormal moving objects within an area covered by a PTZ camera while it is panning. The decision about the block of the image to analyze is based on a mixture distribution composed of two components: a uniform probability distribution, which represents a blind random selection, and a mixture of Gaussian probability distributions. Gaussian distributions represent windows in the image where anomalous objects were detected previously and contribute to generate the next window to analyze close to those windows of interest. The system is implemented on a Raspberry Pi microcontroller-based board, which enables the design and implementation of a low-cost monitoring system that is able to perform image processing.Universidad de Málaga. Campus de Excelencia Internacional Andalucía Tech

Development of artificial neural network-based object detection algorithms for low-cost hardware devices

Finally, the fourth work was published in the “WCCI” conference in 2020 and consisted of an individuals' position estimation algorithm based on a novel neural network model for environments with forbidden regions, named “Forbidden Regions Growing Neural Gas”.The human brain is the most complex, powerful and versatile learning machine ever known. Consequently, many scientists of various disciplines are fascinated by its structures and information processing methods. Due to the quality and quantity of the information extracted from the sense of sight, image is one of the main information channels used by humans. However, the massive amount of video footage generated nowadays makes it difficult to process those data fast enough manually. Thus, computer vision systems represent a fundamental tool in the extraction of information from digital images, as well as a major challenge for scientists and engineers. This thesis' primary objective is automatic foreground object detection and classification through digital image analysis, using artificial neural network-based techniques, specifically designed and optimised to be deployed in low-cost hardware devices. This objective will be complemented by developing individuals' movement estimation methods by using unsupervised learning and artificial neural network-based models. The cited objectives have been addressed through a research work illustrated in four publications supporting this thesis. The first one was published in the “ICAE” journal in 2018 and consists of a neural network-based movement detection system for Pan-Tilt-Zoom (PTZ) cameras deployed in a Raspberry Pi board. The second one was published in the “WCCI” conference in 2018 and consists of a deep learning-based automatic video surveillance system for PTZ cameras deployed in low-cost hardware. The third one was published in the “ICAE” journal in 2020 and consists of an anomalous foreground object detection and classification system for panoramic cameras, based on deep learning and supported by low-cost hardware

Deep learning-based anomalous object detection system for panoramic cameras managed by a Jetson TX2 board

Social conflicts appearing in the media are increas ing public awareness about security issues, resulting in a higher demand of more exhaustive environment monitoring methods. Automatic video surveillance systems are a powerful assistance to public and private security agents. Since the arrival of deep learn ing, object detection and classification systems have experienced a large improvement in both accuracy and versatility. However, deep learning-based object detection and classification systems often require expensive GPU-based hardware to work properly. This paper presents a novel deep learning-based foreground anomalous object detection system for video streams supplied by panoramic cameras, specially designed to build power efficient video surveillance systems. The system optimises the process of searching for anomalous objects through a new potential detection generator managed by three different multivariant homoscedastic distributions. Experimental results obtained after its deployment in a Jetson TX2 board attest the good performance of the system, postulating it as a solvent approach to power saving video surveillance systems.This work is partially supported by the Ministry of Economy and Competitiveness of Spain under grants TIN2016-75097- P and PPIT.UMA.B1.2017. It is also partially supported by the Ministry of Science, Innovation and Universities of Spain under grant RTI2018-094645-B-I00, project name Automated detection with low-cost hardware of unusual activities in video sequences. It is also partially supported by the Autonomous Government of Andalusia (Spain) under project UMA18- FEDERJA-084, project name Detection of anomalous behavior agents by deep learning in low-cost video surveillance intel ligent systems. All of them include funds from the European Regional Development Fund (ERDF). The authors thankfully acknowledge the computer resources, technical expertise and assistance provided by the SCBI (Supercomputing and Bioin formatics) center of the University of Malaga. They also ´ Authorized licensed use limited to: Universidad de Malaga. Downloaded on February 06,2024 at 07:21:43 UTC from IEEE Xplore. Restrictions apply. gratefully acknowledge the support of NVIDIA Corporation with the donation of two Titan X GPUs used for this research. The authors acknowledge the funding from the Universidad de Malaga

Sistema de reconocimiento facial sin reentrenamiento para nuevos usuarios

Deep learning has become increasingly popular and widely applied to computer vision systems. Over the years, researchers have developed various deep learning architectures to solve different kinds of problems. However, these networks are power-hungry and require high-performance computing (i.e., GPU, TPU, etc.) to run appropriately. Moving computation to the cloud may result in traffic, latency, and privacy issues. Edge computing can solve these challenges by moving the computing closer to the edge where the data is generated. One major challenge is to fit the high resource demands of deep learning in less powerful edge computing devices. In this research, we present an implementation of an embedded facial recognition system on a low cost Raspberry Pi, which is based on the FaceNet architecture. For this implementation it was required the development of a library in C++, which allows the deployment of the inference of the Neural Network Architecture. The system had an accuracy and precision of 77.38% and 81.25%, respectively. The time of execution of the program is 11 seconds and it consumes 46 [kB] of RAM. The resulting system could be utilized as a stand-alone access control system. The implemented model and library are released at https://github.com/cristianMiranda-Oro/FaceNet_EmbeddedSystemEl aprendizaje profundo se ha vuelto cada vez más popular y se aplica ampliamente a los sistemas de visión por computadora. A lo largo de los años, los investigadores han desarrollado varias arquitecturas de aprendizaje profundo para resolver diferentes tipos de problemas. Sin embargo, estas redes consumen mucha energía y requieren computación de alto rendimiento (es decir, GPU, TPU, etc.) para funcionar correctamente. Mover la computación a la nube puede resultar en problemas de tráfico, latencia y privacidad. La computación en el borde puede resolver estos desafíos, pues permite acercar el proceso de computación al lugar donde se generan los datos. Un desafío importante es adaptar las altas demandas de recursos del aprendizaje profundo a dispositivos de computación de borde menos potentes. En esta investigación, presentamos una implementación de un sistema de reconocimiento facial integrado en una Raspberry Pi de bajo costo, la cual está basada en la red FaceNet. Esta implementación requirió el desarrollo de una biblioteca en C++ que puede describir la inferencia de la arquitectura de la red neuronal FaceNet. El sistema tuvo una exactitud y precisión de 77.38% y del 81.25 %,  respectivamente. El tiempo de ejecución de cada inferencia es de 11 segundos y consume 46 [kB] de RAM. El sistema resultante podría utilizarse como un sistema de control de acceso independiente. El modelo y la librería implementados están disponibles en https://github.com/cristianMiranda-Oro/FaceNet_EmbeddedSystem

Sistema de reconocimiento facial sin reentrenamiento para nuevos usuarios

Deep learning has become increasingly popular and widely applied to computer vision systems. Over the years, researchers have developed various deep learning architectures to solve different kinds of problems. However, these networks are power-hungry and require high-performance computing (i.e., GPU, TPU, etc.) to run appropriately. Moving computation to the cloud may result in traffic, latency, and privacy issues. Edge computing can solve these challenges by moving the computing closer to the edge where the data is generated. One major challenge is to fit the high resource demands of deep learning in less powerful edge computing devices. In this research, we present an implementation of an embedded facial recognition system on a low cost Raspberry Pi, which is based on the FaceNet architecture. For this implementation it was required the development of a library in C++, which allows the deployment of the inference of the Neural Network Architecture. The system had an accuracy and precision of 77.38% and 81.25%, respectively. The time of execution of the program is 11 seconds and it consumes 46 [kB] of RAM. The resulting system could be utilized as a stand-alone access control system. The implemented model and library are released at https://github.com/cristianMiranda-Oro/FaceNet_EmbeddedSystemEl aprendizaje profundo se ha vuelto cada vez más popular y se aplica ampliamente a los sistemas de visión por computadora. A lo largo de los años, los investigadores han desarrollado varias arquitecturas de aprendizaje profundo para resolver diferentes tipos de problemas. Sin embargo,estas redes consumen mucha energía y requieren computación de alto rendimiento (es decir, GPU, TPU, etc.) para funcionar correctamente. Mover la computación a la nube puede resultar en problemas de tráfico, latencia y privacidad. La computación en el borde puede resolver estos desafíos, pues permite acercar el proceso de computación al lugar donde se generan los datos. Un desafío importante es adaptar las altas demandas de recursos del aprendizaje profundo a dispositivos de computación de borde menos potentes. En esta investigación, presentamos una implementación de unsistema de reconocimiento facial integrado en una Raspberry Pi de bajo costo, la cual está basada en la red FaceNet. Esta implementación requirió el desarrollo de una biblioteca en C++ que puede describir la inferencia dela arquitectura de la red neuronal FaceNet. El sistema tuvo una exactitud y precisión de 77.38% y del 81.25 %,  respectivamente. El tiempo de ejecución de cada inferencia es de 11 segundos y consume 46 [kB] de RAM. El sistema resultante podría utilizarse como un sistema de control de acceso independiente. El modelo y la librería implementados están disponibles en https://github.com/cristianMiranda-Oro/FaceNet_EmbeddedSystem

Low power CMOS vision sensor for foreground segmentation

This thesis focuses on the design of a top-ranked algorithm for background subtraction, the Pixel Adaptive Based Segmenter (PBAS), for its mapping onto a CMOS vision sensor on the focal plane processing. The redesign of PBAS into its hardware oriented version, HO-PBAS, has led to a less number of memories per pixel, along with a simpler overall model, yet, resulting in an acceptable loss of accuracy with respect to its counterpart on CPU. This thesis features two CMOS vision sensors. The first one, HOPBAS1K, has laid out a 24 x 56 pixel array onto a miniasic chip in standard 180 nm CMOS technology. The second one, HOPBAS10K, features an array of 98 x 98 pixels in standard 180 nm CMOS technology too. The second chip fixes some issues found in the first chip, and provides good hardware and background performance metrics

Intelligent Sensor Networks

In the last decade, wireless or wired sensor networks have attracted much attention. However, most designs target general sensor network issues including protocol stack (routing, MAC, etc.) and security issues. This book focuses on the close integration of sensing, networking, and smart signal processing via machine learning. Based on their world-class research, the authors present the fundamentals of intelligent sensor networks. They cover sensing and sampling, distributed signal processing, and intelligent signal learning. In addition, they present cutting-edge research results from leading experts

Enhanced Living Environments

This open access book was prepared as a Final Publication of the COST Action IC1303 “Algorithms, Architectures and Platforms for Enhanced Living Environments (AAPELE)”. The concept of Enhanced Living Environments (ELE) refers to the area of Ambient Assisted Living (AAL) that is more related with Information and Communication Technologies (ICT). Effective ELE solutions require appropriate ICT algorithms, architectures, platforms, and systems, having in view the advance of science and technology in this area and the development of new and innovative solutions that can provide improvements in the quality of life for people in their homes and can reduce the financial burden on the budgets of the healthcare providers. The aim of this book is to become a state-of-the-art reference, discussing progress made, as well as prompting future directions on theories, practices, standards, and strategies related to the ELE area. The book contains 12 chapters and can serve as a valuable reference for undergraduate students, post-graduate students, educators, faculty members, researchers, engineers, medical doctors, healthcare organizations, insurance companies, and research strategists working in this area

Wireless multimedia sensor networks, security and key management

Wireless Multimedia Sensor Networks (WMSNs) have emerged and shifted the focus from the typical scalar wireless sensor networks to networks with multimedia devices that are capable to retrieve video, audio, images, as well as scalar sensor data. WMSNs are able to deliver multimedia content due to the availability of inexpensive CMOS cameras and microphones coupled with the significant progress in distributed signal processing and multimedia source coding techniques. These mentioned characteristics, challenges, and requirements of designing WMSNs open many research issues and future research directions to develop protocols, algorithms, architectures, devices, and testbeds to maximize the network lifetime while satisfying the quality of service requirements of the various applications. In this thesis dissertation, we outline the design challenges of WMSNs and we give a comprehensive discussion of the proposed architectures and protocols for the different layers of the communication protocol stack for WMSNs along with their open research issues. Also, we conduct a comparison among the existing WMSN hardware and testbeds based on their specifications and features along with complete classification based on their functionalities and capabilities. In addition, we introduce our complete classification for content security and contextual privacy in WSNs. Our focus in this field, after conducting a complete survey in WMSNs and event privacy in sensor networks, and earning the necessary knowledge of programming sensor motes such as Micaz and Stargate and running simulation using NS2, is to design suitable protocols meet the challenging requirements of WMSNs targeting especially the routing and MAC layers, secure the wirelessly exchange of data against external attacks using proper security algorithms: key management and secure routing, defend the network from internal attacks by using a light-weight intrusion detection technique, protect the contextual information from being leaked to unauthorized parties by adapting an event unobservability scheme, and evaluate the performance efficiency and energy consumption of employing the security algorithms over WMSNs

Review of Particle Physics

The Review summarizes much of particle physics and cosmology. Using data from previous editions, plus 2,143 new measurements from 709 papers, we list, evaluate, and average measured properties of gauge bosons and the recently discovered Higgs boson, leptons, quarks, mesons, and baryons. We summarize searches for hypothetical particles such as supersymmetric particles, heavy bosons, axions, dark photons, etc. Particle properties and search limits are listed in Summary Tables. We give numerous tables, figures, formulae, and reviews of topics such as Higgs Boson Physics, Supersymmetry, Grand Unified Theories, Neutrino Mixing, Dark Energy, Dark Matter, Cosmology, Particle Detectors, Colliders, Probability and Statistics. Among the 120 reviews are many that are new or heavily revised, including a new review on Machine Learning, and one on Spectroscopy of Light Meson Resonances. The Review is divided into two volumes. Volume 1 includes the Summary Tables and 97 review articles. Volume 2 consists of the Particle Listings and contains also 23 reviews that address specific aspects of the data presented in the Listings