Incremental Learning Using a Grow-and-Prune Paradigm with Efficient Neural Networks

Deep neural networks (DNNs) have become a widely deployed model for numerous machine learning applications. However, their fixed architecture, substantial training cost, and significant model redundancy make it difficult to efficiently update them to accommodate previously unseen data. To solve these problems, we propose an incremental learning framework based on a grow-and-prune neural network synthesis paradigm. When new data arrive, the neural network first grows new connections based on the gradients to increase the network capacity to accommodate new data. Then, the framework iteratively prunes away connections based on the magnitude of weights to enhance network compactness, and hence recover efficiency. Finally, the model rests at a lightweight DNN that is both ready for inference and suitable for future grow-and-prune updates. The proposed framework improves accuracy, shrinks network size, and significantly reduces the additional training cost for incoming data compared to conventional approaches, such as training from scratch and network fine-tuning. For the LeNet-300-100 and LeNet-5 neural network architectures derived for the MNIST dataset, the framework reduces training cost by up to 64% (63%) and 67% (63%) compared to training from scratch (network fine-tuning), respectively. For the ResNet-18 architecture derived for the ImageNet dataset and DeepSpeech2 for the AN4 dataset, the corresponding training cost reductions against training from scratch (network fine-tunning) are 64% (60%) and 67% (62%), respectively. Our derived models contain fewer network parameters but achieve higher accuracy relative to conventional baselines

Design of facial recognition system implemented in an unmanned aerial vehicle for citizen security in Latin America

In the new technological era, facial recognition has become a central issue for a great number of engineers. Currently, there are a great number of techniques for facial recognition, but in this research, we focus on the use of deep learning. The problems with current facial recognition convection systems are that they are developed in non-mobile devices. This research intends to develop a Facial Recognition System implemented in an unmanned aerial vehicle of the quadcopter type. While it is true, there are quadcopters capable of detecting faces and/or shapes and following them, but most are for fun and entertainment. This research focuses on the facial recognition of people with criminal records, for which a neural network is trained. The Caffe framework is used for the training of a convolutional neural network. The system is developed on the NVIDIA Jetson TX2 motherboard. The design and construction of the quadcopter are done from scratch because we need the UAV for adapt to our requirements. This research aims to reduce violence and crime in Latin America