Compact optimized deep learning model for edge: a review

Most real-time computer vision applications, such as pedestrian detection, augmented reality, and virtual reality, heavily rely on convolutional neural networks (CNN) for real-time decision support. In addition, edge intelligence is becoming necessary for low-latency real-time applications to process the data at the source device. Therefore, processing massive amounts of data impact memory footprint, prediction time, and energy consumption, essential performance metrics in machine learning based internet of things (IoT) edge clusters. However, deploying deeper, dense, and hefty weighted CNN models on resource-constraint embedded systems and limited edge computing resources, such as memory, and battery constraints, poses significant challenges in developing the compact optimized model. Reducing the energy consumption in edge IoT networks is possible by reducing the computation and data transmission between IoT devices and gateway devices. Hence there is a high demand for making energy-efficient deep learning models for deploying on edge devices. Furthermore, recent studies show that smaller compressed models achieve significant performance compared to larger deep-learning models. This review article focuses on state-of-the-art techniques of edge intelligence, and we propose a new research framework for designing a compact optimized deep learning (DL) model deployment on edge devices

Deep Learning For Resource Constraint Devices

The amount of Internet-of-things (IoT) devices is rapidly expanding. This also triggered the necessity of smart IoT devices which are capable of conducting any task by itself. Deep learning techniques are also booming due to the increased computing power and refined algorithms. The advantage of deep learning is that it can be tuned into any application without the manual feature extraction process. Now, the combination of deep learning with smart IoT devices/edge devices can result in any application that can be used in machine vision, vision inspection, autonomous vehicle, and many more. These applications can be automated which requires human operation. Now, combining deep learning and edge device together and running the application can be a difficult task. The main reason is that deep learning requires large computation power and edge devices does not have that capability. This study focused on this problem. Ie used techniques to encrypt and compress data which is essential for the edge devices. In addition, we developed novel methods to protect user privacy for data collection and learning on edge devices. Also, we conducted a study to evaluate different edge devices for different application purposes with certain compression technique of the models. Lastly, we conducted a real life experiment of collecting data, creating different models and evaluating it on different edge devices. index terms - IoT, computer vision, deep learning, machine learning, quantization, autoencoder, mobilenet v1, mobilenet v2, inception v3, face mask detectio

Exploiting Features and Logits in Heterogeneous Federated Learning

Due to the rapid growth of IoT and artificial intelligence, deploying neural networks on IoT devices is becoming increasingly crucial for edge intelligence. Federated learning (FL) facilitates the management of edge devices to collaboratively train a shared model while maintaining training data local and private. However, a general assumption in FL is that all edge devices are trained on the same machine learning model, which may be impractical considering diverse device capabilities. For instance, less capable devices may slow down the updating process because they struggle to handle large models appropriate for ordinary devices. In this paper, we propose a novel data-free FL method that supports heterogeneous client models by managing features and logits, called Felo; and its extension with a conditional VAE deployed in the server, called Velo. Felo averages the mid-level features and logits from the clients at the server based on their class labels to provide the average features and logits, which are utilized for further training the client models. Unlike Felo, the server has a conditional VAE in Velo, which is used for training mid-level features and generating synthetic features according to the labels. The clients optimize their models based on the synthetic features and the average logits. We conduct experiments on two datasets and show satisfactory performances of our methods compared with the state-of-the-art methods

Real-Time Streaming Analytics using Big Data Paradigm and Predictive Modelling based on Deep Learning

With the evolution of distributed streaming platforms analysing humongous time series data, which is streamed continuously from IoT devices become lot easier. In most of the IoT networks the data are in motion or in data centre/cloud. It is possible to process this data in real time similar to edge devices using the big data framework.  In data intensive applications predictive analytics require more resources to perform complex computations. Apache Flink framework is capable of performing real time streaming of schema less data and scales very high in distributed environment with low latency, it is used to collect and store the data in the cloud. This work suggests a suitable environment to collect, transport, preprocess and aggregate the data stream to perform predictive analytics using deep learning models. Deep learning automatically extracts features and builds models after training, it has the potential to solve problems that can't be solved by conventional machine learning models. Therefore, the use of algorithms based on deep learning is recommended for forecasting temporal data. Also, we discuss a number of different deep learning forecasting models and analyse the performance of different deep learning forecasting models in order to determine which one is the effective model for single step, multi step and multi variant methods based on error functions with respect to streamed sensor data

Industrial Cyber-Physical Systems-based Cloud IoT Edge for Federated Heterogeneous Distillation.

Deep convoloutional networks have achieved remarkable performance in a wide range of vision-based tasks in modern internet of things (IoT). Due to privacy issue and transmission cost, mannually annotated data for training the deep learning models are usually stored in different sites with fog and edge devices of various computing capacity. It has been proved that knowledge distillation technique can effectively compress well trained neural networks into light-weight models suitable to particular devices. However, different fog and edge devices may perform different sub-tasks, and simplely performing model compression on powerful cloud servers failed to make use of the private data sotred at different sites. To overcome these obstacles, we propose an novel knowledge distillation method for object recognition in real-world IoT sencarios. Our method enables flexible bidirectional online training of heterogeneous models distributed datasets with a new ``brain storming'' mechanism and optimizable temperature parameters. In our comparison experiments, this heterogeneous brain storming method were compared to multiple state-of-the-art single-model compression methods, as well as the newest heterogeneous and homogeneous multi-teacher knowledge distillation methods. Our methods outperformed the state of the arts in both conventional and heterogeneous tasks. Further analysis of the ablation expxeriment results shows that introducing the trainable temperature parameters into the conventional knowledge distillation loss can effectively ease the learning process of student networks in different methods. To the best of our knowledge, this is the IoT-oriented method that allows asynchronous bidirectional heterogeneous knowledge distillation in deep networks

NIPD: A Federated Learning Person Detection Benchmark Based on Real-World Non-IID Data

Federated learning (FL), a privacy-preserving distributed machine learning, has been rapidly applied in wireless communication networks. FL enables Internet of Things (IoT) clients to obtain well-trained models while preventing privacy leakage. Person detection can be deployed on edge devices with limited computing power if combined with FL to process the video data directly at the edge. However, due to the different hardware and deployment scenarios of different cameras, the data collected by the camera present non-independent and identically distributed (non-IID), and the global model derived from FL aggregation is less effective. Meanwhile, existing research lacks public data set for real-world FL object detection, which is not conducive to studying the non-IID problem on IoT cameras. Therefore, we open source a non-IID IoT person detection (NIPD) data set, which is collected from five different cameras. To our knowledge, this is the first true device-based non-IID person detection data set. Based on this data set, we explain how to establish a FL experimental platform and provide a benchmark for non-IID person detection. NIPD is expected to promote the application of FL and the security of smart city.Comment: 8 pages, 5 figures, 3 tables, FL-IJCAI 23 conferenc

Internet of robotic things : converging sensing/actuating, hypoconnectivity, artificial intelligence and IoT Platforms

The Internet of Things (IoT) concept is evolving rapidly and influencing newdevelopments in various application domains, such as the Internet of MobileThings (IoMT), Autonomous Internet of Things (A-IoT), Autonomous Systemof Things (ASoT), Internet of Autonomous Things (IoAT), Internetof Things Clouds (IoT-C) and the Internet of Robotic Things (IoRT) etc.that are progressing/advancing by using IoT technology. The IoT influencerepresents new development and deployment challenges in different areassuch as seamless platform integration, context based cognitive network integration,new mobile sensor/actuator network paradigms, things identification(addressing, naming in IoT) and dynamic things discoverability and manyothers. The IoRT represents new convergence challenges and their need to be addressed, in one side the programmability and the communication ofmultiple heterogeneous mobile/autonomous/robotic things for cooperating,their coordination, configuration, exchange of information, security, safetyand protection. Developments in IoT heterogeneous parallel processing/communication and dynamic systems based on parallelism and concurrencyrequire new ideas for integrating the intelligent “devices”, collaborativerobots (COBOTS), into IoT applications. Dynamic maintainability, selfhealing,self-repair of resources, changing resource state, (re-) configurationand context based IoT systems for service implementation and integrationwith IoT network service composition are of paramount importance whennew “cognitive devices” are becoming active participants in IoT applications.This chapter aims to be an overview of the IoRT concept, technologies,architectures and applications and to provide a comprehensive coverage offuture challenges, developments and applications