IoT meets distributed AI ::deployment scenarios of Bonseyes AI applications on FIWARE

Bonseyes is an Artificial Intelligence (AI) platform composed of a Data Marketplace, a Deep Learning Toolbox, and Developer Reference Platforms with the aim of facilitating tech and non-tech companies a rapid adoption of AI as an enabler for their business. Bonseyes provides methods and tools to speed up the development and deployment of AI solutions on low power Internet of Things (IoT) devices, embedded computing systems, and data centre servers. In this work, we address the deployment and the integration of Bonseyes AI applications in a wider enterprise application landscape involving different applications and services. We leverage the well-established IoT platform FIWARE to integrate the Bonseyes AI applications into an enterprise ecosystem. This paper presents two AI application deployment and integration scenarios using FIWARE. The first scenario addresses use cases where edge devices have enough compute power to run the AI applications and there is only need to transmit the results to the enterprise ecosystem. The second scenario copes with use cases where an edge device may delegate most of the computation to an external/cloud server. Further, we employ FIWARE IoT Agent generic enabler to manage all edge devices related to Bonseyes AI applications. Both scenarios have been validated on concrete use cases and demonstrators

Learning to infer: RL-based search for DNN primitive selection on Heterogeneous Embedded Systems

Deep Learning is increasingly being adopted by industry for computer vision applications running on embedded devices. While Convolutional Neural Networks' accuracy has achieved a mature and remarkable state, inference latency and throughput are a major concern especially when targeting low-cost and low-power embedded platforms. CNNs' inference latency may become a bottleneck for Deep Learning adoption by industry, as it is a crucial specification for many real-time processes. Furthermore, deployment of CNNs across heterogeneous platforms presents major compatibility issues due to vendor-specific technology and acceleration libraries.In this work, we present QS-DNN, a fully automatic search based on Reinforcement Learning which, combined with an inference engine optimizer, efficiently explores through the design space and empirically finds the optimal combinations of libraries and primitives to speed up the inference of CNNs on heterogeneous embedded devices. We show that, an optimized combination can achieve 45x speedup in inference latency on CPU compared to a dependency-free baseline and 2x on average on GPGPU compared to the best vendor library. Further, we demonstrate that, the quality of results and time "to-solution" is much better than with Random Search and achieves up to 15x better results for a short-time search

IoT-based dynamic street light control for smart cities use cases

This paper presents a real-world proven solution for dynamic street light control and management which relies on an open and flexible Internet of Things architecture. Substantial contribution is brought at the interoperability level using novel device connection concept based on model-driven communication agents to speed up the integration of sensors and actuators to Internet of Things platforms. The paper shows also results from real-world tests with deployed dynamic street lights in urban spaces. The proposed dynamic light control solution permits an energy saving of about 56% compared to classical static, time-based street light control

Quantitative modeling and comparison of communication schemes to guarantee quality-of-service in networks-on-chip

Abstract — We address the quantitative comparison of two basic connection schemes for on-chip communication, connectionoriented and connection-less, regarding their enforcement of a defined Quality of Service for the communication. For such comparison, we have built similar models for the two approaches, whose simulation highlights the drawbacks and opportunities of each solution. We demonstrate that for variable bit-rate applications, the end-to-end delay for the individual flows is less stable in the connection-oriented scheme than in the connectionless scheme. I

QUENN ::QUantization engine for low-power neural networks

Deep Learning is moving to edge devices, ushering in a new age of distributed Artificial Intelligence (AI). The high demand of computational resources required by deep neural networks may be alleviated by approximate computing techniques, and most notably reduced-precision arithmetic with coarsely quantized numerical representations. In this context, Bonseyes comes in as an initiative to enable stakeholders to bring AI to low-power and autonomous environments such as: Automotive, Medical Healthcare and Consumer Electronics. To achieve this, we introduce LPDNN, a framework for optimized deployment of Deep Neural Networks on heterogeneous embedded devices. In this work, we detail the quantization engine that is integrated in LPDNN. The engine depends on a fine-grained workflow which enables a Neural Network Design Exploration and a sensitivity analysis of each layer for quantization. We demonstrate the engine with a case study on Alexnet and VGG16 for three different techniques for direct quantization: standard fixed-point, dynamic fixed-point and k-means clustering, and demonstrate the potential of the latter. We argue that using a Gaussian quantizer with k-means clustering can achieve better performance than linear quantizers. Without retraining, we achieve over 55.64% saving for weights' storage and 69.17% for run-time memory accesses with less than 1% drop in top5 accuracy in Imagenet

Dynamic street light management ::towards a citizen centred approach

This paper presents a novel approach towards dynamic street light control, which combines advanced Information and Communication Technologies (ICT) and citizens' involvement and engagement. Our proposal is based on the citizens' involvement which would strongly increases the efficiency and perfromance of technological solutions in smart city context. We believe that serious Games have the potential to strengthen people motivation in this context

BONSEYES ::platform for open development of systems of artificial intelligence

The Bonseyes EU H2020 collaborative project aims to develop a platform consisting of a Data Marketplace, a Deep Learning Toolbox, and Developer Reference Platforms for organizations wanting to adopt Artificial Intelligence. The project will be focused on using artificial intelligence in low power Internet of Things (IoT) devices ("edge computing"), embedded computing systems, and data center servers ("cloud computing"). It will bring about orders of magnitude improvements in efficiency, performance, reliability, security, and productivity in the design and programming of systems of artificial intelligence that incorporate Smart Cyber-Physical Systems (CPS). In addition, it will solve a causality problem for organizations who lack access to Data and Models. Its open software architecture will facilitate adoption of the whole concept on a wider scale. To evaluate the effectiveness, technical feasibility, and to quantify the real-world improvements in efficiency, security, performance, effort and cost of adding AI to products and services using the Bonseyes platform, four complementary demonstrators will be built. Bonseyes platform capabilities are aimed at being aligned with the European FI-PPP activities and take advantage of its flagship project FIWARE. This paper provides a description of the project motivation, goals and preliminary work