Machine learning systems in the IoT: Trustworthiness trade-offs for edge intelligence

Machine learning systems (MLSys) are emerging in the Internet of Things (IoT) to provision edge intelligence, which is paving our way towards the vision of ubiquitous intelligence. However, despite the maturity of machine learning systems and the IoT, we are facing severe challenges when integrating MLSys and IoT in practical context. For instance, many machine learning systems have been developed for large-scale production (e.g., cloud environments), but IoT introduces additional demands due to heterogeneous and resource-constrained devices and decentralized operation environment. To shed light on this convergence of MLSys and IoT, this paper analyzes the tradeoffs by covering the latest developments (up to 2020) on scaling and distributing ML across cloud, edge, and IoT devices. We position machine learning systems as a component of the IoT, and edge intelligence as a socio-technical system. On the challenges of designing trustworthy edge intelligence, we advocate a holistic design approach that takes multi-stakeholder concerns, design requirements and trade-offs into consideration, and highlight the future research opportunities in edge intelligence.Information and Communication Technolog

Bias in Automated Speaker Recognition

Automated speaker recognition uses data processing to identify speakers by their voice. Today, automated speaker recognition is deployed on billions of smart devices and in services such as call centres. Despite their wide-scale deployment and known sources of bias in related domains like face recognition and natural language processing, bias in automated speaker recognition has not been studied systematically. We present an in-depth empirical and analytical study of bias in the machine learning development workflow of speaker verification, a voice biometric and core task in automated speaker recognition. Drawing on an established framework for understanding sources of harm in machine learning, we show that bias exists at every development stage in the well-known VoxCeleb Speaker Recognition Challenge, including data generation, model building, and implementation. Most affected are female speakers and non-US nationalities, who experience significant performance degradation. Leveraging the insights from our findings, we make practical recommendations for mitigating bias in automated speaker recognition, and outline future research directions.Information and Communication Technolog

Energy-efficient Edge Approximation for Connected Vehicular Services

Connected vehicular services depend heavily on communication as they frequently transmit data and AI models/weights within the vehicular ecosystem. Energy efficiency in vehicles is crucial to keep up with the fast-growing demand for vehicular data processing and communication. To tackle this rising challenge, we explore approximation and edge AI techniques for achieving energy efficiency for vehicular services. Focusing on data-intensive vehicular services, we present an experimental case study on the high-definition (HD) map using the model partition approach. Our study compares the AI model energy consumption using multiple approximation ratios over embedded edge devices. Based on experimental insights, we further discuss an envisioned approximate Edge AI pipeline for developing and deploying energy-efficient vehicular services.Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.Information and Communication Technolog

Towards Trustworthy Edge Intelligence: Insights from Voice-Activated Services

In an age of surveillance capitalism, anchoring the design of emerging smart services in trustworthiness is urgent and important. Edge Intelligence, which brings together the fields of AI and Edge computing, is a key enabling technology for smart services. Trustworthy Edge Intelligence should thus be a priority research concern. However, determining what makes Edge Intelligence trustworthy is not straight forward. This paper examines requirements for trustworthy Edge Intelligence in a concrete application scenario of voice-activated services. We contribute to deepening the understanding of trustworthiness in the emerging Edge Intelligence domain in three ways: firstly, we propose a unified framing for trustworthy Edge Intelligence that jointly considers trustworthiness attributes of AI and the IoT. Secondly, we present research outputs of a tangible case study in voice-activated services that demonstrates interdependencies between three important trustworthiness attributes: privacy, security and fairness. Thirdly, based on the empirical and analytical findings, we highlight challenges and open questions that present important future research areas for trustworthy Edge Intelligence.Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.Information and Communication Technolog

The road towards private proximity services

Towards private proximity services we realized a set of proximity services at different spatial resolutions. For small-scale (0.5 m) securing remote access to smart homes and for mid-scale (10-20 m) to manage nearby Internet of Things (IoT) devices and offer fine-grained service discovery in indoor environments. Regarding large-scale services (100 m) we implemented a device grouping via similarity of light patterns ambient sound Wi-Fi signals and ultrasound communication which is naturally restricted by spatial barriers. To improve user's privacy from a system point of view we analyzed different security mechanisms in the domain of device-to-device (D2D) communication such as access control location privacy. Based on visible light communication (VLC) we are implemented and tested a system for private indoor service discovery and distance-bounding authorization. Furthermore we examined the feasibility of homomorphic encryption for time-series data like visible light patterns.Information and Communication Technolog

Edge chaining framework for black ice road fingerprinting

Detecting and reacting efficiently to road condition hazards are challenging given practical restrictions such as limited data availability and lack of infrastructure support. In this paper, we present an edge-cloud chaining solution that bridges the cloud and road infrastructures to enhance road safety. We exploit the roadside infrastructure (e.g., smart lampposts) to form a processing chain at the edge nodes and transmit the essential context to approaching vehicles providing what we refer as road fingerprinting. We approach the problem from two angles: first we focus on semantically defining how an execution pipeline spanning edge and cloud is composed, then we design, implement and evaluate a working prototype based on our assumptions. In addition, we present experimental insights and outline open challenges for next steps.Information and Communication Technolog

LocalVLC: Augmenting smart IoT services with practical visible light communication

Visible Light Communication (VLC)emerges as a communication technology for Internet of Things (IoT)services with appealing benefits not present in existing radio-based communication. However, current VLC designs commonly require dedicated LED lights to emit modulated light beams which entail high energy overhead and unpleasant visual experiences due to the perceptible light blinking effects for end users. This greatly limits the deployment and applicable scenarios of VLC. In this paper, we design and develop LocalVLC, a practical and low-cost VLC system that can be used as a standard light source to augment smart IoT services. LocalVLC introduces a novel Morse-code inspired modulation scheme that can operate on off-the-shelf LEDs with low energy overhead. It can effectively overcome the light flickering by encoding data into high frequency light pulses without requiring extra processing hardware such as FPGA or micro-controller. We have implemented and evaluated a full-fledged system prototype based on LocalVLC design. Under practical settings, our LocalVLC prototype can support up to 10 meters of range, and attain reasonable throughput (up to 1.4 Kbps)with low error rate and energy consumption. Comparing with the widely adopted Manchester encoding, LocalVLC yields 8x improvement on both throughput and energy consumption. In addition, we demonstrate the practicality of LocalVLC through two IoT use cases where we developed two lightweight LocalVLC-based solutions using low-cost off-the-shelf hardware to exemplify the usage of LocalVLC for indoor service discovery and smart home key management.Information and Communication Technolog

The Virtual Factory: Hologram-Enabled Control and Monitoring of Industrial IoT Devices

Augmented reality has been exploited in manifold fields but is yet to be used at its full potential. With the massive diffusion of smart devices, opportunities to build immersive human-computer interfaces are continually expanding. In this study, we conceptualize a virtual factory: an interactive, dynamic, holographic abstraction of the physical machines deployed in a factory. Through our prototype implementation, we conducted a user-study driven evaluation of holographic interfaces compared to traditional interfaces, highlighting its pros and cons. Our study shows that the majority of the participants found holographic manipulation more attractive and natural to interact with. However, current performance characteristics of head-mounted displays must be improved to be applied in production.Information and Communication Technolog

Trustworthy and Sustainable Edge AI: A Research Agenda

As a fast evolving domain that merges edge computing, data analytics and AI/ML, commonly referred as Edge AI, the community of Edge AI is establishing and gradually finds its way to connect with mainstream research communities of distributed systems, IoT, and embedded machine learning. Meanwhile, despite of its well-claimed potential to transform cloud and IoT industry, Edge AI is still a complex subject that faces critical challenges from the trustworthy and sustainable concerns. To shed light on these pressing matters, this paper aims to develop a research agenda for trustworthy and sustainable Edge AI. We clarify the concepts, define the proper scoping and propose a research agenda for Edge AI to be trustworthy and sustainable. To illustrate the research agenda in practice, we highlight two active RD projects: the SPATIAL project on trustworthy Edge AI and the APROPOS project on sustainable computing. The projects serve as concrete use cases to explore the agenda development. Our goal is to equip researchers, engineers, service providers, government and public sectors with a better understanding of the underlying concepts and to raise awareness of emerging directions in trustworthy and sustainable Edge AI.Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.Information and Communication Technolog

Multimodal Co-Presence Detection with Varying Spatio-Temporal Granularity

Pervasive computing environments are characterized by a plethora of sensing and communication-enabled devices that diffuse themselves among different users. Built-in sensors and telecommunication infrastructure allow co-presence detection. In turn, co-presence detection enables context-aware applications, like those for social networking among close-by users, and for modeling human behavior. We aim to support developers building better context-aware applications by a deepened understanding of which set of context information is appropriate for co-presence detection. We have gathered a multimodal dataset for proximity sensing, including several proximity verification sets, like Bluetooth, Wi-Fi, and GSM encounters, to be able to associate sensor's data with a spatial granularity. We show that sensor modalities are suitable to recognize the spatial adjacency of users with different spatio-temporal granularity. We find that individual user mobility has only a minor, negligible effect on co-presence detection. In contrast, the heterogeneity of device's sensor hardware has a major negative impact on co-presence detection. To reveal energy pitfalls with respect to usability, we perform an energy analysis pertaining to the usage stemming from different sensors for co-presence detection. Accepted Author ManuscriptInformation and Communication Technolog