RL-IoT: Reinforcement Learning to Interact with IoT Devices

Our life is getting filled by Internet of Things (IoT) devices. These devices often rely on closed or poorly documented protocols, with unknown formats and semantics. Learning how to interact with such devices in an autonomous manner is the key for interoperability and automatic verification of their capabilities. In this paper, we propose RL-IoT, a system that explores how to automatically interact with possibly unknown IoT devices. We leverage reinforcement learning (RL) to recover the semantics of protocol messages and to take control of the device to reach a given goal, while minimizing the number of interactions. We assume to know only a database of possible IoT protocol messages, whose semantics are however unknown. RL-IoT exchanges messages with the target IoT device, learning those commands that are useful to reach the given goal. Our results show that RL-IoT is able to solve both simple and complex tasks. With properly tuned parameters, RL-IoT learns how to perform actions with the target device, a Yeelight smart bulb in our case study, completing non-trivial patterns with as few as 400 interactions. RL-IoT paves the road for automatic interactions with poorly documented IoT protocols, thus enabling interoperable systems

Internet of robotic things : converging sensing/actuating, hypoconnectivity, artificial intelligence and IoT Platforms

The Internet of Things (IoT) concept is evolving rapidly and influencing newdevelopments in various application domains, such as the Internet of MobileThings (IoMT), Autonomous Internet of Things (A-IoT), Autonomous Systemof Things (ASoT), Internet of Autonomous Things (IoAT), Internetof Things Clouds (IoT-C) and the Internet of Robotic Things (IoRT) etc.that are progressing/advancing by using IoT technology. The IoT influencerepresents new development and deployment challenges in different areassuch as seamless platform integration, context based cognitive network integration,new mobile sensor/actuator network paradigms, things identification(addressing, naming in IoT) and dynamic things discoverability and manyothers. The IoRT represents new convergence challenges and their need to be addressed, in one side the programmability and the communication ofmultiple heterogeneous mobile/autonomous/robotic things for cooperating,their coordination, configuration, exchange of information, security, safetyand protection. Developments in IoT heterogeneous parallel processing/communication and dynamic systems based on parallelism and concurrencyrequire new ideas for integrating the intelligent “devices”, collaborativerobots (COBOTS), into IoT applications. Dynamic maintainability, selfhealing,self-repair of resources, changing resource state, (re-) configurationand context based IoT systems for service implementation and integrationwith IoT network service composition are of paramount importance whennew “cognitive devices” are becoming active participants in IoT applications.This chapter aims to be an overview of the IoRT concept, technologies,architectures and applications and to provide a comprehensive coverage offuture challenges, developments and applications

Event processing in web of things

The incoming digital revolution has the potential to drastically improve our productivity, reduce operational costs and improve the quality of the products. However, the realization of these promises requires the convergence of technologies — from edge computing to cloud, artificial intelligence, and the Internet of Things — blurring the lines between the physical and digital worlds. Although these technologies evolved independently over time, they are increasingly becoming intertwined. Their convergence will create an unprecedented level of automation, achieved via massive machine-to-machine interactions whose cornerstone are event processing tasks. This thesis explores the intersection of these technologies by making an in-depth analysis of their role in the life-cycle of event processing tasks, including their creation, placement and execution. First, it surveys currently existing Web standards, Internet drafts, and design patterns that are used in the creation of cloud-based event processing. Then, it investigates the reasons for event processing to start shifting towards the edge, alongside with the standards that are necessary for a smooth transition to occur. Finally, this work proposes the use of deep reinforcement learning methods for the placement and distribution of event processing tasks at the edge. Obtained results show that the proposed neural-based event placement method is capable of obtaining (near) optimal solutions in several scenarios and provide hints about future research directions.A nova revolução digital promete melhorar drasticamente a nossa produtividade, reduzir os custos operacionais e melhorar a qualidade dos produtos. A concretizac¸ ˜ao dessas promessas requer a convergˆencia de tecnologias – desde edge computing à cloud, inteligência artificial e Internet das coisas (IoT) – atenuando a linha que separa o mundo físico do digital. Embora as quatro tecnologias mencionadas tenham evoluído de forma independente ao longo do tempo, atualmente elas estão cada vez mais interligadas. A convergência destas tecnologias irá criar um nível de automatização sem precedentes.The research published in this work was supported by the Portuguese Foundation for Science and Technology (FCT) through CEOT (Center for Electronic, Optoelectronic and Telecommunications) funding (UID/MULTI/00631/2020) and by FCT Ph.D grant to Andriy Mazayev (SFRH/BD/138836/2018)