Agency in the Internet of Things

This report summarises and extends the work done for the task force on IoT terminated in 2012. In response to DG CNECT request, the JRC studied this emergent technology following the methodologies pertaining to the Science and Technology Studies field. The aim of this document is therefore to present and to explore, on the basis of present day conceptions of relevant values, rights and norms, some of the “ethical issues” arising from the research, development and deployment of IoT, focusing on agency, autonomy and social justice. We start by exploring the types of imaginaries that seem to be entrenched and inspiring the developments of IoT and how they become portrayed in “normal” communication from corporations and promoters to the ordinary citizen (chapter 2). We report the empirical work we have conducted, namely the JRC contribution to the limited public debate initiated by the European Commission via the Your Voice portal during the Spring of 2012 (chapter 3) and an empirical exercise involving participants of two IoT conferences (chapter 4). This latter exercise sought to illustrate how our notions of goodness, trust, relationships, agency and autonomy are negotiated through the appropriation of unnoticed ordinary objects; this contributes to the discussion about ethical issues at stake with the emerging IoT vision beyond the right to privacy, data protection and security. Furthermore, based on literature review the report reflects on two of the main ethical issues that arise with the IoT vision: agency (and autonomy) and social justice (chapter 5), examining eventually governance alternatives of the challenged ethical issues (chapter 6).JRC.G.7-Digital Citizen Securit

Perspectives and approaches for the internet of things

Dissertação para obtenção do Grau de Mestre em Engenharia Electrotécnica e de ComputadoresThis thesis was developed based on a scenario in which a CEO of a certain company asked the author to conduct an exploratory work evaluating the potential opportunities and limitations of this emerging area described as the future of the Internet, the Internet of Things (IoT). The objective is thus to provide the reader with a wide view of the vital points for the implementation and exploitation of the IoT, a technology that promises to deliver a new and wider range of applications to the society. In this subject there is a need to gather and organize information produced by several researchers and contributors. Due to the fact of being a new area and researchers work independently of each other, the work is scattered and inconsistencies can be found among different projects and publications. As such, in a first stage some definitions are provided and an attempt to clarify concepts is made. To support and emphasize the exponential growth of IoT, a brief historical overview is provided to the reader. This overview is based on the new trends and expectations that arise every day through news, potential businesses and also in important tools such as Google Trends. Several examples of applications in the context of the IoT, illustrate the benefits, not only in terms of society, but also for business opportunities, safety, and well-being. The main areas of interest to achieve the IoT such as: hardware, software, modeling, methods of connection, security and integration are studied in this work, in order to provide some insight into current strong and weak points. As the Internet of Things become a matter of large interest, various research groups are active in exploring and organizing projects in this area. Some of these projects, namely the ones considered the most important, are also presented in this thesis. Taking into account the facts surrounding this new technology, it becomes quite important to bring them together, clarifying them and trying to open new perspectives for further studies and improvements. Finally, in order to allow a practical evaluation of the technology, a prototype is developed around the connection of an intelligent object – a small mobile robot – to the Internet. A set of conclusions and future work directions are then presented which take into account the findings of the bibliographic analysis as well as the acquired experience with the implementation of the prototype

Autonomous behaviour in tangible user interfaces as a design factor

PhD ThesisThis thesis critically explores the design space of autonomous and actuated artefacts, considering how autonomous behaviours in interactive technologies might shape and influence users’ interactions and behaviours. Since the invention of gearing and clockwork, mechanical devices were built that both fascinate and intrigue people through their mechanical actuation. There seems to be something magical about moving devices, which draws our attention and piques our interest. Progress in the development of computational hardware is allowing increasingly complex commercial products to be available to broad consumer-markets. New technologies emerge very fast, ranging from personal devices with strong computational power to diverse user interfaces, like multi-touch surfaces or gestural input devices. Electronic systems are becoming smaller and smarter, as they comprise sensing, controlling and actuation. From this, new opportunities arise in integrating more sensors and technology in physical objects. These trends raise some specific questions around the impacts smarter systems might have on people and interaction: how do people perceive smart systems that are tangible and what implications does this perception have for user interface design? Which design opportunities are opened up through smart systems? There is a tendency in humans to attribute life-like qualities onto non-animate objects, which evokes social behaviour towards technology. Maybe it would be possible to build user interfaces that utilise such behaviours to motivate people towards frequent use, or even motivate them to build relationships in which the users care for their devices. Their aim is not to increase the efficiency of user interfaces, but to create interfaces that are more engaging to interact with and excite people to bond with these tangible objects. This thesis sets out to explore autonomous behaviours in physical interfaces. More specifically, I am interested in the factors that make a user interpret an interface as autonomous. Through a review of literature concerned with animated objects, autonomous technology and robots, I have mapped out a design space exploring the factors that are important in developing autonomous interfaces. Building on this and utilising workshops conducted with other researchers, I have vi developed a framework that identifies key elements for the design of Tangible Autonomous Interfaces (TAIs). To validate the dimensions of this framework and to further unpack the impacts on users of interacting with autonomous interfaces I have adopted a ‘research through design’ approach. I have iteratively designed and realised a series of autonomous, interactive prototypes, which demonstrate the potential of such interfaces to establish themselves as social entities. Through two deeper case studies, consisting of an actuated helium balloon and desktop lamp, I provide insights into how autonomy could be implemented into Tangible User Interfaces. My studies revealed that through their autonomous behaviour (guided by the framework) these devices established themselves, in interaction, as social entities. They furthermore turned out to be acceptable, especially if people were able to find a purpose for them in their lives. This thesis closes with a discussion of findings and provides specific implications for design of autonomous behaviour in interfaces

Un Intergiciel de Gestion du Contexte basé Multi-Agent pour les Applications d'Intelligence Ambiante

The complexity and magnitude of Ambient Intelligence scenarios imply that attributes such as modeling expressiveness, flexibility of representation and deployment, as well as ease of configuration and development become central features for context management systems.However, existing works in the literature seem to explore these development-oriented attributes at a low degree.Our goal is to create a flexible and well configurable context management middleware, able to respond to different scenarios. To this end, our solution is built on the basis of principles and techniques of the Semantic Web and Multi-Agent Systems.We use the Semantic Web to provide a new context meta-model, allowing for an expressive and extensible modeling of content, meta-properties (e.g. temporal validity, quality parameters) and dependencies (e.g. integrity constraints).In addition, we develop a middleware architecture that relies on Multi-Agent Systems and a service component based design. Each agent of the system encapsulates a functional aspect of the context provisioning processes (acquisition, coordination, distribution, use).We introduce a new way to structure the deployment of agents depending on the multi-dimensionality aspects of the application's context model. Furthermore, we develop declarative policies governing the adaptation behavior of the agents managing the provisioning of context information.Simulations of an intelligent university scenario show that appropriate tooling built around our middleware can provide significant advantages in the engineering of context-aware applications.La complexité et l'ampleur des scénarios de l'Intelligence Ambiante impliquent que des attributs tels que l'expressivité de modelisation, la flexibilité de representation et de deploiement et la facilité de configuration et de developpement deviennent des caracteristiques centrales pour les systèmes de gestion de contexte. Cependant, les ouvrages existants semblent explorer ces attributs orientés-developpement a un faible degré.Notre objectif est de créer un intergiciel de gestion de contexte flexible et bien configurable, capable de répondre aux différents scenarios. A cette fin, notre solution est construite a base de techniques et principes du Web Semantique (WS) et des systèmes multi-agents (SMA).Nous utilisons le WS pour proposer un noveau meta-modèle de contexte, permettant une modelisation expressive et extensible du contenu, des meta-proprietés (e.g. validité temporelle, parametres de qualité) et des dépendances (e.g. les contraintes d'integrité) du contexte.De plus, une architecture a base de SMA et des composants logiciels, ou chaque agent encapsule un aspect fonctionnel du processus de gestion de contexte (acquisition, coordination, diffusion, utilisation) est developpée.Nous introduisons un nouveau moyen de structurer le deploiement d'agents selon les dimensions du modèle de contexte de l'application et nous elaborons des politiques déclaratives gouvernant le comportement d'adaptation du provisionnement contextuel des agents. Des simulations d'un scenario d'université intelligente montrent que un bon outillage construit autour de notre intergiciel peut apporter des avantages significatifs dans la génie des applications sensibles au contexte

Designing Sound for Social Robots: Advancing Professional Practice through Design Principles

Sound is one of the core modalities social robots can use to communicate with the humans around them in rich, engaging, and effective ways. While a robot's auditory communication happens predominantly through speech, a growing body of work demonstrates the various ways non-verbal robot sound can affect humans, and researchers have begun to formulate design recommendations that encourage using the medium to its full potential. However, formal strategies for successful robot sound design have so far not emerged, current frameworks and principles are largely untested and no effort has been made to survey creative robot sound design practice. In this dissertation, I combine creative practice, expert interviews, and human-robot interaction studies to advance our understanding of how designers can best ideate, create, and implement robot sound. In a first step, I map out a design space that combines established sound design frameworks with insights from interviews with robot sound design experts. I then systematically traverse this space across three robot sound design explorations, investigating (i) the effect of artificial movement sound on how robots are perceived, (ii) the benefits of applying compositional theory to robot sound design, and (iii) the role and potential of spatially distributed robot sound. Finally, I implement the designs from prior chapters into humanoid robot Diamandini, and deploy it as a case study. Based on a synthesis of the data collection and design practice conducted across the thesis, I argue that the creation of robot sound is best guided by four design perspectives: fiction (sound as a means to convey a narrative), composition (sound as its own separate listening experience), plasticity (sound as something that can vary and adapt over time), and space (spatial distribution of sound as a separate communication channel). The conclusion of the thesis presents these four perspectives and proposes eleven design principles across them which are supported by detailed examples. This work contributes an extensive body of design principles, process models, and techniques providing researchers and designers with new tools to enrich the way robots communicate with humans

Pathway to Future Symbiotic Creativity

This report presents a comprehensive view of our vision on the development path of the human-machine symbiotic art creation. We propose a classification of the creative system with a hierarchy of 5 classes, showing the pathway of creativity evolving from a mimic-human artist (Turing Artists) to a Machine artist in its own right. We begin with an overview of the limitations of the Turing Artists then focus on the top two-level systems, Machine Artists, emphasizing machine-human communication in art creation. In art creation, it is necessary for machines to understand humans' mental states, including desires, appreciation, and emotions, humans also need to understand machines' creative capabilities and limitations. The rapid development of immersive environment and further evolution into the new concept of metaverse enable symbiotic art creation through unprecedented flexibility of bi-directional communication between artists and art manifestation environments. By examining the latest sensor and XR technologies, we illustrate the novel way for art data collection to constitute the base of a new form of human-machine bidirectional communication and understanding in art creation. Based on such communication and understanding mechanisms, we propose a novel framework for building future Machine artists, which comes with the philosophy that a human-compatible AI system should be based on the "human-in-the-loop" principle rather than the traditional "end-to-end" dogma. By proposing a new form of inverse reinforcement learning model, we outline the platform design of machine artists, demonstrate its functions and showcase some examples of technologies we have developed. We also provide a systematic exposition of the ecosystem for AI-based symbiotic art form and community with an economic model built on NFT technology. Ethical issues for the development of machine artists are also discussed

Advances in Technology Enhanced Learning

‘Advances in Technology Enhanced Learning’ presents a range of research projects which aim to explore how to make engagement in learning (and teaching) more passionate. This interactive and experimental resource discusses innovations which pave the way to open collaboration at scale. The book introduces methodological and technological breakthroughs via twelve chapters to learners, instructors, and decision-makers in schools, universities, and workplaces. The Open University's Knowledge Media Institute and the EU TELMap project have brought together the luminaries from the European research area to showcase their vision of the future of learning with technology via their recent research project work. The projects discussed range widely over the Technology Enhanced Learning area from: environments for responsive open learning, work-based reflection, work-based social creativity, serious games and many more

Ubiquitous Robotics System for Knowledge-based Auto-configuration System for Service Delivery within Smart Home Environments

The future smart home will be enhanced and driven by the recent advance of the Internet of Things (IoT), which advocates the integration of computational devices within an Internet architecture on a global scale [1, 2]. In the IoT paradigm, the smart home will be developed by interconnecting a plethora of smart objects both inside and outside the home environment [3-5]. The recent take-up of these connected devices within home environments is slowly and surely transforming traditional home living environments. Such connected and integrated home environments lead to the concept of the smart home, which has attracted significant research efforts to enhance the functionality of home environments with a wide range of novel services. The wide availability of services and devices within contemporary smart home environments make their management a challenging and rewarding task. The trend whereby the development of smart home services is decoupled from that of smart home devices increases the complexity of this task. As such, it is desirable that smart home services are developed and deployed independently, rather than pre-bundled with specific devices, although it must be recognised that this is not always practical. Moreover, systems need to facilitate the deployment process and cope with any changes in the target environment after deployment. Maintaining complex smart home systems throughout their lifecycle entails considerable resources and effort. These challenges have stimulated the need for dynamic auto-configurable services amongst such distributed systems. Although significant research has been directed towards achieving auto-configuration, none of the existing solutions is sufficient to achieve auto-configuration within smart home environments. All such solutions are considered incomplete, as they lack the ability to meet all smart home requirements efficiently. These requirements include the ability to adapt flexibly to new and dynamic home environments without direct user intervention. Fulfilling these requirements would enhance the performance of smart home systems and help to address cost-effectiveness, considering the financial implications of the manual configuration of smart home environments. Current configuration approaches fail to meet one or more of the requirements of smart homes. If one of these approaches meets the flexibility criterion, the configuration is either not executed online without affecting the system or requires direct user intervention. In other words, there is no adequate solution to allow smart home systems to adapt dynamically to changing circumstances, hence to enable the correct interconnections among its components without direct user intervention and the interruption of the whole system. Therefore, it is necessary to develop an efficient, adaptive, agile and flexible system that adapts dynamically to each new requirement of the smart home environment. This research aims to devise methods to automate the activities associated with customised service delivery for dynamic home environments by exploiting recent advances in the field of ubiquitous robotics and Semantic Web technologies. It introduces a novel approach called the Knowledge-based Auto-configuration Software Robot (Sobot) for Smart Home Environments, which utilises the Sobot to achieve auto-configuration of the system. The research work was conducted under the Distributed Integrated Care Services and Systems (iCARE) project, which was designed to accomplish and deliver integrated distributed ecosystems with a homecare focus. The auto-configuration Sobot which is the focus of this thesis is a key component of the iCARE project. It will become one of the key enabling technologies for generic smart home environments. It has a profound impact on designing and implementing a high quality system. Its main role is to generate a feasible configuration that meets the given requirements using the knowledgebase of the smart home environment as a core component. The knowledgebase plays a pivotal role in helping the Sobot to automatically select the most appropriate resources in a given context-aware system via semantic searching and matching. Ontology as a technique of knowledgebase representation generally helps to design and develop a specific domain. It is also a key technology for the Semantic Web, which enables a common understanding amongst software agents and people, clarifies the domain assumptions and facilitates the reuse and analysis of its knowledge. The main advantages of the Sobot over traditional applications is its awareness of the changing digital and physical environments and its ability to interpret these changes, extract the relevant contextual data and merge any new information or knowledge. The Sobot is capable of creating new or alternative feasible configurations to meet the system’s goal by utilising inferred facts based on the smart home ontological model, so that the system can adapt to the changed environment. Furthermore, the Sobot has the capability to execute the generated reconfiguration plan without interrupting the running of the system. A proof-of-concept testbed has been designed and implemented. The case studies carried out have shown the potential of the proposed approach to achieve flexible and reliable auto-configuration of the smart home system, with promising directions for future research