Towards Reinforcement Learning-based Aggregate Computing

Recent trends in pervasive computing promote the vision of Collective Adaptive Systems (CASs): large-scale collections of relatively simple agents that act and coordinate with no central orchestrator to support distributed applications. Engineering global behaviour out of local activity and interaction, however, is a difficult task, typically addressed by try-and-error approaches in simulation environments. In the context of Aggregate Computing (AC), a prominent functional programming approach for CASs based on field-based coordination, this difficulty is reflected in the design of versatile algorithms preserving efficiency in a variety of environments. To deal with this complexity, in this work we propose to apply Machine Learning techniques to automatically devise local actions to improve over manually-defined AC algorithms specifications. Most specifically, we adopt a Reinforcement Learning-based approach to let a collective learn local policies to improve over the standard gradient algorithm—a cornerstone brick of several higher-level self-organisation algorithms. Our evaluation shows that the learned policies can speed up the self-stabilisation of the gradient to external perturbations

Simulating social relations in multi-agent systems

Open distributed systems are comprised of a large number of heterogeneous nodes with disparate requirements and objectives, a number of which may not conform to the system specification. This thesis argues that activity in such systems can be regulated by using distributed mechanisms inspired by social science theories regarding similarity /kinship, trust, reputation, recommendation and economics. This makes it possible to create scalable and robust agent societies which can adapt to overcome structural impediments and provide inherent defence against malicious and incompetent action, without detriment to system functionality and performance. In particular this thesis describes: • an agent based simulation and animation platform (PreSage), which offers the agent developer and society designer a suite of powerful tools for creating, simulating and visualising agent societies from both a local and global perspective. • a social information dissemination system (SID) based on principles of self organisation which personalises recommendation and directs information dissemination. • a computational socio-cognitive and economic framework (CScEF) which integrates and extends socio-cognitive theories of trust, reputation and recommendation with basic economic theory. • results from two simulation studies investigating the performance of SID and the CScEF. The results show the production of a generic, reusable and scalable platform for developing and animating agent societies, and its contribution to the community as an open source tool. Secondly specific results, regarding the application of SID and CScEF, show that revealing outcomes of using socio-technical mechanisms to condition agent interactions can be demonstrated and identified by using Presage.Open Acces

Organic Service-Level Management in Service-Oriented Environments

Dynamic service-oriented environments (SOEs) are characterised by a large number of heterogeneous service components that are expected to support the business as a whole. The present work provides a negotiation-based approach to facilitate automated and multi-level service-level management in an SOE, where each component autonomously arranges its contribution to the whole operational goals. Evaluation experiments have shown an increased responsiveness and stability of an SOE in case of changes