Academic Panel: Can Self-Managed Systems be trusted?

Trust can be defined as to have confidence or faith in; a form of reliance or certainty based on past experience; to allow without fear; believe; hope: expect and wish; and extend credit to. The issue of trust in computing has always been a hot topic, especially notable with the proliferation of services over the Internet, which has brought the issue of trust and security right into the ordinary home. Autonomic computing brings its own complexity to this. With systems that self-manage, the internal decision making process is less transparent and the ‘intelligence’ possibly evolving and becoming less tractable. Such systems may be used from anything from environment monitoring to looking after Granny in the home and thus the issue of trust is imperative. To this end, we have organised this panel to examine some of the key aspects of trust. The first section discusses the issues of self-management when applied across organizational boundaries. The second section explores predictability in self-managed systems. The third part examines how trust is manifest in electronic service communities. The final discussion demonstrates how trust can be integrated into an autonomic system as the core intelligence with which to base adaptivity choices upon

Joint Channel Selection and Power Control in Infrastructureless Wireless Networks: A Multi-Player Multi-Armed Bandit Framework

This paper deals with the problem of efficient resource allocation in dynamic infrastructureless wireless networks. Assuming a reactive interference-limited scenario, each transmitter is allowed to select one frequency channel (from a common pool) together with a power level at each transmission trial; hence, for all transmitters, not only the fading gain, but also the number of interfering transmissions and their transmit powers are varying over time. Due to the absence of a central controller and time-varying network characteristics, it is highly inefficient for transmitters to acquire global channel and network knowledge. Therefore a reasonable assumption is that transmitters have no knowledge of fading gains, interference, and network topology. Each transmitting node selfishly aims at maximizing its average reward (or minimizing its average cost), which is a function of the action of that specific transmitter as well as those of all other transmitters. This scenario is modeled as a multi-player multi-armed adversarial bandit game, in which multiple players receive an a priori unknown reward with an arbitrarily time-varying distribution by sequentially pulling an arm, selected from a known and finite set of arms. Since players do not know the arm with the highest average reward in advance, they attempt to minimize their so-called regret, determined by the set of players' actions, while attempting to achieve equilibrium in some sense. To this end, we design in this paper two joint power level and channel selection strategies. We prove that the gap between the average reward achieved by our approaches and that based on the best fixed strategy converges to zero asymptotically. Moreover, the empirical joint frequencies of the game converge to the set of correlated equilibria. We further characterize this set for two special cases of our designed game

Dispatching Requests for Agent-Based Online Vehicle Routing Problems with Time Windows

Vehicle routing problems are highly complex problems. The proposals to solve them traditionally concern the optimization of conventional criteria, such as the number of mobilized vehicles and the total costs. However, in online vehicle routing problems, the optimization of the response time to the connected travelers is at least as important as the optimization of the classical criteria. Multi-agent systems on the one hand and greedy insertion heuristics on the other are among the most promising approaches to this end. In this paper, we propose a multi-agent system coupled with a regret insertion heuristic. We focus on the real-time dispatching of the travelers\u27 requests to the vehicles and its efficiency. A dispatching protocol determines which agents perform the computation to answer the travelers\u27 requests. We evaluate three dispatching protocols: centralized, decentralized and hybrid. We compare them experimentally based on their response time to online travelers. Two computational types are implemented: a sequential implementation and a distributed implementation. The results show the superiority of the centralized dispatching protocol in the sequential implementation (32.80% improvement in average compared to the distributed dispatching protocol) and the superiority of the hybrid dispatching protocol in the distributed implementation (59.66% improvement in average, compared with the centralized dispatching protocol)

Decentralized Multi-Agent Reinforcement Learning for Continuous-Space Stochastic Games

Stochastic games are a popular framework for studying multi-agent reinforcement learning (MARL). Recent advances in MARL have focused primarily on games with finitely many states. In this work, we study multi-agent learning in stochastic games with general state spaces and an information structure in which agents do not observe each other's actions. In this context, we propose a decentralized MARL algorithm and we prove the near-optimality of its policy updates. Furthermore, we study the global policy-updating dynamics for a general class of best-reply based algorithms and derive a closed-form characterization of convergence probabilities over the joint policy space