4 research outputs found
Multi-agent Q-Learning of Channel Selection in Multi-user Cognitive Radio Systems: A Two by Two Case
Resource allocation is an important issue in cognitive radio systems. It can
be done by carrying out negotiation among secondary users. However, significant
overhead may be incurred by the negotiation since the negotiation needs to be
done frequently due to the rapid change of primary users' activity. In this
paper, a channel selection scheme without negotiation is considered for
multi-user and multi-channel cognitive radio systems. To avoid collision
incurred by non-coordination, each user secondary learns how to select channels
according to its experience. Multi-agent reinforcement leaning (MARL) is
applied in the framework of Q-learning by considering the opponent secondary
users as a part of the environment. The dynamics of the Q-learning are
illustrated using Metrick-Polak plot. A rigorous proof of the convergence of
Q-learning is provided via the similarity between the Q-learning and
Robinson-Monro algorithm, as well as the analysis of convergence of the
corresponding ordinary differential equation (via Lyapunov function). Examples
are illustrated and the performance of learning is evaluated by numerical
simulations.Comment: submitted to 2009 IEEE International Conference on Systems, Man, and
Cybernetics; the results of general n by m case will be published soo
An agent based architecture for cognitive spectrum management
In the recent years, wireless technologies and devices have progressed dramatically that has augmented the demand for electromagnetic spectrum. Some research work showed that spectrum access and provision to user is not possible due to shortage of spectrum but federal communication commission refused to accept this theory and indicated that the spectrum is available since most of the frequency bands are underutilized. In order to allow the use of these frequency bands without interference, cognitive radio was proposed that characterizes the growing intelligence of radio systems can adapt to the radio environment, allowing opportunistic usage and sharing with the existing uses of spectrum. To take this concept a step further, we propose to use intelligent agent for spectrum management in the context of cognitive radio in this paper. In our proposed architecture, agents are embedded in the radio devices that coordinate their operations to benefit from network and avoid interference with the primary user. Agents carry a set of modules to gather information about the terminal status and the radio environment and act accordingly to the constraints of the user application