84,878 research outputs found
Situation-Dependent Causal Influence-Based Cooperative Multi-agent Reinforcement Learning
Learning to collaborate has witnessed significant progress in multi-agent
reinforcement learning (MARL). However, promoting coordination among agents and
enhancing exploration capabilities remain challenges. In multi-agent
environments, interactions between agents are limited in specific situations.
Effective collaboration between agents thus requires a nuanced understanding of
when and how agents' actions influence others. To this end, in this paper, we
propose a novel MARL algorithm named Situation-Dependent Causal Influence-Based
Cooperative Multi-agent Reinforcement Learning (SCIC), which incorporates a
novel Intrinsic reward mechanism based on a new cooperation criterion measured
by situation-dependent causal influence among agents. Our approach aims to
detect inter-agent causal influences in specific situations based on the
criterion using causal intervention and conditional mutual information. This
effectively assists agents in exploring states that can positively impact other
agents, thus promoting cooperation between agents. The resulting update links
coordinated exploration and intrinsic reward distribution, which enhance
overall collaboration and performance. Experimental results on various MARL
benchmarks demonstrate the superiority of our method compared to
state-of-the-art approaches
Modeling Mutual Influence in Multi-Agent Reinforcement Learning
In multi-agent systems (MAS), agents rarely act in isolation but tend to achieve their goals through interactions with other agents. To be able to achieve their ultimate goals, individual agents should actively evaluate the impacts on themselves of other agents' behaviors before they decide which actions to take. The impacts are reciprocal, and it is of great interest to model the mutual influence of agent's impacts with one another when they are observing the environment or taking actions in the environment. In this thesis, assuming that the agents are aware of each other's existence and their potential impact on themselves, I develop novel multi-agent reinforcement learning (MARL) methods that can measure the mutual influence between agents to shape learning. The first part of this thesis outlines the framework of recursive reasoning in deep multi-agent reinforcement learning. I hypothesize that it is beneficial for each agent to consider how other agents react to their behavior. I start from Probabilistic Recursive Reasoning (PR2) using level-1 reasoning and adopt variational Bayes methods to approximate the opponents' conditional policies. Each agent shapes the individual Q-value by marginalizing the conditional policies in the joint Q-value and finding the best response to improving their policies. I further extend PR2 to Generalized Recursive Reasoning (GR2) with different hierarchical levels of rationality. GR2 enables agents to possess various levels of thinking ability, thereby allowing higher-level agents to best respond to less sophisticated learners. The first part of the thesis shows that eliminating the joint Q-value to an individual Q-value via explicitly recursive reasoning would benefit the learning. In the second part of the thesis, in reverse, I measure the mutual influence by approximating the joint Q-value based on the individual Q-values. I establish Q-DPP, an extension of the Determinantal Point Process (DPP) with partition constraints, and apply it to multi-agent learning as a function approximator for the centralized value function. An attractive property of using Q-DPP is that when it reaches the optimum value, it can offer a natural factorization of the centralized value function, representing both quality (maximizing reward) and diversity (different behaviors). In the third part of the thesis, I depart from the action-level mutual influence and build a policy-space meta-game to analyze agents' relationship between adaptive policies. I present a Multi-Agent Trust Region Learning (MATRL) algorithm that augments single-agent trust region policy optimization with a weak stable fixed point approximated by the policy-space meta-game. The algorithm aims to find a game-theoretic mechanism to adjust the policy optimization steps that force the learning of all agents toward the stable point
Higher order feature extraction and selection for robust human gesture recognition using CSI of COTS Wi-Fi devices
Device-free human gesture recognition (HGR) using commercial o the shelf (COTS) Wi-Fi
devices has gained attention with recent advances in wireless technology. HGR recognizes the human
activity performed, by capturing the reflections ofWi-Fi signals from moving humans and storing
them as raw channel state information (CSI) traces. Existing work on HGR applies noise reduction
and transformation to pre-process the raw CSI traces. However, these methods fail to capture
the non-Gaussian information in the raw CSI data due to its limitation to deal with linear signal
representation alone. The proposed higher order statistics-based recognition (HOS-Re) model extracts
higher order statistical (HOS) features from raw CSI traces and selects a robust feature subset for the
recognition task. HOS-Re addresses the limitations in the existing methods, by extracting third order
cumulant features that maximizes the recognition accuracy. Subsequently, feature selection methods
derived from information theory construct a robust and highly informative feature subset, fed as
input to the multilevel support vector machine (SVM) classifier in order to measure the performance.
The proposed methodology is validated using a public database SignFi, consisting of 276 gestures
with 8280 gesture instances, out of which 5520 are from the laboratory and 2760 from the home
environment using a 10 5 cross-validation. HOS-Re achieved an average recognition accuracy of
97.84%, 98.26% and 96.34% for the lab, home and lab + home environment respectively. The average
recognition accuracy for 150 sign gestures with 7500 instances, collected from five di erent users was
96.23% in the laboratory environment.Taylor's University through its TAYLOR'S PhD SCHOLARSHIP Programmeinfo:eu-repo/semantics/publishedVersio
Magnification Control in Winner Relaxing Neural Gas
An important goal in neural map learning, which can conveniently be
accomplished by magnification control, is to achieve information optimal coding
in the sense of information theory. In the present contribution we consider the
winner relaxing approach for the neural gas network. Originally, winner
relaxing learning is a slight modification of the self-organizing map learning
rule that allows for adjustment of the magnification behavior by an a priori
chosen control parameter. We transfer this approach to the neural gas
algorithm. The magnification exponent can be calculated analytically for
arbitrary dimension from a continuum theory, and the entropy of the resulting
map is studied numerically conf irming the theoretical prediction. The
influence of a diagonal term, which can be added without impacting the
magnification, is studied numerically. This approach to maps of maximal mutual
information is interesting for applications as the winner relaxing term only
adds computational cost of same order and is easy to implement. In particular,
it is not necessary to estimate the generally unknown data probability density
as in other magnification control approaches.Comment: 14pages, 2 figure
Are screening methods useful in feature selection? An empirical study
Filter or screening methods are often used as a preprocessing step for
reducing the number of variables used by a learning algorithm in obtaining a
classification or regression model. While there are many such filter methods,
there is a need for an objective evaluation of these methods. Such an
evaluation is needed to compare them with each other and also to answer whether
they are at all useful, or a learning algorithm could do a better job without
them. For this purpose, many popular screening methods are partnered in this
paper with three regression learners and five classification learners and
evaluated on ten real datasets to obtain accuracy criteria such as R-square and
area under the ROC curve (AUC). The obtained results are compared through curve
plots and comparison tables in order to find out whether screening methods help
improve the performance of learning algorithms and how they fare with each
other. Our findings revealed that the screening methods were useful in
improving the prediction of the best learner on two regression and two
classification datasets out of the ten datasets evaluated.Comment: 29 pages, 4 figures, 21 table
- …