29 research outputs found
Solving dynamic bandit problems and decentralized games using the kalman bayesian learning automaton
Multi-armed bandit problems have been subject to a lot of research in computer science because it captures
the fundamental dilemma of exploration versus exploitation in reinforcement learning. The goal of
a bandit problem is to determine the optimal balance between the gain of new information (exploration)
and immediate reward maximization (exploitation). Dynamic bandit problems are especially challenging
because they involve changing environments. Combined with game theory, where one analyze the
behavior of agents in multi-agent settings, bandit problems serves as a framework for benchmarking the
applicability of learning algorithms in various situations.
In this thesis, we investigate a novel approach to the multi-armed bandit problem, the Kalman Bayesian
Learning Automaton, an algorithm which applies concepts from Kalman filtering, a powerful technique
for probabilistic reasoning over time. To determine the effectiveness of such an approach we have conducted
an empirical study of the Kalman Bayesian Learning Automaton in multi-armed dynamic bandit
problems and selected games from game theory. Specifically, we evaluate the performance of the Kalman
Bayesian Learning Automaton in randomly changing environments, switching environments, the Goore
game, the Prisoners Dilemma and zero-sum games. The scalability and robustness of the algorithm are
also examined.
Indeed, we reveal that the strength of the Kalman Bayesian Learning Automatons lies in its excellent
tracking abilities, and are among the top performers in all experiments. Unfortunately, it is dependent on
tuning of parameters. We believe further work on the approach could solve the parameter problem, but
even with the need to tune parameters we consider the Kalman Bayesian Learning Automaton a strong
solution to dynamic multi-armed bandit problems and definitely has the potential to be applied in various
applications and multi-agent settings
Aspects of command language portability incorporating a machine-independent filestore concept
A brief summary of job control language development
precedes a general discussion of possible improvements
in command language practice. The user requirements of
a command language are considered with special reference
to a machine independent basis. "Primitive" functions
are defined from this viewpoint.
To meet the proposed objective of portability it is
suggested that an appreciation of the user interaction
with the computer operating system is necessary. This
provides the definition of the user profile model based
on the user requirements of a command language. A second
model is then developed to represent the structure of the
operating system. [Continues.
A Hybrid Multi-Robot Control Architecture
Multi-robot systems provide system redundancy and enhanced capability versus single robot systems. Implementations of these systems are varied, each with specific design approaches geared towards an application domain. Some traditional single robot control architectures have been expanded for multi-robot systems, but these expansions predominantly focus on the addition of communication capabilities. Both design approaches are application specific and limit the generalizability of the system. This work presents a redesign of a common single robot architecture in order to provide a more sophisticated multi-robot system. The single robot architecture chosen for application is the Three Layer Architecture (TLA). The primary strength of TLA is in the ability to perform both reactive and deliberative decision making, enabling the robot to be both sophisticated and perform well in stochastic environments. The redesign of this architecture includes incorporation of the Unified Behavior Framework (UBF) into the controller layer and an addition of a sequencer-like layer (called a Coordinator) to accommodate the multi-robot system. These combine to provide a robust, independent, and taskable individual architecture along with improved cooperation and collaboration capabilities, in turn reducing communication overhead versus many traditional approaches. This multi-robot systems architecture is demonstrated on the RoboCup Soccer Simulator showing its ability to perform well in a dynamic environment where communication constraints are high
Foundations of Software Science and Computation Structures
This open access book constitutes the proceedings of the 24th International Conference on Foundations of Software Science and Computational Structures, FOSSACS 2021, which was held during March 27 until April 1, 2021, as part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2021. The conference was planned to take place in Luxembourg and changed to an online format due to the COVID-19 pandemic. The 28 regular papers presented in this volume were carefully reviewed and selected from 88 submissions. They deal with research on theories and methods to support the analysis, integration, synthesis, transformation, and verification of programs and software systems