37 research outputs found
Aktionenlernen mit Selbstorganisierenden Karten und Reinforcement Learning
This doctoral thesis deals with the development of a function approximator
and its application to methods for learning discrete and continuous
actions:
1. A general function approximator ? Locally Weighted Interpolating
Growing Neural Gas (LWIGNG) ? is developed from Growing Neural Gas
(GNG). The topological neighbourhood structure is used for calculating
interpolations between neighbouring neurons and for applying a local
weighting scheme. The capabilities of this method are shown in several
experiments, with special considerations given to changing target
functions and changing input distributions.
2. To learn discrete actions LWIGNG is combined with Q-Learning forming
the Q-LWIGNG method. The underlying GNG-algorithm has to be changed
to take care of the special order of the input data in action learning.
Q-LWIGNG achieves very good results in experiments with the pole
balancing and the mountain car problems, and good results with the
acrobot problem.
3. To learn continuous actions a REINFORCE algorithm is combined with
LWIGNG forming the ReinforceGNG method. An actor-critic architecture
is used for learning from delayed rewards. LWIGNG approximates both
the state-value function and the policy. The policy is given by the
situation dependent parameters of a normal distribution. ReinforceGNG
is applied successfully to learn continuous actions of a simulated
2-wheeled robot which has to intercept a rolling ball under certain
conditions
Aktionenlernen mit Selbstorganisierenden Karten und Reinforcement Learning
This doctoral thesis deals with the development of a function approximator
and its application to methods for learning discrete and continuous
actions:
1. A general function approximator ? Locally Weighted Interpolating
Growing Neural Gas (LWIGNG) ? is developed from Growing Neural Gas
(GNG). The topological neighbourhood structure is used for calculating
interpolations between neighbouring neurons and for applying a local
weighting scheme. The capabilities of this method are shown in several
experiments, with special considerations given to changing target
functions and changing input distributions.
2. To learn discrete actions LWIGNG is combined with Q-Learning forming
the Q-LWIGNG method. The underlying GNG-algorithm has to be changed
to take care of the special order of the input data in action learning.
Q-LWIGNG achieves very good results in experiments with the pole
balancing and the mountain car problems, and good results with the
acrobot problem.
3. To learn continuous actions a REINFORCE algorithm is combined with
LWIGNG forming the ReinforceGNG method. An actor-critic architecture
is used for learning from delayed rewards. LWIGNG approximates both
the state-value function and the policy. The policy is given by the
situation dependent parameters of a normal distribution. ReinforceGNG
is applied successfully to learn continuous actions of a simulated
2-wheeled robot which has to intercept a rolling ball under certain
conditions
Learning to Approach a Moving Ball with a Simulated Two-Wheeled Robot
We show how a two-wheeled robot can learn to approach a moving ball
using Reinforcement Learning. The robot is controlled by setting
the velocities of its two wheels. It has to reach the ball under
certain conditions to be able to kick it towards a given target.
In order to kick, the ball has to be in front of the robot. The robot
also has to reach the ball at a certain angle in relation to the
target, because the ball is always kicked in the direction from the
center of the robot to the ball. The robot learns which velocity
differences should be applied to the wheels: one of the wheels is
set to the maximum velocity, the other one according to this difference.We
apply a REINFORCE algorithm [1] in combination with some kind of
extended Growing Neural Gas (GNG) [2] to learn these continuous actions.
The resulting algorithm, called ReinforceGNG, is tested in a simulated
environment with and without noise
Implementation -- Service -- Effect: The ISE Metamodel of Critical Infrastructures
The ISE (Implementation - Service - Effect) metamodel is a general
modelling framework for systems of critical infrastructures taking
the various viewpoints from different sectors and professions into
account. While not neglecting the technical basis, it provides the
necessary abstractions needed for risk or emergency management of
critical infrastructures in a complex environment. ISE supports an
iterative modelling approach that allows ongoing refinement steps
based on the analysis of the current model. This iterative approach
is able to minimise some of the existing problems commonly found
in critical infrastructure modelling and simulation. By focusing
on the services provided by critical infrastructures it is possible
to bridge the gap between the business view and the engineering view
on critical infrastructures. The technical realisation of services
is described in the implementation layer; the effects of the successful
or unsuccessful delivery of services are described using the effect
layer. A sound mathematical foundation provides the basis for all
kinds of analysis starting with topological analysis of the dependency
structures up to statistical analysis of results obtained by the
simulation of complex agent-based models
Aktionenlernen mit Selbstorganisierenden Karten und Reinforcement Learning
Die vorliegende Arbeit beschäftigt sich mit der Entwicklung eines Funktionsapproximators und dessen Verwendung in Verfahren zum Lernen von diskreten und kontinuierlichen Aktionen:
1. Ein allgemeiner Funktionsapproximator – Locally Weighted Interpolating Growing Neural Gas (LWIGNG) – wird auf Basis eines Wachsenden Neuralen Gases (GNG) entwickelt. Die topologische Nachbarschaft in der Neuronenstruktur wird verwendet, um zwischen benachbarten Neuronen zu interpolieren und durch lokale Gewichtung die Approximation zu berechnen. Die Leistungsfähigkeit des Ansatzes, insbesondere in Hinsicht auf sich verändernde Zielfunktionen und sich verändernde Eingabeverteilungen, wird in verschiedenen Experimenten unter Beweis gestellt.
2. Zum Lernen diskreter Aktionen wird das LWIGNG-Verfahren mit Q-Learning zur Q-LWIGNG-Methode verbunden. DafĂĽr muss der zugrunde
liegende GNG-Algorithmus abgeändert werden, da die Eingabedaten beim Aktionenlernen eine bestimmte Reihenfolge haben. Q-LWIGNG erzielt sehr gute Ergebnisse beim Stabbalance- und beim Mountain-Car-Problem und gute Ergebnisse beim Acrobot-Problem.
3. Zum Lernen kontinuierlicher Aktionen wird ein REINFORCE-Algorithmus mit LWIGNG zur ReinforceGNG-Methode verbunden. Dabei wird eine Actor-Critic-Architektur eingesetzt, um aus zeitverzögerten Belohnungen zu lernen. LWIGNG approximiert sowohl die Zustands-Wertefunktion als auch die Politik, die in Form von situationsabhängigen Parametern einer Normalverteilung repräsentiert wird. ReinforceGNG wird erfolgreich zum Lernen von Bewegungen für einen simulierten 2-rädrigen Roboter eingesetzt, der einen rollenden Ball unter bestimmten Bedingungen abfangen soll.This doctoral thesis deals with the development of a function approximator and its application to methods for learning discrete
and continuous actions:
1. A general function approximator – Locally Weighted Interpolating Growing Neural Gas (LWIGNG) – is developed from Growing Neural Gas (GNG). The topological neighbourhood structure is used for calculating interpolations between neighbouring neurons and for applying a local weighting scheme. The capabilities of this method are shown in several experiments, with special considerations given to changing target functions and changing input distributions.
2. To learn discrete actions LWIGNG is combined with Q-Learning forming the Q-LWIGNG method. The underlying GNG-algorithm has to be changed to take care of the special order of the input data in action learning. Q-LWIGNG achieves very good results in experiments with the pole balancing and the mountain car problems, and good results with the acrobot problem.
3. To learn continuous actions a REINFORCE algorithm is combined with LWIGNG forming the ReinforceGNG method. An actor-critic architecture is used for learning
from delayed rewards. LWIGNG approximates both the state-value function and the policy. The policy is given by the situation dependent parameters of a normal distribution. ReinforceGNG is applied successfully to learn continuous actions of a simulated 2-wheeled robot which has to intercept a rolling ball under certain conditions
Towards a Holistic Metamodel for Systems of Critical Infrastructures
The Implementation-Service-Effect (ISE) metamodel describes Critical
Infrastructures from different perspectives in a well-defined way
to provide a sound basis for the analysis of their dependencies and
interdependencie
Implementation - service -effect : the ISE metamodel of critical infrastructures
The ISE (Implementation - Service - Effect) metamodel is a general
modelling framework for systems of critical infrastructures taking
the various viewpoints from different sectors and professions into
account. While not neglecting the technical basis, it provides the
necessary abstractions needed for risk or emergency management of
critical infrastructures in a complex environment. ISE supports an
iterative modelling approach that allows ongoing refinement steps
based on the analysis of the current model. This iterative approach
is able to minimise some of the existing problems commonly found
in critical infrastructure modelling and simulation. By focusing
on the services provided by critical infrastructures it is possible
to bridge the gap between the business view and the engineering view
on critical infrastructures. The technical realisation of services
is described in the implementation layer; the effects of the successful
or unsuccessful delivery of services are described using the effect
layer. A sound mathematical foundation provides the basis for all
kinds of analysis starting with topological analysis of the dependency
structures up to statistical analysis of results obtained by the
simulation of complex agent-based models