405 research outputs found
Adding Neural Network Controllers to Behavior Trees without Destroying Performance Guarantees
In this paper, we show how Behavior Trees that have performance guarantees,
in terms of safety and goal convergence, can be extended with components that
were designed using machine learning, without destroying those performance
guarantees.
Machine learning approaches such as reinforcement learning or learning from
demonstration can be very appealing to AI designers that want efficient and
realistic behaviors in their agents. However, those algorithms seldom provide
guarantees for solving the given task in all different situations while keeping
the agent safe. Instead, such guarantees are often easier to find for manually
designed model based approaches. In this paper we exploit the modularity of
Behavior trees to extend a given design with an efficient, but possibly
unreliable, machine learning component in a way that preserves the guarantees.
The approach is illustrated with an inverted pendulum example.Comment: Submitted to IEEE Transactions on Game
Improving the performance of Learned Controllers in Behavior Trees using Value Function Estimates at Switching Boundaries
Behavior trees represent a modular way to create an overall controller from a
set of sub-controllers solving different sub-problems. These sub-controllers
can be created in different ways, such as classical model based control or
reinforcement learning (RL). If each sub-controller satisfies the preconditions
of the next sub-controller, the overall controller will achieve the overall
goal. However, even if all sub-controllers are locally optimal in achieving the
preconditions of the next, with respect to some performance metric such as
completion time, the overall controller might be far from optimal with respect
to the same performance metric. In this paper we show how the performance of
the overall controller can be improved if we use approximations of value
functions to inform the design of a sub-controller of the needs of the next
one. We also show how, under certain assumptions, this leads to a globally
optimal controller when the process is executed on all sub-controllers.
Finally, this result also holds when some of the sub-controllers are already
given, i.e., if we are constrained to use some existing sub-controllers the
overall controller will be globally optimal given this constraint
On the Implementation of Behavior Trees in Robotics
There is a growing interest in Behavior Trees (BTs) as a tool to describe and
implement robot behaviors. BTs were devised in the video game industry and
their adoption in robotics resulted in the development of ad-hoc libraries to
design and execute BTs that fit complex robotics software architectures.
While there is broad consensus on how BTs work, some characteristics rely on
the implementation choices done in the specific software library used.
In this letter, we outline practical aspects in the adoption of BTs and the
solutions devised by the robotics community to fully exploit the advantages of
BTs in real robots. We also overview the solutions proposed in open-source
libraries used in robotics, we show how BTs fit in robotic software
architecture, and we present a use case example
An Extended Convergence Result for Behaviour Tree Controllers
Behavior trees (BTs) are an optimally modular framework to assemble
hierarchical hybrid control policies from a set of low-level control policies
using a tree structure. Many robotic tasks are naturally decomposed into a
hierarchy of control tasks, and modularity is a well-known tool for handling
complexity, therefor behavior trees have garnered widespread usage in the
robotics community. In this paper, we study the convergence of BTs, in the
sense of reaching a desired part of the state space. Earlier results on BT
convergence were often tailored to specific families of BTs, created using
different design principles. The results of this paper generalize the earlier
results and also include new cases of cyclic switching not covered in the
literature.Comment: Submitted to the IEEE Transactions on Robotics (T-RO
Evolving robots: from simple behaviours to complete systems
Building robots is generally considered difficult, because the designer not only has to
predict the interaction between the robot and the environment, but also has to deal
with the ensuing problems. This thesis examines the use of the evolutionary approach
in designing robots; the explorations range from evolving simple behaviours for real
robots, to complex behaviours (also for real robots), and finally to complete robot
systems — including controllers and body plans.
A framework is presented for evolving robot control systems. It includes two components: a task independent Genetic Programming sub-system and a task dependent
controller evaluation sub-system. The performance evaluation of each robot controller
is done in a simulator to reduce the evaluation time, and then the evolved controllers
are downloaded to a real robot for performance verification. In addition, a special rep¬
resentation is designed for the reactive robot controller. It is succinct and can capture
the important characteristics of a reactive control system, so that the evolutionary system can efficiently evolve the controllers of the desired behaviours for the robots. The
framework has been successfully used to evolve controllers for real robots to achieve a
variety of simple tasks, such as obstacle avoidance, safe exploration and box-pushing.
A methodology is then proposed to scale up the system to evolve controllers for more
complicated tasks. It involves adopting the architecture of a behaviour-based system,
and evolving separate behaviour controllers and arbitrators for coordination. This
allows robot controllers for more complex skills to be constructed in an incremental
manner. Therefore the whole control system becomes easy to evolve; moreover, the
resulting control system can be explicitly distributed, understandable to the system
designer, and easy to maintain. The methodology has been used to evolve control
systems for more complex tasks with good results.
Finally, the evolutionary mechanism of the framework described above is extended
to include a Genetic Algorithm sub-system for the co-evolution of robot body plans
— structuralparametersofphysicalrobotsencodedaslinearstringsofrealnumbers.
An individual in the extended system thus consists of a brain(controller) and a body.
Whenever the individual is evaluated, the controller is executed on the corresponding
body for a period of time to measure the performance. In such a system the Genetic
Programming part evolves the controller; and the Genetic Algorithm part, the robot
body. The results show that the complete robot system can be evolved in this manner.
i
- …