2,436 research outputs found
Combining Subgoal Graphs with Reinforcement Learning to Build a Rational Pathfinder
In this paper, we present a hierarchical path planning framework called SG-RL
(subgoal graphs-reinforcement learning), to plan rational paths for agents
maneuvering in continuous and uncertain environments. By "rational", we mean
(1) efficient path planning to eliminate first-move lags; (2) collision-free
and smooth for agents with kinematic constraints satisfied. SG-RL works in a
two-level manner. At the first level, SG-RL uses a geometric path-planning
method, i.e., Simple Subgoal Graphs (SSG), to efficiently find optimal abstract
paths, also called subgoal sequences. At the second level, SG-RL uses an RL
method, i.e., Least-Squares Policy Iteration (LSPI), to learn near-optimal
motion-planning policies which can generate kinematically feasible and
collision-free trajectories between adjacent subgoals. The first advantage of
the proposed method is that SSG can solve the limitations of sparse reward and
local minima trap for RL agents; thus, LSPI can be used to generate paths in
complex environments. The second advantage is that, when the environment
changes slightly (i.e., unexpected obstacles appearing), SG-RL does not need to
reconstruct subgoal graphs and replan subgoal sequences using SSG, since LSPI
can deal with uncertainties by exploiting its generalization ability to handle
changes in environments. Simulation experiments in representative scenarios
demonstrate that, compared with existing methods, SG-RL can work well on
large-scale maps with relatively low action-switching frequencies and shorter
path lengths, and SG-RL can deal with small changes in environments. We further
demonstrate that the design of reward functions and the types of training
environments are important factors for learning feasible policies.Comment: 20 page
A vision-guided parallel parking system for a mobile robot using approximate policy iteration
Reinforcement Learning (RL) methods enable autonomous robots to learn skills from scratch by interacting with the environment. However, reinforcement learning can be very time consuming. This paper focuses on accelerating the reinforcement learning process on a mobile robot in an unknown environment. The presented algorithm is based on approximate policy iteration with a continuous state space and a fixed number of actions. The action-value function is represented by a weighted combination of basis functions.
Furthermore, a complexity analysis is provided to show that the implemented approach is guaranteed to converge on an optimal policy with less computational time.
A parallel parking task is selected for testing purposes. In the experiments, the efficiency of the proposed approach is demonstrated and analyzed through a set of simulated and real robot experiments, with comparison drawn from two well known algorithms (Dyna-Q and Q-learning)
AltURI: a thin middleware for simulated robot vision applications
Fast software performance is often the focus when developing real-time vision-based control applications for robot simulators. In this paper we have developed a thin, high performance middleware for USARSim and other simulators designed for real-time vision-based control applications. It includes a fast image server providing images in OpenCV, Matlab or web formats and a simple command/sensor processor. The interface has been tested in USARSim with an Unmanned Aerial Vehicle using two control applications; landing using a reinforcement learning algorithm and altitude control using elementary motion detection. The middleware has been found to be fast enough to control the flying robot as well as very easy to set up and use
Spatial representation for planning and executing robot behaviors in complex environments
Robots are already improving our well-being and productivity in
different applications such as industry, health-care and indoor
service applications. However, we are still far from developing (and
releasing) a fully functional robotic agent that can autonomously
survive in tasks that require human-level
cognitive capabilities. Robotic systems on the market, in fact, are
designed to address specific applications, and can only run
pre-defined behaviors to robustly repeat few tasks (e.g., assembling
objects parts, vacuum cleaning). They internal representation of the
world is usually constrained to the task they are performing, and
does not allows for generalization to other
scenarios. Unfortunately, such a paradigm only apply to a very
limited set of domains, where the environment can be assumed to be
static, and its dynamics can be handled before
deployment. Additionally, robots configured in this way will
eventually fail if their "handcrafted'' representation of the
environment does not match the external world.
Hence, to enable more sophisticated cognitive skills, we investigate
how to design robots to properly represent the environment and
behave accordingly. To this end, we formalize a representation of
the environment that enhances the robot spatial knowledge to
explicitly include a representation of its own actions. Spatial
knowledge constitutes the core of the robot understanding of the
environment, however it is not sufficient to represent what the
robot is capable to do in it. To overcome such a limitation, we
formalize SK4R, a spatial knowledge representation for robots which
enhances spatial knowledge with a novel and "functional"
point of view that explicitly models robot actions. To this end, we
exploit the concept of affordances, introduced to express
opportunities (actions) that objects offer to an agent. To encode
affordances within SK4R, we define the "affordance
semantics" of actions that is used to annotate an environment, and
to represent to which extent robot actions support goal-oriented
behaviors.
We demonstrate the benefits of a functional representation of the
environment in multiple robotic scenarios that traverse and
contribute different research topics relating to: robot knowledge
representations, social robotics, multi-robot systems and robot
learning and planning. We show how a domain-specific representation,
that explicitly encodes affordance semantics, provides the robot
with a more concrete understanding of the environment and of the
effects that its actions have on it. The goal of our work is to
design an agent that will no longer execute an action, because of
mere pre-defined routine, rather, it will execute an actions because
it "knows'' that the resulting state leads one step closer to
success in its task
Keep Rollin' - Whole-Body Motion Control and Planning for Wheeled Quadrupedal Robots
We show dynamic locomotion strategies for wheeled quadrupedal robots, which
combine the advantages of both walking and driving. The developed optimization
framework tightly integrates the additional degrees of freedom introduced by
the wheels. Our approach relies on a zero-moment point based motion
optimization which continuously updates reference trajectories. The reference
motions are tracked by a hierarchical whole-body controller which computes
optimal generalized accelerations and contact forces by solving a sequence of
prioritized tasks including the nonholonomic rolling constraints. Our approach
has been tested on ANYmal, a quadrupedal robot that is fully torque-controlled
including the non-steerable wheels attached to its legs. We conducted
experiments on flat and inclined terrains as well as over steps, whereby we
show that integrating the wheels into the motion control and planning framework
results in intuitive motion trajectories, which enable more robust and dynamic
locomotion compared to other wheeled-legged robots. Moreover, with a speed of 4
m/s and a reduction of the cost of transport by 83 % we prove the superiority
of wheeled-legged robots compared to their legged counterparts.Comment: IEEE Robotics and Automation Letter
Localisation and navigation in GPS-denied environments using RFID tags
Includes bibliographical references.This dissertation addresses the autonomous localisation and navigation problem in the context of an underground mining environment. This kind of environment has little or no features as well as no access to GPS or stationary towers, which are usually used for navigation. In addition dust and debris may hinder optical methods for ranging. This study looks at the feasibility of using randomly distributed RFID tags to autonomously navigate in this environment. Clustering of observed tags are used for localisation, subsequently value iteration is used to navigate to a defined goal. Results are presented, concluding that it is feasible to localise and navigate using only RFID tags, in simulation. Localisation feasibility is also confirmed by experimental measurements
Recommended from our members
Localization from semantic observations via the matrix permanent
Most approaches to robot localization rely on low-level geometric features such as points, lines, and planes. In this paper, we use object recognition to obtain semantic information from the robot’s sensors and consider the task of localizing the robot within a prior map of landmarks, which are annotated with semantic labels. As object recognition algorithms miss detections and produce false alarms, correct data association between the detections and the landmarks on the map is central to the semantic localization problem. Instead of the traditional vector-based representation, we propose a sensor model, which encodes the semantic observations via random finite sets and enables a unified treatment of missed detections, false alarms, and data association. Our second contribution is to reduce the problem of computing the likelihood of a set-valued observation to the problem of computing a matrix permanent. It is this crucial transformation that allows us to solve the semantic localization problem with a polynomial-time approximation to the set-based Bayes filter. Finally, we address the active semantic localization problem, in which the observer’s trajectory is planned in order to improve the accuracy and efficiency of the localization process. The performance of our approach is demonstrated in simulation and in real environments using deformable-part-model-based object detectors. Robust global localization from semantic observations is demonstrated for a mobile robot, for the Project Tango phone, and on the KITTI visual odometry dataset. Comparisons are made with the traditional lidar-based geometric Monte Carlo localization
- …