Anytime Point-Based Approximations for Large POMDPs

The Partially Observable Markov Decision Process has long been recognized as a rich framework for real-world planning and control problems, especially in robotics. However exact solutions in this framework are typically computationally intractable for all but the smallest problems. A well-known technique for speeding up POMDP solving involves performing value backups at specific belief points, rather than over the entire belief simplex. The efficiency of this approach, however, depends greatly on the selection of points. This paper presents a set of novel techniques for selecting informative belief points which work well in practice. The point selection procedure is combined with point-based value backups to form an effective anytime POMDP algorithm called Point-Based Value Iteration (PBVI). The first aim of this paper is to introduce this algorithm and present a theoretical analysis justifying the choice of belief selection technique. The second aim of this paper is to provide a thorough empirical comparison between PBVI and other state-of-the-art POMDP methods, in particular the Perseus algorithm, in an effort to highlight their similarities and differences. Evaluation is performed using both standard POMDP domains and realistic robotic tasks

What is This?

Downloaded from ijr.sagepub.com at UNIV CALIFORNIA BERKELEY LIB on June 18, 2014Article Motion planning with sequential convex optimization and convex collision checkin

Designing algorithms to aid discovery by chemical robots

Recently, automated robotic systems have become very efficient, thanks to improved coupling between sensor systems and algorithms, of which the latter have been gaining significance thanks to the increase in computing power over the past few decades. However, intelligent automated chemistry platforms for discovery orientated tasks need to be able to cope with the unknown, which is a profoundly hard problem. In this Outlook, we describe how recent advances in the design and application of algorithms, coupled with the increased amount of chemical data available, and automation and control systems may allow more productive chemical research and the development of chemical robots able to target discovery. This is shown through examples of workflow and data processing with automation and control, and through the use of both well-used and cutting-edge algorithms illustrated using recent studies in chemistry. Finally, several algorithms are presented in relation to chemical robots and chemical intelligence for knowledge discovery

Particle Swarm Optimization Based Source Seeking

Signal source seeking using autonomous vehicles is a complex problem. The complexity increases manifold when signal intensities captured by physical sensors onboard are noisy and unreliable. Added to the fact that signal strength decays with distance, noisy environments make it extremely difficult to describe and model a decay function. This paper addresses our work with seeking maximum signal strength in a continuous electromagnetic signal source with mobile robots, using Particle Swarm Optimization (PSO). A one to one correspondence with swarm members in a PSO and physical Mobile robots is established and the positions of the robots are iteratively updated as the PSO algorithm proceeds forward. Since physical robots are responsive to swarm position updates, modifications were required to implement the interaction between real robots and the PSO algorithm. The development of modifications necessary to implement PSO on mobile robots, and strategies to adapt to real life environments such as obstacles and collision objects are presented in this paper. Our findings are also validated using experimental testbeds.Comment: 13 pages, 12 figure

Multi-agent pathfinding for unmanned aerial vehicles

Unmanned aerial vehicles (UAVs), commonly known as drones, have become more and more prevalent in recent years. In particular, governmental organizations and companies around the world are starting to research how UAVs can be used to perform tasks such as package deliver, disaster investigation and surveillance of key assets such as pipelines, railroads and bridges. NASA is currently in the early stages of developing an air traffic control system specifically designed to manage UAV operations in low-altitude airspace. Companies such as Amazon and Rakuten are testing large-scale drone deliver services in the USA and Japan. To perform these tasks, safe and conflict-free routes for concurrently operating UAVs must be found. This can be done using multi-agent pathfinding (mapf) algorithms, although the correct choice of algorithms is not clear. This is because many state of the art mapf algorithms have only been tested in 2D space in maps with many obstacles, while UAVs operate in 3D space in open maps with few obstacles. In addition, when an unexpected event occurs in the airspace and UAVs are forced to deviate from their original routes while inflight, new conflict-free routes must be found. Planning for these unexpected events is commonly known as contingency planning. With manned aircraft, contingency plans can be created in advance or on a case-by-case basis while inflight. The scale at which UAVs operate, combined with the fact that unexpected events may occur anywhere at any time make both advanced planning and planning on a case-by-case basis impossible. Thus, a new approach is needed. Online multi-agent pathfinding (online mapf) looks to be a promising solution. Online mapf utilizes traditional mapf algorithms to perform path planning in real-time. That is, new routes for UAVs are found while inflight. The primary contribution of this thesis is to present one possible approach to UAV contingency planning using online multi-agent pathfinding algorithms, which can be used as a baseline for future research and development. It also provides an in-depth overview and analysis of offline mapf algorithms with the goal of determining which ones are likely to perform best when applied to UAVs. Finally, to further this same goal, a few different mapf algorithms are experimentally tested and analyzed