Sampling-based planning algorithms are the most common probabilistically
complete algorithms and are widely used on many robot platforms. Within this
class of algorithms, many variants have been proposed over the last 20 years,
yet there is still no characterization of which algorithms are well-suited for
which classes of problems. This has motivated us to develop a benchmarking
infrastructure for motion planning algorithms. It consists of three main
components. First, we have created an extensive benchmarking software framework
that is included with the Open Motion Planning Library (OMPL), a C++ library
that contains implementations of many sampling-based algorithms. Second, we
have defined extensible formats for storing benchmark results. The formats are
fairly straightforward so that other planning libraries could easily produce
compatible output. Finally, we have created an interactive, versatile
visualization tool for compact presentation of collected benchmark data. The
tool and underlying database facilitate the analysis of performance across
benchmark problems and planners.Comment: Submitted to IEEE Robotics & Automation Magazine (Special Issue on
Replicable and Measurable Robotics Research), 201
