Learning in Real-Time Search: A Unifying Framework

Real-time search methods are suited for tasks in which the agent is interacting with an initially unknown environment in real time. In such simultaneous planning and learning problems, the agent has to select its actions in a limited amount of time, while sensing only a local part of the environment centered at the agents current location. Real-time heuristic search agents select actions using a limited lookahead search and evaluating the frontier states with a heuristic function. Over repeated experiences, they refine heuristic values of states to avoid infinite loops and to converge to better solutions. The wide spread of such settings in autonomous software and hardware agents has led to an explosion of real-time search algorithms over the last two decades. Not only is a potential user confronted with a hodgepodge of algorithms, but he also faces the choice of control parameters they use. In this paper we address both problems. The first contribution is an introduction of a simple three-parameter framework (named LRTS) which extracts the core ideas behind many existing algorithms. We then prove that LRTA*, epsilon-LRTA*, SLA*, and gamma-Trap algorithms are special cases of our framework. Thus, they are unified and extended with additional features. Second, we prove completeness and convergence of any algorithm covered by the LRTS framework. Third, we prove several upper-bounds relating the control parameters and solution quality. Finally, we analyze the influence of the three control parameters empirically in the realistic scalable domains of real-time navigation on initially unknown maps from a commercial role-playing game as well as routing in ad hoc sensor networks

Parallel processing for scientific computations

The main contribution of the effort in the last two years is the introduction of the MOPPS system. After doing extensive literature search, we introduced the system which is described next. MOPPS employs a new solution to the problem of managing programs which solve scientific and engineering applications on a distributed processing environment. Autonomous computers cooperate efficiently in solving large scientific problems with this solution. MOPPS has the advantage of not assuming the presence of any particular network topology or configuration, computer architecture, or operating system. It imposes little overhead on network and processor resources while efficiently managing programs concurrently. The core of MOPPS is an intelligent program manager that builds a knowledge base of the execution performance of the parallel programs it is managing under various conditions. The manager applies this knowledge to improve the performance of future runs. The program manager learns from experience

Asymptotically Optimal Sampling-Based Motion Planning Methods

Motion planning is a fundamental problem in autonomous robotics that requires finding a path to a specified goal that avoids obstacles and takes into account a robot's limitations and constraints. It is often desirable for this path to also optimize a cost function, such as path length. Formal path-quality guarantees for continuously valued search spaces are an active area of research interest. Recent results have proven that some sampling-based planning methods probabilistically converge toward the optimal solution as computational effort approaches infinity. This survey summarizes the assumptions behind these popular asymptotically optimal techniques and provides an introduction to the significant ongoing research on this topic.Comment: Posted with permission from the Annual Review of Control, Robotics, and Autonomous Systems, Volume 4. Copyright 2021 by Annual Reviews, https://www.annualreviews.org/. 25 pages. 2 figure

Exploring the deployment of autonomous medical emergency vessels in island and coastal regions : An overview of the opportunities and challenges

Introduction: Applications of vessels in emergency medical systems: Method: Discussion: Conclusion: Emergency medical systems in island and coastal regions face challenges such as supply and staffing shortages and a dispersion of resources and people, which negatively affect the timely and efficient delivery of emergency medical services. This thesis explores the opportunities and challenges of using autonomous vessels in these systems to start a discussion, as current research in this area is very limited. Currently, emergency vessels are primarily used to transfer patients to hospitals, doctors to emergency sites as well as equipment between islands. Floating hospitals ships generally combine these functions by enable comprehensive consultation, diagnosis, and treatment at the emergency sites. Additionally, rescue and search operations can also be counted among the tasks of emergency medical systems if one considers an extended range of tasks for these systems. The location of hub facilities, where autonomous vessels are stationed when not in operation, is one of the first decisions to be made when integrating those vessels into current emergency systems. Therefore, the model for solving the maximal covering location problem is applied and adjusted to cover a wider range of application of vessels in emergency medical systems. Simulations are conducted to identify opportunities to improve system performance when setting hub facilities for autonomous vessels. Hub facilities for autonomous vessels can be located at a greater number of locations, leading to better population coverage in some cases. Furthermore, the complexity of response routes can be decreased by the ability of autonomous vessels to transform current applications of vessels in emergency medical systems. Despite several other opportunities to reduce response times and use resources more efficiently, there are also challenges associated with the use of autonomous vessels. Some main challenges are to successfully integrate the new vessels into the existing system and to ensure their use by the population. Additionally, the costs of autonomous vessels are likely to exceed those of conventional vessels requiring in-depth cost-benefit considerations. Autonomous vessels have a great potential to enhance the performance of emergency medical systems in island and coastal regions. Most of the challenges can be mitigated by carefully planning their operations and introduction of the vessels into the existing system. However, in the context of scarce funding, higher costs compared to conventional vessels are likely to be the most significant challenge for the introduction of autonomous vessels.nhhma

Metamorphic testing: testing the untestable

What if we could know that a program is buggy, even if we could not tell whether or not its observed output is correct? This is one of the key strengths of metamorphic testing, a technique where failures are not revealed by checking an individual concrete output, but by checking the relations among the inputs and outputs of multiple executions of the program under test. Two decades after its introduction, metamorphic testing has become a fully-fledged testing technique with successful applications in multiple domains, including online search engines, autonomous machinery, compilers, Web APIs, and deep learning programs, among others. This article serves as a hands-on entry point for newcomers to metamorphic testing, describing examples, possible applications, and current limitations, providing readers with the basics for the application of the technique in their own projects. IEE

Combining Planning and Deep Reinforcement Learning in Tactical Decision Making for Autonomous Driving

Tactical decision making for autonomous driving is challenging due to the diversity of environments, the uncertainty in the sensor information, and the complex interaction with other road users. This paper introduces a general framework for tactical decision making, which combines the concepts of planning and learning, in the form of Monte Carlo tree search and deep reinforcement learning. The method is based on the AlphaGo Zero algorithm, which is extended to a domain with a continuous state space where self-play cannot be used. The framework is applied to two different highway driving cases in a simulated environment and it is shown to perform better than a commonly used baseline method. The strength of combining planning and learning is also illustrated by a comparison to using the Monte Carlo tree search or the neural network policy separately

Using Taint Analysis and Reinforcement Learning (TARL) to Repair Autonomous Robot Software

It is important to be able to establish formal performance bounds for autonomous systems. However, formal verification techniques require a model of the environment in which the system operates; a challenge for autonomous systems, especially those expected to operate over longer timescales. This paper describes work in progress to automate the monitor and repair of ROS-based autonomous robot software written for an a-priori partially known and possibly incorrect environment model. A taint analysis method is used to automatically extract the data-flow sequence from input topic to publish topic, and instrument that code. A unique reinforcement learning approximation of MDP utility is calculated, an empirical and non-invasive characterization of the inherent objectives of the software designers. By comparing off-line (a-priori) utility with on-line (deployed system) utility, we show, using a small but real ROS example, that it's possible to monitor a performance criterion and relate violations of the criterion to parts of the software. The software is then patched using automated software repair techniques and evaluated against the original off-line utility.Comment: IEEE Workshop on Assured IEEE Workshop on Assured Autonomous Systems, May, 202