Learning in Real-Time Search: A Unifying Framework

Real-time search methods are suited for tasks in which the agent is interacting with an initially unknown environment in real time. In such simultaneous planning and learning problems, the agent has to select its actions in a limited amount of time, while sensing only a local part of the environment centered at the agents current location. Real-time heuristic search agents select actions using a limited lookahead search and evaluating the frontier states with a heuristic function. Over repeated experiences, they refine heuristic values of states to avoid infinite loops and to converge to better solutions. The wide spread of such settings in autonomous software and hardware agents has led to an explosion of real-time search algorithms over the last two decades. Not only is a potential user confronted with a hodgepodge of algorithms, but he also faces the choice of control parameters they use. In this paper we address both problems. The first contribution is an introduction of a simple three-parameter framework (named LRTS) which extracts the core ideas behind many existing algorithms. We then prove that LRTA*, epsilon-LRTA*, SLA*, and gamma-Trap algorithms are special cases of our framework. Thus, they are unified and extended with additional features. Second, we prove completeness and convergence of any algorithm covered by the LRTS framework. Third, we prove several upper-bounds relating the control parameters and solution quality. Finally, we analyze the influence of the three control parameters empirically in the realistic scalable domains of real-time navigation on initially unknown maps from a commercial role-playing game as well as routing in ad hoc sensor networks

Envisionment-Based Scheduling Using Time Interval Petri Networks: Representation, Inference, and Learning

331 p.Thesis (Ph.D.)--University of Illinois at Urbana-Champaign, 2000.This interdisciplinary research has made the following five main contributions: (1) A Petri Nets based approach to decision-making scheduling through environment modeling is presented. Petri Nets formalism is known for its solid theoretical base, clear syntax and semantics, intuitive graphic representation, and native concurrency support. (2) The classical Petri Net model is extended in various ways to make it suitable for AI scheduling tasks. The main extensions concern explicit temporal reasoning, context, and operator support. The new formalism is hence called Time Interval Petri Nets (TIPNs). (3) TIPN properties and relation to other Al and Petri Nets formalisms are studied. We also present analysis methods facilitating verification and refinement of Petri Net models. (4) Two machine learning algorithms are developed to synthesize Petri Net models automatically or semi-automatically. One learning algorithm exploits the connection between Petri Nets and Horn clauses by using inductive logic programming methods (ILP) to learn Horn-clauses first and then convert them to TIPNs. The other algorithm employs a general-to-specific search in the space of Petri Net topologies starting with a given initial topology. (5) The framework is applied in the real-time decision-making domain of ship damage control for the tasks of automated problem-solving and intelligent tutoring (advising, critiquing, and scoring). In a large exercise involving approximately 500 simulated ship crisis scenarios, our decision-making expert system showed a 318% improvement over Navy officers by saving 89 more ships.U of I OnlyRestricted to the U of I community idenfinitely during batch ingest of legacy ETD

Automated instructor assistant for ship damage control

The decision making task of ship damage control includes addressing problems such as fire spread, flooding, smoke, equipment failures, and personnel casualties. It is a challenging and highly stressful domain with a limited provision for real-life training. In response to this need, a multimedia interactive damage control simulator system, called DC-Train 2.0 was recently deployed at a Navy officer training school; it provides officers with an immersive environment for damage control training. This paper describes a component of the DC-Train 2.0 system that provides feedback to the user, called the automated instructor assistant. This assistant is based on a blackboardbased expert system called Minerva-DCA, which is capable of solving damage control scenarios at the “expert ” level. It

Index permutations and classes of additive cellular automata rules with isomorphic STD

First we consider the question of identifying linear transformations that transform any additive CA rule into an additive CA rule with an isomorphic STD. A general condition is derived. Following on this, we consider a subclass of such transformations (index permutations). This allows, on one hand, a complete description and on the other hand, generalization of the results for the class of linear rules. Then the case of binary valued 1-dimensional additive cellular automata (d = 1, p = 2) and classes of isomorphisms of STDs that contain only a single rule (singletons) are considered. It is shown how singletons can be used to extend known systems of isomorphic STD classes. Finally we study how the baker transformation provides information about singletons and, by using this, present an algorithm that generates all singletons for one-dimensional additive CA of odd sizes