A Survey of Knowledge-based Sequential Decision Making under Uncertainty

Reasoning with declarative knowledge (RDK) and sequential decision-making (SDM) are two key research areas in artificial intelligence. RDK methods reason with declarative domain knowledge, including commonsense knowledge, that is either provided a priori or acquired over time, while SDM methods (probabilistic planning and reinforcement learning) seek to compute action policies that maximize the expected cumulative utility over a time horizon; both classes of methods reason in the presence of uncertainty. Despite the rich literature in these two areas, researchers have not fully explored their complementary strengths. In this paper, we survey algorithms that leverage RDK methods while making sequential decisions under uncertainty. We discuss significant developments, open problems, and directions for future work

iCORPP: Interleaved Commonsense Reasoning and Probabilistic Planning on Robots

Robot sequential decision-making in the real world is a challenge because it requires the robots to simultaneously reason about the current world state and dynamics, while planning actions to accomplish complex tasks. On the one hand, declarative languages and reasoning algorithms well support representing and reasoning with commonsense knowledge. But these algorithms are not good at planning actions toward maximizing cumulative reward over a long, unspecified horizon. On the other hand, probabilistic planning frameworks, such as Markov decision processes (MDPs) and partially observable MDPs (POMDPs), well support planning to achieve long-term goals under uncertainty. But they are ill-equipped to represent or reason about knowledge that is not directly related to actions. In this article, we present a novel algorithm, called iCORPP, to simultaneously estimate the current world state, reason about world dynamics, and construct task-oriented controllers. In this process, robot decision-making problems are decomposed into two interdependent (smaller) subproblems that focus on reasoning to "understand the world" and planning to "achieve the goal" respectively. Contextual knowledge is represented in the reasoning component, which makes the planning component epistemic and enables active information gathering. The developed algorithm has been implemented and evaluated both in simulation and on real robots using everyday service tasks, such as indoor navigation, dialog management, and object delivery. Results show significant improvements in scalability, efficiency, and adaptiveness, compared to competitive baselines including handcrafted action policies

Computing action equivalences for planning under time-constraints

In order for autonomous artificial decision-makers to solverealistic tasks, they need to deal with the dual problems of searching throughlarge state and action spaces under time pressure.We study the problem of planning in domains with lots of objects. Structuredrepresentations of action can help provide guidance when the number of actionchoices and size of the state space is large.We show how structured representations ofaction effects can help us partition the action space in to a smallerset of approximate equivalence classes. Then, the pared-downaction space can be used to identify a useful subset of the state space in whichto search for a solution. As computational resources permit, we thenallow ourselves to elaborate the original solution. This kind of analysisallows us to collapse the action space and permits faster planning in muchlarger domains than before

Learning and Reasoning for Robot Dialog and Navigation Tasks

You are viewing an article from the Proceedings of the 21st Annual Meeting of the Special Interest Group on Discourse and Dialogue that was in the Good Systems Network Digest in 2020.Office of the VP for Researc