Pond-Hindsight: Applying Hindsight Optimization to Partially-Observable Markov Decision Processes

Partially-observable Markov decision processes (POMDPs) are especially good at modeling real-world problems because they allow for sensor and effector uncertainty. Unfortunately, such uncertainty makes solving a POMDP computationally challenging. Traditional approaches, which are based on value iteration, can be slow because they find optimal actions for every possible situation. With the help of the Fast Forward (FF) planner, FF- Replan and FF-Hindsight have shown success in quickly solving fully-observable Markov decision processes (MDPs) by solving classical planning translations of the problem. This thesis extends the concept of problem determination to POMDPs by sampling action observations (similar to how FF-Replan samples action outcomes) and guiding the construction of policy trajectories with a conformant (as opposed to classical) planning heuristic. The resultant planner is called POND-Hindsight

THE LAST PLANNER® SYSTEM PATH CLEARING APPROACH IN ACTION: A CASE STUDY

The “Last Planner® System” (LPS) is commonly viewed as the foundation of Lean Project Delivery. It is increasingly used in certain parts of the globe. However, LPS implementation often fades off due to issues reported at organisational, project and external levels. The LPS Path Clearing Approach (PCA) offers an antidote to these issues. The goal of this paper is to outline how the LPS-PCA helped restart a stalled implementation of the LPS through a “shallow and wide” organisational approach rather than a more traditional “narrow and deep” project approach. The LPS-PCA in action is documented within an on-going UK case study organisation. Action and covert research methods were used to introduce LPS principles, thinking and language without attributing them to LPS in response to resistance to the actual LPS. The 15 step actions within the LPS-PCA are expanded from a past, current and future state perspective. The study found that the LPS-PCA’s 15 step actions were useful as a benchmark to continuously remove constraints that blocked the implementation of the LPS. In summary, the use of the LPSPCA is recommended before, during and after organisations engage with LPS Consultants if organisations are serious about sustaining the implementation of the LPS

Enroute flight planning: The design of cooperative planning systems

Design concepts and principles to guide in the building of cooperative problem solving systems are being developed and evaluated. In particular, the design of cooperative systems for enroute flight planning is being studied. The investigation involves a three stage process, modeling human performance in existing environments, building cognitive artifacts, and studying the performance of people working in collaboration with these artifacts. The most significant design concepts and principles identified thus far are the principle focus

Shared Autonomy via Hindsight Optimization

In shared autonomy, user input and robot autonomy are combined to control a robot to achieve a goal. Often, the robot does not know a priori which goal the user wants to achieve, and must both predict the user's intended goal, and assist in achieving that goal. We formulate the problem of shared autonomy as a Partially Observable Markov Decision Process with uncertainty over the user's goal. We utilize maximum entropy inverse optimal control to estimate a distribution over the user's goal based on the history of inputs. Ideally, the robot assists the user by solving for an action which minimizes the expected cost-to-go for the (unknown) goal. As solving the POMDP to select the optimal action is intractable, we use hindsight optimization to approximate the solution. In a user study, we compare our method to a standard predict-then-blend approach. We find that our method enables users to accomplish tasks more quickly while utilizing less input. However, when asked to rate each system, users were mixed in their assessment, citing a tradeoff between maintaining control authority and accomplishing tasks quickly