Human Intent Prediction Using Markov Decision Processes

Peer Reviewedhttps://deepblue.lib.umich.edu/bitstream/2027.42/140661/1/1.i010090.pd

Safe and Efficient Robot Action Choice Using Human Intent Prediction in Physically-Shared Space Environments.

Emerging robotic systems are capable of autonomously planning and executing well-defined tasks, particularly when the environment can be accurately modeled. Robots supporting human space exploration must be able to safely interact with human astronaut companions during intravehicular and extravehicular activities. Given a shared workspace, efficiency can be gained by leveraging robotic awareness of its human companion. This dissertation presents a modular architecture that allows a human and robotic manipulator to efficiently complete independent sets of tasks in a shared physical workspace without the robot requiring oversight or situational awareness from its human companion. We propose that a robot requires four capabilities to act safely and optimally with awareness of its companion: sense the environment and the human within it; translate sensor data into a form useful for decision-making; use this data to predict the human’s future intent; and then use this information to inform its action-choice based also on the robot’s goals and safety constraints. We first present a series of human subject experiments demonstrating that human intent can help a robot predict and avoid conflict, and that sharing the workspace need not degrade human performance so long as the manipulator does not distract or introduce conflict. We describe an architecture that relies on Markov Decision Processes (MDPs) to support robot decision-making. A key contribution of our architecture is its decomposition of the decision problem into two parts: human intent prediction (HIP) and robot action choice (RAC). This decomposition is made possible by an assumption that the robot’s actions will not influence human intent. Presuming an observer that can feedback human actions in real-time, we leverage the well-known space environment and task scripts astronauts rehearse in advance to devise models for human intent prediction and robot action choice. We describe a series of case studies for HIP and RAC using a minimal set of state attributes, including an abbreviated action-history. MDP policies are evaluated in terms of model fitness and safety/efficiency performance tradeoffs. Simulation results indicate that incorporation of both observed and predicted human actions improves robot action choice. Future work could extend to more general human-robot interaction.PhDAerospace EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/107160/1/cmcghan_1.pd

A risk-aware architecture for resilient spacecraft operations

In this paper we discuss a resilient, risk-aware software architecture for onboard, real-time autonomous operations that is intended to robustly handle uncertainty in space-craft behavior within hazardous and unconstrained environments, without unnecessarily increasing complexity. This architecture, the Resilient Spacecraft Executive (RSE), serves three main functions: (1) adapting to component failures to allow graceful degradation, (2) accommodating environments, science observations, and spacecraft capabilities that are not fully known in advance, and (3) making risk-aware decisions without waiting for slow ground-based reactions. This RSE is made up of four main parts: deliberative, habitual, and reflexive layers, and a state estimator that interfaces with all three. We use a risk-aware goal-directed executive within the deliberative layer to perform risk-informed planning, to satisfy the mission goals (specified by mission control) within the specified priorities and constraints. Other state-of-the-art algorithms to be integrated into the RSE include correct-by-construction control synthesis and model-based estimation and diagnosis. We demonstrate the feasibility of the architecture in a simple implementation of the RSE for a simulated Mars rover scenario

A Comparison of Stimulus Set Size on Tact Training for Children with Autism Spectrum Disorder

Previous studies on skill acquisition have taught targets in stimulus sets composed of different numbers of stimuli. Although the rationale for selection of a stimulus set size is not clear, the number of target stimuli trained within a set is a treatment decision for which there is limited empirical support. The current investigation compared the efficiency of tact training in 4 stimulus set sizes, each of which included 12 stimuli grouped into (a) 4 sets of 3 stimuli, (b) 3 sets of 4 stimuli, (c) 2 sets of 6 stimuli, and (d) 1 set of 12 stimuli. Results of all 4 participants with autism spectrum disorder show tact training with larger (i.e., 6 and 12) stimulus set sizes was more efficient than training with smaller (i.e., 3 and 4) stimulus set sizes

Identifying and characterizing COPD patients in US managed care. A retrospective, cross-sectional analysis of administrative claims data

<p>Abstract</p> <p>Background</p> <p>Chronic obstructive pulmonary disease (COPD) is the fourth leading cause of death among US adults and is projected to be the third by 2020. In anticipation of the increasing burden imposed on healthcare systems and payers by patients with COPD, a means of identifying COPD patients who incur higher healthcare utilization and costs is needed.</p> <p>Methods</p> <p>This retrospective, cross-sectional analysis of US managed care administrative claims data describes a practical way to identify COPD patients. We analyze 7.79 million members for potential inclusion in the COPD cohort, who were continuously eligible during a 1-year study period. A younger commercial population (7.7 million) is compared with an older Medicare population (0.115 million). We outline a novel approach to stratifying COPD patients using "complexity" of illness, based on occurrence of claims for given comorbid conditions. Additionally, a unique algorithm was developed to identify and stratify COPD exacerbations using claims data.</p> <p>Results</p> <p>A total of 42,565 commercial (median age 56 years; 51.4% female) and 8507 Medicare patients (median 75 years; 53.1% female) were identified as having COPD. Important differences were observed in comorbidities between the younger commercial versus the older Medicare population. Stratifying by complexity, 45.0%, 33.6%, and 21.4% of commercial patients and 36.6%, 35.8%, and 27.6% of older patients were low, moderate, and high, respectively. A higher proportion of patients with high complexity disease experienced multiple (≥2) exacerbations (61.7% commercial; 49.0% Medicare) than patients with moderate- (56.9%; 41.6%), or low-complexity disease (33.4%; 20.5%). Utilization of healthcare services also increased with an increase in complexity.</p> <p>Conclusion</p> <p>In patients with COPD identified from Medicare or commercial claims data, there is a relationship between complexity as determined by pulmonary and non-pulmonary comorbid conditions and the prevalence of exacerbations and utilization of healthcare services. Identification of COPD patients at highest risk of exacerbations using complexity stratification may facilitate improved disease management by targeting those most in need of treatment.</p

Comparing Skill Acquisition Under Varying Onsets of Differential Reinforcement: A Preliminary Analysis

The purpose of the current study was to evaluate the effect of implementing differential reinforcement at different times relative to the onset of teaching new skills to learners with autism spectrum disorder. Specifically, we first determined the most efficient differential reinforcement arrangement for each participant. Using the most efficient arrangement, we evaluated if differential reinforcement from the immediate onset, early onset, or late onset is the most efficient for learners to acquire a new skill. Three children diagnosed with autism spectrum disorder who have a history of receiving intervention based on the principles of applied behavior analysis participated in this study. The immediate onset of differential reinforcement resulted in the most efficient instruction in 6 of 7 comparisons. The results are discussed in light of previous studies and suggestions for future research are provided

Multicenter evaluation of the clinical utility of laparoscopy-assisted ERCP in patients with Roux-en-Y gastric bypass

Background and Aims The obesity epidemic has led to increased use of Roux-en-Y gastric bypass (RYGB). These patients have an increased incidence of pancreaticobiliary diseases yet standard ERCP is not possible due to surgically altered gastroduodenal anatomy. Laparoscopic-ERCP (LA-ERCP) has been proposed as an option but supporting data are derived from single center small case-series. Therefore, we conducted a large multicenter study to evaluate the feasibility, safety, and outcomes of LA-ERCP. Methods This is retrospective cohort study of adult patients with RYGB who underwent LA-ERCP in 34 centers. Data on demographics, indications, procedure success, and adverse events were collected. Procedure success was defined when all of the following were achieved: reaching the papilla, cannulating the desired duct and providing endoscopic therapy as clinically indicated. Results A total of 579 patients (median age 51, 84% women) were included. Indication for LA-ERCP was biliary in 89%, pancreatic in 8%, and both in 3%. Procedure success was achieved in 98%. Median total procedure time was 152 minutes (IQR 109-210) with median ERCP time 40 minutes (IQR 28-56). Median hospital stay was 2 days (IQR 1-3). Adverse events were 18% (laparoscopy-related 10%, ERCP-related 7%, both 1%) with the clear majority (92%) classified as mild/moderate whereas 8% were severe and 1 death occurred. Conclusion Our large multicenter study indicates that LA-ERCP in patients with RYGB is feasible with a high procedure success rate comparable with that of standard ERCP in patients with normal anatomy. ERCP-related adverse events rate is comparable with conventional ERCP, but the overall adverse event rate was higher due to the added laparoscopy-related events