Optimistic Planning for Markov Decision Processes

International audienceThe reinforcement learning community has recently intensified its interest in online planning methods, due to their relative independence on the state space size. However, tight near-optimality guarantees are not yet available for the general case of stochastic Markov decision processes and closed-loop, state-dependent planning policies. We therefore consider an algorithm related to AO* that optimistically explores a tree representation of the space of closed-loop policies, and we analyze the near-optimality of the action it returns after n tree node expansions. While this optimistic planning requires a finite number of actions and possible next states for each transition, its asymptotic performance does not depend directly on these numbers, but only on the subset of nodes that significantly impact near-optimal policies. We characterize this set by introducing a novel measure of problem complexity, called the near-optimality exponent. Specializing the exponent and performance bound for some interesting classes of MDPs illustrates the algorithm works better when there are fewer near-optimal policies and less uniform transition probabilities

The FreeMABSys project and the BLAD libraries

International audienceNo abstrac

Robotic fluidic coupling and interrogation of multiple vascularized organ chips

Organ chips can recapitulate organ-level (patho)physiology, yet pharmacokinetic and pharmacodynamic analyses require multi-organ systems linked by vascular perfusion. Here, we describe an ???interrogator??? that employs liquid-handling robotics, custom software and an integrated mobile microscope for the automated culture, perfusion, medium addition, fluidic linking, sample collection and in situ microscopy imaging of up to ten organ chips inside a standard tissue-culture incubator. The robotic interrogator maintained the viability and organ-specific functions of eight vascularized, two-channel organ chips (intestine, liver, kidney, heart, lung, skin, blood???brain barrier and brain) for 3 weeks in culture when intermittently fluidically coupled via a common blood substitute through their reservoirs of medium and endothelium-lined vascular channels. We used the robotic interrogator and a physiological multicompartmental reduced-order model of the experimental system to quantitatively predict the distribution of an inulin tracer perfused through the multi-organ human-body-on-chips. The automated culture system enables the imaging of cells in the organ chips and the repeated sampling of both the vascular and interstitial compartments without compromising fluidic coupling