Scalable Planning and Learning for Multiagent POMDPs: Extended Version

Online, sample-based planning algorithms for POMDPs have shown great promise in scaling to problems with large state spaces, but they become intractable for large action and observation spaces. This is particularly problematic in multiagent POMDPs where the action and observation space grows exponentially with the number of agents. To combat this intractability, we propose a novel scalable approach based on sample-based planning and factored value functions that exploits structure present in many multiagent settings. This approach applies not only in the planning case, but also in the Bayesian reinforcement learning setting. Experimental results show that we are able to provide high quality solutions to large multiagent planning and learning problems

Sign switch of Gaussian bending modulus for microemulsions; a self-consistent field analysis exploring scale invariant curvature energies

Bending rigidities of tensionless balanced liquid-liquid interfaces as occurring in microemulsions are predicted using self-consistent field theory for molecularly inhomogeneous systems. Considering geometries with scale invariant curvature energies gives unambiguous bending rigidities for systems with fixed chemical potentials: The minimal surface Im3m cubic phase is used to find the Gaussian bending rigidity, $\bar{\kappa}$, and a torus with Willmore energy $W=2 \pi^2$ allows for direct evaluation of the mean bending modulus, $\kappa$. Consistent with this, the spherical droplet gives access to $2 \kappa + \bar{\kappa}$. We observe that $\bar{\kappa}$ tends to be negative for strong segregation and positive for weak segregation; a finding which is instrumental for understanding phase transitions from a lamellar to a sponge-like microemulsion. Invariably, $\kappa$ remains positive and increases with increasing strength of segregation.Comment: 7 pages, 5 figure

Exploiting Anonymity in Approximate Linear Programming: Scaling to Large Multiagent MDPs (Extended Version)

Many exact and approximate solution methods for Markov Decision Processes (MDPs) attempt to exploit structure in the problem and are based on factorization of the value function. Especially multiagent settings, however, are known to suffer from an exponential increase in value component sizes as interactions become denser, meaning that approximation architectures are restricted in the problem sizes and types they can handle. We present an approach to mitigate this limitation for certain types of multiagent systems, exploiting a property that can be thought of as "anonymous influence" in the factored MDP. Anonymous influence summarizes joint variable effects efficiently whenever the explicit representation of variable identity in the problem can be avoided. We show how representational benefits from anonymity translate into computational efficiencies, both for general variable elimination in a factor graph but in particular also for the approximate linear programming solution to factored MDPs. The latter allows to scale linear programming to factored MDPs that were previously unsolvable. Our results are shown for the control of a stochastic disease process over a densely connected graph with 50 nodes and 25 agents.Comment: Extended version of AAAI 2016 pape

Integral multidisciplinary rehabilitation treatment planning

This paper presents a methodology to plan treatments for rehabilitation outpatients. These patients require a series of treatments by therapists from various disciplines. In current practice, when treatments are planned, a lack of coordination between the different disciplines, along with a failure to plan the entire treatment plan at once, often occurs. This situation jeopardizes both the quality of care and the logistical performance. The multidisciplinary nature of the rehabilitation process complicates planning and control. An integral treatment planning methodology, based on an integer linear programming (ILP) formulation, ensures continuity of the rehabilitation process while simultaneously controlling seven performance indicators including access times, combination appointments, and therapist utilization. We apply our approach to the rehabilitation outpatient clinic of the Academic Medical Center (AMC) in Amsterdam, the Netherlands. Based on the results of this case, we are convinced that our approach can be valuable for decision-making support in resource capacity planning and control at many rehabilitation outpatient clinics. The developed model will be part of the new hospital information system of the AMC

Structure in the Value Function of Two-Player Zero-Sum Games of Incomplete Information

Zero-sum stochastic games provide a rich model for competitive decision making. However, under general forms of state uncertainty as considered in the Partially Observable Stochastic Game (POSG), such decision making problems are still not very well understood. This paper makes a contribution to the theory of zero-sum POSGs by characterizing structure in their value function. In particular, we introduce a new formulation of the value function for zs-POSGs as a function of the "plan-time sufficient statistics" (roughly speaking the information distribution in the POSG), which has the potential to enable generalization over such information distributions. We further delineate this generalization capability by proving a structural result on the shape of value function: it exhibits concavity and convexity with respect to appropriately chosen marginals of the statistic space. This result is a key pre-cursor for developing solution methods that may be able to exploit such structure. Finally, we show how these results allow us to reduce a zs-POSG to a "centralized" model with shared observations, thereby transferring results for the latter, narrower class, to games with individual (private) observations

Influence-Optimistic Local Values for Multiagent Planning --- Extended Version

Recent years have seen the development of methods for multiagent planning under uncertainty that scale to tens or even hundreds of agents. However, most of these methods either make restrictive assumptions on the problem domain, or provide approximate solutions without any guarantees on quality. Methods in the former category typically build on heuristic search using upper bounds on the value function. Unfortunately, no techniques exist to compute such upper bounds for problems with non-factored value functions. To allow for meaningful benchmarking through measurable quality guarantees on a very general class of problems, this paper introduces a family of influence-optimistic upper bounds for factored decentralized partially observable Markov decision processes (Dec-POMDPs) that do not have factored value functions. Intuitively, we derive bounds on very large multiagent planning problems by subdividing them in sub-problems, and at each of these sub-problems making optimistic assumptions with respect to the influence that will be exerted by the rest of the system. We numerically compare the different upper bounds and demonstrate how we can achieve a non-trivial guarantee that a heuristic solution for problems with hundreds of agents is close to optimal. Furthermore, we provide evidence that the upper bounds may improve the effectiveness of heuristic influence search, and discuss further potential applications to multiagent planning.Comment: Long version of IJCAI 2015 paper (and extended abstract at AAMAS 2015

Infrared spectra of Mg-SiO smokes : comparison with analytical electron microscopy studies

An important component of current models for interstellar and circumstellar evolution is the infrared (IR)spectral data collected from stellar outflows around oxygen-rich stars and from the general interstellar medium [1]. IR spectra from these celestial bodies are usually interpreted as showing the general properties of sub-micron sized silicate grains [2]. Two major features at 10 and 20 microns are reasonably attributed to amorphous olivine or pyroxene (e.g. Mg2Si04 or MgSi03) on the basis of comparisons with natural standards and vapor condensed silicates [3-6]. In an attempt to define crystallisation rates for spectrally amorphous condensates, Nuth and Donn [5] annealed experimentally produced amorphous magnesium silicate smokes at 1000K. On analysing these smokes at various annealing times, Nuth and Donn [5] showed that changes in crystallinity measured by bulk X-ray diffraction occured at longer annealing times (days) than changes measured by IR spectra (a few hours). To better define the onset of crystallinity in these magnesium silicates, we have examined each annealed product using a JEOL 1OOCX analytical electron microscope (AEM). In addition, the development of chemical diversity with annealing has been monitored using energy dispersive spectroscopy of individual grains from areas <20nm in diameter. Furthermore, the crystallisation kinetics of these smokes under ambient, room temperature conditions have been examined using bulk and fourier transform infrared (FTIR)spectra

Predictors of barefoot plantar pressure during walking in patients with diabetes, peripheral neuropathy and a history of ulceration

OBJECTIVE:Elevated dynamic plantar foot pressures significantly increase the risk of foot ulceration in diabetes mellitus. The aim was to determine which factors predict plantar pressures in a population of diabetic patients who are at high-risk of foot ulceration. METHODS:Patients with diabetes, peripheral neuropathy and a history of ulceration were eligible for inclusion in this cross sectional study. Demographic data, foot structure and function, and disease-related factors were recorded and used as potential predictor variables in the analyses. Barefoot peak pressures during walking were calculated for the heel, midfoot, forefoot, lesser toes, and hallux regions. Potential predictors were investigated using multivariate linear regression analyses. 167 participants with mean age of 63 years contributed 329 feet to the analyses. RESULTS:The regression models were able to predict between 6% (heel) and 41% (midfoot) of the variation in peak plantar pressures. The largest contributing factor in the heel model was glycosylated haemoglobin concentration, in the midfoot Charcot deformity, in the forefoot prominent metatarsal heads, in the lesser toes hammer toe deformity and in the hallux previous ulceration. Variables with local effects (e.g. foot deformity) were stronger predictors of plantar pressure than global features (e.g. body mass, age, gender, or diabetes duration). CONCLUSION:The presence of local deformity was the largest contributing factor to barefoot dynamic plantar pressure in high-risk diabetic patients and should therefore be adequately managed to reduce plantar pressure and ulcer risk. However, a significant amount of variance is unexplained by the models, which advocates the quantitative measurement of plantar pressures in the clinical risk assessment of the patient