Can bounded and self-interested agents be teammates? Application to planning in ad hoc teams

Planning for ad hoc teamwork is challenging because it involves agents collaborating without any prior coordination or communication. The focus is on principled methods for a single agent to cooperate with others. This motivates investigating the ad hoc teamwork problem in the context of self-interested decision-making frameworks. Agents engaged in individual decision making in multiagent settings face the task of having to reason about other agents’ actions, which may in turn involve reasoning about others. An established approximation that operationalizes this approach is to bound the infinite nesting from below by introducing level 0 models. For the purposes of this study, individual, self-interested decision making in multiagent settings is modeled using interactive dynamic influence diagrams (I-DID). These are graphical models with the benefit that they naturally offer a factored representation of the problem, allowing agents to ascribe dynamic models to others and reason about them. We demonstrate that an implication of bounded, finitely-nested reasoning by a self-interested agent is that we may not obtain optimal team solutions in cooperative settings, if it is part of a team. We address this limitation by including models at level 0 whose solutions involve reinforcement learning. We show how the learning is integrated into planning in the context of I-DIDs. This facilitates optimal teammate behavior, and we demonstrate its applicability to ad hoc teamwork on several problem domains and configurations

Approximating behavioral equivalence for scaling solutions of I-DIDs

Interactive dynamic influence diagram (I-DID) is a recognized graphical framework for sequential multiagent decision making under uncertainty. I-DIDs concisely represent the problem of how an individual agent should act in an uncertain environment shared with others of unknown types. I-DIDs face the challenge of solving a large number of models that are ascribed to other agents. A known method for solving I-DIDs is to group models of other agents that are behaviorally equivalent. Identifying model equivalence requires solving models and comparing their solutions generally represented as policy trees. Because the trees grow exponentially with the number of decision time steps, comparing entire policy trees becomes intractable, thereby limiting the scalability of previous I-DID techniques. In this article, our specific approaches focus on utilizing partial policy trees for comparison and determining the distance between updated beliefs at the leaves of the trees. We propose a principled way to determine how much of the policy trees to consider, which trades off solution quality for efficiency. We further improve on this technique by allowing the partial policy trees to have paths of differing lengths. We evaluate these approaches in multiple problem domains and demonstrate significantly improved scalability over previous approaches

An ontological knowledge and multiple abstraction level decision support system in healthcare

The rationalization of the healthcare processes and organizations is a task of fundamental importance to grant both the quality and the standardization of healthcare services, and the minimization of costs. Clinical Practice Guidelines (CPGs) are one of the major tools that have been introduced to achieve such a challenging task. CPGs are widely used to provide decision support to physicians, supplying them with evidence-based predictive and prescriptive information about patients' status and treatments, but usually on individual pathologies. This sets up the urgent need for developing decision support methodologies to assist physicians and healthcare managers in the detection of interactions between guidelines, to help them to devise appropriate patterns of treatment for comorbid patients (i.e., patients affected by multiple diseases). We identify different levels of abstractions in the analysis of interactions, based on both the hierarchical organization of clinical guidelines (in which composite actions are refined into their components) and the hierarchy of drug categories. We then propose a general methodology (data/knowledge structures and reasoning algorithms operating on them) supporting user-driven and flexible interaction detection over the multiple levels of abstraction. Finally, we discuss the impact of the adoption of computerized clinical guidelines in general, and of our methodology in particular, for patients (quality of the received healthcare services), physicians (decision support and quality of provided services), and healthcare managers and organizations (quality and optimization of provided services)

The dissociation of the fluid and particle phase in the forestomach as a physiological characteristic of large grazing ruminants: an evaluation of available, comparable ruminant passage data

Whether differences in digestive physiology exist between different ruminant feeding types has been an ongoing debate. In this regard, potential differences in ingesta retention have been understood to be of particular importance. We analyzed a data pool in which only mean retention time (MRT) data for the ruminoreticulum (RR) were collated that were obtained using comparable techniques with either chromium or cobalt EDTA as a fluid marker and/or with chromium-mordanted fiber of less than 2 mm in size as a particle marker. Data were compared using one averaged value per species. In general, the paucity of species in such a collection is striking and does not allow—in contrast to earlier statements—any final conclusions regarding the influence of body weight (BW) or feeding type on ruminant MRTs. In particular, there was no significant correlation between MRTparticlesRR or MRTfluidRR and BW, neither in the interspecific nor in the intraspecific comparisons, and no difference between the feeding types. The trend that indicates longer MRTparticlesRR in grazers is based on too few species to be conclusive. Small browsers seemed to have shorter MRTfluidRR than similar-sized grazers. In contrast, there was a trend for large grazers to have shorter MRTfluidRR than large browsers. In direct pair-wise comparisons between cattle and the browsers giraffe, moose, and okapi, the latter difference was significant. Cattle also had the highest relative RR fluid outflow rates among the species investigated. This is in accord with the observation that grazers have larger omasa, a major function of which is water-reabsorption distal to the RR. Grazers seem to have longer MRTparticlesRR per unit MRTfluidRR, and cattle are particular outliers in this respect. It is hypothesized that potentially shorter MRTfluidRR in large grazers and higher relative outflow rates are linked to a higher saliva production and a lesser viscosity of both saliva and RR fluids. A constant supply of a fluid phase of low viscosity is proposed to be the prerogative for the physical mechanisms of flotation and sedimentation that result in the stratification of RR contents and its selective particle retention typical for large grazing species