Can bounded and self-interested agents be teammates? Application to planning in ad hoc teams

Planning for ad hoc teamwork is challenging because it involves agents collaborating without any prior coordination or communication. The focus is on principled methods for a single agent to cooperate with others. This motivates investigating the ad hoc teamwork problem in the context of self-interested decision-making frameworks. Agents engaged in individual decision making in multiagent settings face the task of having to reason about other agents’ actions, which may in turn involve reasoning about others. An established approximation that operationalizes this approach is to bound the infinite nesting from below by introducing level 0 models. For the purposes of this study, individual, self-interested decision making in multiagent settings is modeled using interactive dynamic influence diagrams (I-DID). These are graphical models with the benefit that they naturally offer a factored representation of the problem, allowing agents to ascribe dynamic models to others and reason about them. We demonstrate that an implication of bounded, finitely-nested reasoning by a self-interested agent is that we may not obtain optimal team solutions in cooperative settings, if it is part of a team. We address this limitation by including models at level 0 whose solutions involve reinforcement learning. We show how the learning is integrated into planning in the context of I-DIDs. This facilitates optimal teammate behavior, and we demonstrate its applicability to ad hoc teamwork on several problem domains and configurations

Approximating behavioral equivalence for scaling solutions of I-DIDs

Interactive dynamic influence diagram (I-DID) is a recognized graphical framework for sequential multiagent decision making under uncertainty. I-DIDs concisely represent the problem of how an individual agent should act in an uncertain environment shared with others of unknown types. I-DIDs face the challenge of solving a large number of models that are ascribed to other agents. A known method for solving I-DIDs is to group models of other agents that are behaviorally equivalent. Identifying model equivalence requires solving models and comparing their solutions generally represented as policy trees. Because the trees grow exponentially with the number of decision time steps, comparing entire policy trees becomes intractable, thereby limiting the scalability of previous I-DID techniques. In this article, our specific approaches focus on utilizing partial policy trees for comparison and determining the distance between updated beliefs at the leaves of the trees. We propose a principled way to determine how much of the policy trees to consider, which trades off solution quality for efficiency. We further improve on this technique by allowing the partial policy trees to have paths of differing lengths. We evaluate these approaches in multiple problem domains and demonstrate significantly improved scalability over previous approaches

The dissociation of the fluid and particle phase in the forestomach as a physiological characteristic of large grazing ruminants: an evaluation of available, comparable ruminant passage data

Whether differences in digestive physiology exist between different ruminant feeding types has been an ongoing debate. In this regard, potential differences in ingesta retention have been understood to be of particular importance. We analyzed a data pool in which only mean retention time (MRT) data for the ruminoreticulum (RR) were collated that were obtained using comparable techniques with either chromium or cobalt EDTA as a fluid marker and/or with chromium-mordanted fiber of less than 2 mm in size as a particle marker. Data were compared using one averaged value per species. In general, the paucity of species in such a collection is striking and does not allow—in contrast to earlier statements—any final conclusions regarding the influence of body weight (BW) or feeding type on ruminant MRTs. In particular, there was no significant correlation between MRTparticlesRR or MRTfluidRR and BW, neither in the interspecific nor in the intraspecific comparisons, and no difference between the feeding types. The trend that indicates longer MRTparticlesRR in grazers is based on too few species to be conclusive. Small browsers seemed to have shorter MRTfluidRR than similar-sized grazers. In contrast, there was a trend for large grazers to have shorter MRTfluidRR than large browsers. In direct pair-wise comparisons between cattle and the browsers giraffe, moose, and okapi, the latter difference was significant. Cattle also had the highest relative RR fluid outflow rates among the species investigated. This is in accord with the observation that grazers have larger omasa, a major function of which is water-reabsorption distal to the RR. Grazers seem to have longer MRTparticlesRR per unit MRTfluidRR, and cattle are particular outliers in this respect. It is hypothesized that potentially shorter MRTfluidRR in large grazers and higher relative outflow rates are linked to a higher saliva production and a lesser viscosity of both saliva and RR fluids. A constant supply of a fluid phase of low viscosity is proposed to be the prerogative for the physical mechanisms of flotation and sedimentation that result in the stratification of RR contents and its selective particle retention typical for large grazing species