Save Money or Feel Cozy?: A Field Experiment Evaluation of a Smart Thermostat that Learns Heating Preferences

We present the design of a fully autonomous smart thermostat that supports end-users in managing their heating preferences in a realtime pricing regime. The thermostat uses a machine learning algorithm to learn how a user wants to trade off comfort versus cost. We evaluate the thermostat in a field experiment in the UK involving 30 users over a period of 30 days. We make two main contributions. First, we study whether our smart thermostat enables end-users to handle real-time prices, and in particular, whether machine learning can help them. We find that the users trust the system and that they can successfully express their preferences; overall, the smart thermostat enables the users to manage their heating given real-time prices. Moreover, our machine learning-based thermostats outperform a baseline without machine learning in terms of usability. Second, we present a quantitative analysis of the users’ economic behavior, including their reaction to price changes, their price sensitivity, and their comfort-cost trade-offs. We find a wide variety regarding the users’ willingness to make trade-offs. But in aggregate, the users’ settings enabled a large amount of demand response, reducing the average energy consumption during peak hours by 38%

Designing core-selecting payment rules: a computational search approach

CMMI-1761163 - National Science Foundationhttps://doi.org/10.1287/isre.2022.1108Published versio

Can bounded and self-interested agents be teammates? Application to planning in ad hoc teams

Planning for ad hoc teamwork is challenging because it involves agents collaborating without any prior coordination or communication. The focus is on principled methods for a single agent to cooperate with others. This motivates investigating the ad hoc teamwork problem in the context of self-interested decision-making frameworks. Agents engaged in individual decision making in multiagent settings face the task of having to reason about other agents’ actions, which may in turn involve reasoning about others. An established approximation that operationalizes this approach is to bound the infinite nesting from below by introducing level 0 models. For the purposes of this study, individual, self-interested decision making in multiagent settings is modeled using interactive dynamic influence diagrams (I-DID). These are graphical models with the benefit that they naturally offer a factored representation of the problem, allowing agents to ascribe dynamic models to others and reason about them. We demonstrate that an implication of bounded, finitely-nested reasoning by a self-interested agent is that we may not obtain optimal team solutions in cooperative settings, if it is part of a team. We address this limitation by including models at level 0 whose solutions involve reinforcement learning. We show how the learning is integrated into planning in the context of I-DIDs. This facilitates optimal teammate behavior, and we demonstrate its applicability to ad hoc teamwork on several problem domains and configurations

Error-Bounded Approximations for Infinite-Horizon Discounted Decentralized POMDPs

International audienceWe address decentralized stochastic control problems represented as decentralized partially observable Markov decision processes (Dec-POMDPs). This formalism provides a general model for decision-making under uncertainty in cooperative, decentralized settings, but the worst-case complexity makes it difficult to solve optimally (NEXP-complete). Recent advances suggest recasting Dec-POMDPs into continuous-state and deterministic MDPs. In this form, however, states and actions are embedded into high-dimensional spaces, making accurate estimate of states and greedy selection of actions intractable for all but trivial-sized problems. The primary contribution of this paper is the first framework for error-monitoring during approximate estimation of states and selection of actions. Such a framework permits us to convert state-of-the-art exact methods into error-bounded algorithms, which results in a scalability increase as demonstrated by experiments over problems of unprecedented sizes

Distributed Decision-Theoretic Active Perception for Multi-robot Active Information Gathering

International audienceMultirobot systems have made tremendous progress in exploration and surveillance. However, information gathering tasks remain passive: the agents receive information from their sensors without acting in order to gather it. In this paper, we present a model and an algorithm for active information gathering using the infor-mation relevance concept. In this model, robots explore, assess the relevance, update their beliefs and communicate the appropriate in-formation to relevant robots. To do so, we propose a distributed de-cision process where a robot maintains a belief matrix representing its beliefs and beliefs about the beliefs of the other robots. This deci-sion process uses entropy and Hellinger distance in a reward function to access the relevance of their beliefs and their divergence with the other robots. This model allows to derive a policy for gathering in-formation to make the entropy low and a communication policy to reduce the divergence between robot's beliefs. An experimental sce-nario has been developed for an indoor information gathering mis-sion. Our model has been compared to two different systems : one without communication and one communicating each received ob-servation. The results show that our approach is more efficient than both systems

NOMU: Neural Optimization-based Model Uncertainty

We study methods for estimating model uncertainty for neural networks (NNs) in regression. To isolate the effect of model uncertainty, we focus on a noiseless setting with scarce training data. We introduce five important desiderata regarding model uncertainty that any method should satisfy. However, we find that established benchmarks often fail to reliably capture some of these desiderata, even those that are required by Bayesian theory. To address this, we introduce a new approach for capturing model uncertainty for NNs, which we call Neural Optimization-based Model Uncertainty (NOMU). The main idea of NOMU is to design a network architecture consisting of two connected sub-NNs, one for model prediction and one for model uncertainty, and to train it using a carefully-designed loss function. Importantly, our design enforces that NOMU satisfies our five desiderata. Due to its modular architecture, NOMU can provide model uncertainty for any given (previously trained) NN if given access to its training data. We evaluate NOMU in various regressions tasks and noiseless Bayesian optimization (BO) with costly evaluations. In regression, NOMU performs at least as well as state-of-the-art methods. In BO, NOMU even outperforms all considered benchmarks