Satisficing in multi-armed bandit problems

Satisficing is a relaxation of maximizing and allows for less risky decision making in the face of uncertainty. We propose two sets of satisficing objectives for the multi-armed bandit problem, where the objective is to achieve reward-based decision-making performance above a given threshold. We show that these new problems are equivalent to various standard multi-armed bandit problems with maximizing objectives and use the equivalence to find bounds on performance. The different objectives can result in qualitatively different behavior; for example, agents explore their options continually in one case and only a finite number of times in another. For the case of Gaussian rewards we show an additional equivalence between the two sets of satisficing objectives that allows algorithms developed for one set to be applied to the other. We then develop variants of the Upper Credible Limit (UCL) algorithm that solve the problems with satisficing objectives and show that these modified UCL algorithms achieve efficient satisficing performance.Comment: To appear in IEEE Transactions on Automatic Contro

Online optimal variable charge-rate coordination of plug-in electric vehicles to maximize customer satisfaction and improve grid performance

© 2016 Elsevier B.V. Participation of plug-in electric vehicles (PEVs) is expected to grow in emerging smart grids. A strategy to overcome potential grid overloading caused by large penetrations of PEVs is to optimize their battery charge-rates to fully explore grid capacity and maximize the customer satisfaction for all PEV owners. This paper proposes an online dynamically optimized algorithm for optimal variable charge-rate scheduling of PEVs based on coordinated aggregated particle swarm optimization (CAPSO). The online algorithm is updated at regular intervals of Δt = 5 min to maximize the customers’ satisfactions for all PEV owners based on their requested plug-out times, requested battery state of charges (SOCReq) and willingness to pay the higher charging energy prices. The algorithm also ensures that the distribution transformer is not overloaded while grid losses and node voltage deviations are minimized. Simulation results for uncoordinated PEV charging as well as CAPSO with fixed charge-rate coordination (FCC) and variable charge-rate coordination (VCC) strategies are compared for a 449-node network with different levels of PEV penetrations. The key contributions are optimal VCC of PEVs considering battery modeling, chargers’ efficiencies and customer satisfaction based on requested plug-out times, driving pattern, desired final SOCs and their interest to pay for energy at a higher rate

A model of learning and emulation with artificial adaptive agents

We study adaptive learning behavior in a sequence of n-period endowment overlapping generations economies with fiat currency, where n refers to the number of periods in agents' lifetimes. Agents initially have heterogeneous beliefs and seek to form multi-step-ahead forecasts of future prices using a forecast rule chosen from a vast set of possible forecast rules. Agents take optimal actions given their forecasts of future prices. They learn in every period by creating new forecast rules and by emulating the forecast rules of other agents. Computational experiments with artificial adaptive agents are conducted. These experiments yield three qualitatively different types of outcomes. In one, the initially heterogeneous population of artificial agents learns to coordinate on a low inflation, stationary perfect foresight equilibrium. In another, we observe persistent currency collapse. The third outcome is a lack of coordination within the allotted time frame. One possible outcome, a stationary perfect foresight equilibrium with a relatively high inflation rate, is never observed.Consumption (Economics)

Strengthening Primary and Chronic Care: State Innovations to Transform and Link Small Practices

Presents case studies of state policies for reorganizing and improving primary and chronic care delivery among small practices, including leadership and convening, payment incentives, infrastructure support, feedback and monitoring, and certification

Communication for Teams of Networked Robots

There are a large class of problems, from search and rescue to environmental monitoring, that can benefit from teams of mobile robots in environments where there is no existing infrastructure for inter-agent communication. We seek to address the problems necessary for a team of small, low-power, low-cost robots to deploy in such a way that they can dynamically provide their own multi-hop communication network. To do so, we formulate a situational awareness problem statement that specifies both the physical task and end-to-end communication rates that must be maintained. In pursuit of a solution to this problem, we address topics ranging from the modeling of point-to-point wireless communication to mobility control for connectivity maintenance. Since our focus is on developing solutions to these problems that can be experimentally verified, we also detail the design and implantation of a decentralized testbed for multi-robot research. Experiments on this testbed allow us to determine data-driven models for point-to-point wireless channel prediction, test relative signal-strength-based localization methods, and to verify that our algorithms for mobility control maintain the desired instantaneous rates when routing through the wireless network. The tools we develop are integral to the fielding of teams of robots with robust wireless network capabilities