On the Re-Solving Heuristic for (Binary) Contextual Bandits with Knapsacks

In the problem of (binary) contextual bandits with knapsacks (CBwK), the agent receives an i.i.d. context in each of the $T$ rounds and chooses an action, resulting in a random reward and a random consumption of resources that are related to an i.i.d. external factor. The agent's goal is to maximize the accumulated reward under the initial resource constraints. In this work, we combine the re-solving heuristic, which proved successful in revenue management, with distribution estimation techniques to solve this problem. We consider two different information feedback models, with full and partial information, which vary in the difficulty of getting a sample of the external factor. Under both information feedback settings, we achieve two-way results: (1) For general problems, we show that our algorithm gets an $\widetilde O(T^{\alpha_u} + T^{\alpha_v} + T^{1/2})$ regret against the fluid benchmark. Here, $\alpha_u$ and $\alpha_v$ reflect the complexity of the context and external factor distributions, respectively. This result is comparable to existing results. (2) When the fluid problem is linear programming with a unique and non-degenerate optimal solution, our algorithm leads to an $\widetilde O(1)$ regret. To the best of our knowledge, this is the first $\widetilde O(1)$ regret result in the CBwK problem regardless of information feedback models. We further use numerical experiments to verify our results.Comment: 43 pages, 2 figures, 1 tabl

On the effects of changing mortality patterns on investment, labour and consumption under uncertainty

In this paper we extend the consumption-investment life cycle model for an uncertain-lived agent, proposed by Richard (1974), to allow for exible labor supply. We further study the consumption, labor supply and portfolio decisions of an agent facing age-dependent mortality risk, as presented by UK actuarial life tables spanning the time period from 1951-2060 (including mortality forecasts). We find that historical changes in mortality produces significant changes in portfolio investment (more risk taking), labour (decrease of hours) and consumption level (shift to higher level) contributing up to 5% to GDP growth during the period from 1980 until 2010

Debt stabilization in a Non-Ricardian economy

In models with a representative infinitely lived household, tax smoothing implies that the steady state of government debt should follow a random walk. This is unlikely to be the case in overlapping generations (OLG) economies, where the equilibrium interest rate may differ from the policy maker's rate of time preference. It may therefore be optimal to reduce debt today to reduce distortionary taxation in the future. In addition, the level of the capital stock in these economies is likely to be suboptimally low, and reducing government debt will crowd in additional capital. Using a version of the Blanchard-Yaari model of perpetual youth, with both public and private capital, we show that it is optimal in steady state for the government to hold assets. However, we also show how and why this level of government assets can fall short of both the level of debt that achieves the optimal capital stock and the level that eliminates income taxes. Finally, we compute the optimal adjustment path to this steady state

Voting over economic plans

We review and provide motivation for a one-sector model of economic growth in which decisions about capital accumulation are made by a political process. If it is possible to commit for at least three periods into the future, then for any feasible consumption plan, there is a perturbation that is majority-preferred to it. Furthermore, plans that minimize the maximum vote that can be obtained against them yield a political business cycle. If it is impossible to commit, voters select the optimal consumption plan for the median voter

Reclaiming the energy of a schedule: models and algorithms

We consider a task graph to be executed on a set of processors. We assume that the mapping is given, say by an ordered list of tasks to execute on each processor, and we aim at optimizing the energy consumption while enforcing a prescribed bound on the execution time. While it is not possible to change the allocation of a task, it is possible to change its speed. Rather than using a local approach such as backfilling, we consider the problem as a whole and study the impact of several speed variation models on its complexity. For continuous speeds, we give a closed-form formula for trees and series-parallel graphs, and we cast the problem into a geometric programming problem for general directed acyclic graphs. We show that the classical dynamic voltage and frequency scaling (DVFS) model with discrete modes leads to a NP-complete problem, even if the modes are regularly distributed (an important particular case in practice, which we analyze as the incremental model). On the contrary, the VDD-hopping model leads to a polynomial solution. Finally, we provide an approximation algorithm for the incremental model, which we extend for the general DVFS model.Comment: A two-page extended abstract of this work appeared as a short presentation in SPAA'2011, while the long version has been accepted for publication in "Concurrency and Computation: Practice and Experience