Regret in Dynamic Decision Problems

The paper proposes a framework to extend regret theory to dynamic contexts. The key idea is to conceive of a dynamic decision problem with regret as an intra-personal game in which the agent forms conjectures about the behaviour of the various counterfactual selves that he could have been. We derive behavioural implications in situations in which payoffs are correlated across either time or contingencies. In the first case, regret might lead to excess conservatism or a tendency to make up for missed opportunities. In the second case, behaviour is shaped by the agent’s self-conception. We relate our results to empirical evidence

Multicriteria Dynamic Optimization Problems and Cooperative Dynamic Games

We survey some recent research results in the field of dynamic cooperative differential games with non-transferable utilities. Problems which fit into this framework occur for instance if a person has more than one objective he likes to optimize or if several persons decide to combine efforts in trying to realize their individual goals. We assume that all persons act in a dynamic environment and that no side-payments take place. For these kind of problems the notion of Pareto efficiency plays a fundamental role. In economic terms, an allocation in which no one can be made better-off without someone else becoming worseoff is called Pareto efficient. In this paper we present as well necessary as sufficient conditions for existence of a Pareto optimum for general non-convex games. These results are elaborated for the special case that the environment can be modeled by a set of linear differential equations and the objectives can be modeled as functions containing just affine quadratic terms. Furthermore we will consider for these games the convex case. In general there exists a continuum of Pareto solutions and the question arises which of these solutions will be chosen by the participating persons. We will flash some ideas from the axiomatic theory of bargaining, which was initiated by Nash [16, 17], to predict the compromise the persons will reach.Dynamic Optimization;Pareto Efficiency;Cooperative Differential Games;LQ The- ory;Riccati Equations;Bargaining

Benchmark generator for CEC 2009 competition on dynamic optimization

Evolutionary algorithms(EAs) have been widely applied to solve stationary optimization problems. However, many real-world applications are actually dynamic. In order to study the performance of EAs in dynamic environments, one important task is to develop proper dynamic benchmark problems. Over the years, researchers have applied a number of dynamic test problems to compare the performance of EAs in dynamic environments, e.g., the “moving peaks ” benchmark (MPB) proposed by Branke [1], the DF1 generator introduced by Morrison and De Jong [6], the singleand multi-objective dynamic test problem generator by dynamically combining different objective functions of exiting stationary multi-objective benchmark problems suggested by Jin and Sendhoff [2], Yang and Yao’s exclusive-or (XOR) operator [10, 11, 12], Kang’s dynamic traveling salesman problem (DTSP) [3] and dynamic multi knapsack problem (DKP), etc. Though a number of DOP generators exist in the literature, there is no unified approach of constructing dynamic problems across the binary space, real space and combinatorial space so far. This report uses the generalized dynamic benchmark generator (GDBG) proposed in [4], which construct dynamic environments for all the three solution spaces. Especially, in the rea

Reversibility in Dynamic Coordination Problems

Agents at the beginning of a dynamic coordination process (1) are uncertain about actions of their fellow players and (2) anticipate receiving strategically relevant information later on in the process. In such environments, the irreversibility of early actions plays an important role in the choice among them. We characterize the strategic effects of the reversibility option on the coordination outcome. Such an option can either enhance or hamper efficient coordination, and we determine the direction of the effect based only on simple features of the coordination problem. The analysis is based on a generalization of the Laplacian property known from static global games: players at the beginning of a dynamic game act as if they were entirely uninformed about aggregate play of fellow players in each stage of the coordination process.: Delay, Exit, Global Games, Laplacian Belief, Learning, Option, Reversibility.

Dynamic programming approach to principal-agent problems

We consider a general formulation of the Principal-Agent problem with a lump-sum payment on a finite horizon, providing a systematic method for solving such problems. Our approach is the following: we first find the contract that is optimal among those for which the agent's value process allows a dynamic programming representation, for which the agent's optimal effort is straightforward to find. We then show that the optimization over the restricted family of contracts represents no loss of generality. As a consequence, we have reduced this non-zero sum stochastic differential game to a stochastic control problem which may be addressed by the standard tools of control theory. Our proofs rely on the backward stochastic differential equations approach to non-Markovian stochastic control, and more specifically, on the recent extensions to the second order case

Dynamic consistency for Stochastic Optimal Control problems

For a sequence of dynamic optimization problems, we aim at discussing a notion of consistency over time. This notion can be informally introduced as follows. At the very first time step $t_0$, the decision maker formulates an optimization problem that yields optimal decision rules for all the forthcoming time step $t_0, t_1, ..., T$; at the next time step $t_1$, he is able to formulate a new optimization problem starting at time $t_1$ that yields a new sequence of optimal decision rules. This process can be continued until final time $T$ is reached. A family of optimization problems formulated in this way is said to be time consistent if the optimal strategies obtained when solving the original problem remain optimal for all subsequent problems. The notion of time consistency, well-known in the field of Economics, has been recently introduced in the context of risk measures, notably by Artzner et al. (2007) and studied in the Stochastic Programming framework by Shapiro (2009) and for Markov Decision Processes (MDP) by Ruszczynski (2009). We here link this notion with the concept of "state variable" in MDP, and show that a significant class of dynamic optimization problems are dynamically consistent, provided that an adequate state variable is chosen

Fast Dynamic Graph Algorithms for Parameterized Problems

Fully dynamic graph is a data structure that (1) supports edge insertions and deletions and (2) answers problem specific queries. The time complexity of (1) and (2) are referred to as the update time and the query time respectively. There are many researches on dynamic graphs whose update time and query time are $o(|G|)$, that is, sublinear in the graph size. However, almost all such researches are for problems in P. In this paper, we investigate dynamic graphs for NP-hard problems exploiting the notion of fixed parameter tractability (FPT). We give dynamic graphs for Vertex Cover and Cluster Vertex Deletion parameterized by the solution size $k$. These dynamic graphs achieve almost the best possible update time $O(\mathrm{poly}(k)\log n)$ and the query time $O(f(\mathrm{poly}(k),k))$, where $f(n,k)$ is the time complexity of any static graph algorithm for the problems. We obtain these results by dynamically maintaining an approximate solution which can be used to construct a small problem kernel. Exploiting the dynamic graph for Cluster Vertex Deletion, as a corollary, we obtain a quasilinear-time (polynomial) kernelization algorithm for Cluster Vertex Deletion. Until now, only quadratic time kernelization algorithms are known for this problem. We also give a dynamic graph for Chromatic Number parameterized by the solution size of Cluster Vertex Deletion, and a dynamic graph for bounded-degree Feedback Vertex Set parameterized by the solution size. Assuming the parameter is a constant, each dynamic graph can be updated in $O(\log n)$ time and can compute a solution in $O(1)$ time. These results are obtained by another approach.Comment: SWAT 2014 to appea

Explicit memory schemes for evolutionary algorithms in dynamic environments

Copyright @ 2007 Springer-VerlagProblem optimization in dynamic environments has atrracted a growing interest from the evolutionary computation community in reccent years due to its importance in real world optimization problems. Several approaches have been developed to enhance the performance of evolutionary algorithms for dynamic optimization problems, of which the memory scheme is a major one. This chapter investigates the application of explicit memory schemes for evolutionary algorithms in dynamic environments. Two kinds of explicit memory schemes: direct memory and associative memory, are studied within two classes of evolutionary algorithms: genetic algorithms and univariate marginal distribution algorithms for dynamic optimization problems. Based on a series of systematically constructed dynamic test environments, experiments are carried out to investigate these explicit memory schemes and the performance of direct and associative memory schemes are campared and analysed. The experimental results show the efficiency of the memory schemes for evolutionary algorithms in dynamic environments, especially when the environment changes cyclically. The experimental results also indicate that the effect of the memory schemes depends not only on the dynamic problems and dynamic environments but also on the evolutionary algorithm used