Generalised Entropy MDPs and Minimax Regret

Bayesian methods suffer from the problem of how to specify prior beliefs. One interesting idea is to consider worst-case priors. This requires solving a stochastic zero-sum game. In this paper, we extend well-known results from bandit theory in order to discover minimax-Bayes policies and discuss when they are practical.Comment: 7 pages, NIPS workshop "From bad models to good policies

Adaptive Regret Minimization in Bounded-Memory Games

Online learning algorithms that minimize regret provide strong guarantees in situations that involve repeatedly making decisions in an uncertain environment, e.g. a driver deciding what route to drive to work every day. While regret minimization has been extensively studied in repeated games, we study regret minimization for a richer class of games called bounded memory games. In each round of a two-player bounded memory-m game, both players simultaneously play an action, observe an outcome and receive a reward. The reward may depend on the last m outcomes as well as the actions of the players in the current round. The standard notion of regret for repeated games is no longer suitable because actions and rewards can depend on the history of play. To account for this generality, we introduce the notion of k-adaptive regret, which compares the reward obtained by playing actions prescribed by the algorithm against a hypothetical k-adaptive adversary with the reward obtained by the best expert in hindsight against the same adversary. Roughly, a hypothetical k-adaptive adversary adapts her strategy to the defender's actions exactly as the real adversary would within each window of k rounds. Our definition is parametrized by a set of experts, which can include both fixed and adaptive defender strategies. We investigate the inherent complexity of and design algorithms for adaptive regret minimization in bounded memory games of perfect and imperfect information. We prove a hardness result showing that, with imperfect information, any k-adaptive regret minimizing algorithm (with fixed strategies as experts) must be inefficient unless NP=RP even when playing against an oblivious adversary. In contrast, for bounded memory games of perfect and imperfect information we present approximate 0-adaptive regret minimization algorithms against an oblivious adversary running in time n^{O(1)}.Comment: Full Version. GameSec 2013 (Invited Paper

Approachability in Stackelberg Stochastic Games with Vector Costs

The notion of approachability was introduced by Blackwell [1] in the context of vector-valued repeated games. The famous Blackwell's approachability theorem prescribes a strategy for approachability, i.e., for `steering' the average cost of a given agent towards a given target set, irrespective of the strategies of the other agents. In this paper, motivated by the multi-objective optimization/decision making problems in dynamically changing environments, we address the approachability problem in Stackelberg stochastic games with vector valued cost functions. We make two main contributions. Firstly, we give a simple and computationally tractable strategy for approachability for Stackelberg stochastic games along the lines of Blackwell's. Secondly, we give a reinforcement learning algorithm for learning the approachable strategy when the transition kernel is unknown. We also recover as a by-product Blackwell's necessary and sufficient condition for approachability for convex sets in this set up and thus a complete characterization. We also give sufficient conditions for non-convex sets.Comment: 18 Pages, Submitted to Dynamic Games and Application

Approachability in Stackelberg Stochastic Games with Vector Costs

The notion of approachability was introduced by Blackwell [1] in the context of vector-valued repeated games. The famous Blackwell's approachability theorem prescribes a strategy for approachability, i.e., for `steering' the average cost of a given agent towards a given target set, irrespective of the strategies of the other agents. In this paper, motivated by the multi-objective optimization/decision making problems in dynamically changing environments, we address the approachability problem in Stackelberg stochastic games with vector valued cost functions. We make two main contributions. Firstly, we give a simple and computationally tractable strategy for approachability for Stackelberg stochastic games along the lines of Blackwell's. Secondly, we give a reinforcement learning algorithm for learning the approachable strategy when the transition kernel is unknown. We also recover as a by-product Blackwell's necessary and sufficient condition for approachability for convex sets in this set up and thus a complete characterization. We also give sufficient conditions for non-convex sets.Comment: 18 Pages, Submitted to Dynamic Games and Application

Market-based capabilities, perceived quality and firm performance

The historical roots of the marketing concept are traceable to the early 1950s (Drucker, 1954). However, the field of strategic marketing did not begin to bloom until late 1980s and begin 1990s. In this period various scholars begin to develop a better and more precise understanding of the marketing concept, its antecedents and consequences (Kohli and Jaworski, 1990; Narver and Slater, 1990). Some even suggest that the intellectual foundation for today’s strategic marketing starts early 1980s with the writings of Day and Wensley (1983, 1988). During the late 1990s and early 2000s, various critics begin to rebel at the widespread use of present conceptualizations of market orientation.1 In this thesis, we argue that the present market orientation conceptualizations are becoming outdated (after more than 15 years). We use herebyWeiner’s (2000, p. 382) philosophical words, that a marketing: “theory, like a cat or a dog, has a life of about 10-12 years, which is the equivalent of around 70-84 years of human existence. Longevity in part depends on the size of the pet (the bigger the theory, the earlier the demise), its level of activity, breed, and so on. At around the age of 10, the theory begins to weaken, does not see things too well, and is unable to adapt to the new circumstances and to the many obstacles in life. It can remember and account for the distant past better than recent events, and it acts with rigidity.” The diminishing attractiveness of the present conceptualizations of the marketing concept lead some researchers to look for or move off into new directions, such as (1) the market-based capabilities perspective, where market orientation only represents one of the components (Day, 1994), and (2) the strategic orientation construct, where market orientation is also incorporated as a dimension (Gatignon and Xuereb, 1997). The first perspective deals with the classification of market-based capabilities, which suggests a balanced perspective of inside-out and outside-in capabilities (e.g., Day 1994; Mizik and Jacobson 2003; Noble, Sinha and Kumar 2002; Slack and Lewis 2003; Srivastava, Fahey and Christensen 2001; Vargo and Lusch 2004; Zwart and Postma, 1998). Although a number of classifications exists, these models 1 Especially the Nordic Schools (i.e., Gummesson and Gr¨onroos) go rather far in their criticism. largely incorporate market-driven, relationship-driven and supply-chain capabilities as relevant market-based resources. Another perspective that gains popularity in recent years is the strategic orientation model. The strategic orientation direction incorporates variables like customer orientation, competitor orientation, technology orientation and relational orientation. This perspective integrates the classical strategic management literature with that of market orientation. Although we do not claim that the classical market orientation movement begins to fully lose its early enthusiasm, energy and adherents, we believe it is a good time to explore, synthesize, integrate and extend the previously mentioned directions. By doing so, we also provide evidence whether firms with (several) strong marketing capabilities are in a better position to satisfy the needs of their customers and shareholders. To investigate the propositions we use a dyadic approach, data generated from both customers of wholesalers and suppliers/wholesalers. Furthermore, we investigate, using several statistical methods, the effectiveness of attempting to develop several marketing capabilities simultaneously. In short, the primary purpose of this study is theory building, extension of previous research in the field of market orientation and applying several recently proposed statistical methods to further explore the developed frameworks. However, this study is not only useful from the point of view of advancement of science in marketing, but also from the point of view of advancing managerial decision making. The results derived from the developed models and proposed methods form an essential piece of information to improve marketing decisions. This enables (top) managers faced with the problem of how to trade off competing strategic marketing initiatives to further optimize their decision-making process.