Hopkins, Ed

Results in learning theory show that in many games adaptive learning converges to Nash equilibrium, the type of equilibrium most commonly used in studying strategic situations. However, there is a dark side, rarely acknowledged, to the theory. In some other games learning never converges but rather continues to cycle

Edinburgh Research Explorer

     Edinburgh Research Explorer                                      Ed Hopkins discusses Understanding Cyclical Behaviour inStrategic SituationsCitation for published version:Hopkins, E Ed Hopkins discusses Understanding Cyclical Behaviour in Strategic Situations.Link:Link to publication record in Edinburgh Research ExplorerDocument Version:Publisher's PDF, also known as Version of recordPublisher Rights Statement:© Hopkins, E. Ed Hopkins discusses Understanding Cyclical Behaviour in Strategic SituationsGeneral rightsCopyright for the publications made accessible via the Edinburgh Research Explorer is retained by the author(s)and / or other copyright owners and it is a condition of accessing these publications that users recognise andabide by the legal requirements associated with these rights.Take down policyThe University of Edinburgh has made every reasonable effort to ensure that Edinburgh Research Explorercontent complies with UK legislation. If you believe that the public display of this file breaches copyright pleasecontact openaccess@ed.ac.uk providing details, and we will remove access to the work immediately andinvestigate your claim.Download date: 05. Apr. 2019Focus paperMuch of economic theory is based on the idea of rational maximising individuals. Suppose instead individuals follow simple adaptive learning rules. Does this result in the same outcomes, at least in the long run?  In many situations, the answer, perhaps surprisingly, is yes. Results in learning theory show that in many games adaptive learning converges to Nash equilibrium, the type of equilibrium most commonly used in studying strategic situations.  However, there is a dark side, rarely acknowledged, to the theory. In some other games learning never converges but rather continues to cycle. cyclical behaviourRecent work by Ed Hopkins and coauthors specifically addresses instability.  While classic work by the Nobel prize winner Lloyd Shapley had first identified the possibility of a learning process converging to a persistent cycle, much subsequent work has merely treated this as an inconvenience rather than an opportunity. To be fair, it has not been clear what exactly we should expect to see when adaptive play diverges from equilibrium. Nash equilibrium is relatively simple and parsimonious, whereas cyclical behaviour seems both complex and somewhat implausible. For example, in the original work of Shapley, the prediction is a cycle of every increasing length around the exterior of what is now called a “Shapley polygon”. The first paper, “Learning in Games with Unstable Equilibria”, presents new results on cyclical behaviour that are simpler, more plausible and directly testable. The classical model of learning used by Shapley, fictitious play, is modified to allow for individuals placing greater weight on more recent experience, a modification inspired by experimental evidence.  It then follows that cycles in learning that arise when play diverges from equilibrium do not increase in length. Further, if one takes an average of play over time, then this average converges to a particular point, a point we label the TASP, for “Time Average of the Shapley Polygon”. In many cases the TASP is close to the Nash equilibrium.  In others it is quite distinct.These results potentially explain otherwise anomalous results in economics experiments. For example, in experiments on price setting games conducted by Cason and Friedman (2003), it was found that on average across the whole experiment the prices charged on average were very similar to those predicted by Nash equilibrium.  But it was also found that prices followed clearly cyclical movements over time, which is not consistent with play being in equilibrium.   School of Economicswww.econ.ed.ac.ukUnderstanding Cyclical Behaviour in Strategic SituationsEd HopkinsBut both these observations are consistent with the new theory, where play follows a cycle but average behaviour can be close to equilibrium.But this raises a question. If the TASP and Nash equilibrium can be mistaken for each other, is it possible to tell them apart? The second paper, “Testing the TASP: an Experimental Investigation of Learning in Games with Unstable Equilibria”, concerns experiments designed to test this new theory. Participants were randomly matched to play one of two 4 × 4 games each with a unique mixed Nash equilibrium. In one game, the equilibrium is predicted to be stable under learning, and in the other unstable. Both games are versions of Rock–Paper–Scissors with the addition of a fourth strategy, Dumb. The mixed equilibrium in both games is (1, 1, 1, 3)/6: Dumb is thus the most frequent strategy. In the unstable game, however, adaptive learning is predicted to diverge from the equilibrium to a cycle, a “Shapley polygon”, that places no weight upon Dumb.  zero. That is, we find support for the idea that the stability or instability of an equilibrium, as predicted by learning theory, can influence human behaviour. The data also reject Nash equilibrium, which predicts no difference between the treatments. So, the experiments do find support for learning theory over the static equilibrium predictions. However, the actual difference is smaller than the theory predicts. This is not so surprising in that experimental data, and real life, are much noisier than the abstract picture painted in theoretical models. Nonetheless, these abstract models do predict actual human behaviour in these strategic situations.Toy games such as rock-scissors-paper might seem far removed from real economic situations. However, it is important to realise that they are analogous to many problems faced in market settings.  The aim in this experimental work is to start with a simple situation where there is greater control. We now have a much clearer picture of the dynamic behaviour.  Further research already underway will carry out similar experiments in a much more realistic setting. This is illustrated in Figure 1.  The pyramid shape represents possible mixed strategies, with the frequency of the fourth strategy on the vertical.  The Nash equilibrium (N) is in the interior. However, the Shapley polygon is the dashed triangle on the base of the pyramid where the fourth strategy has zero frequency.  The TASP is the average of an orbit around this Shapley polygon and thus must also be on the base of the pyramid. The TASP is given by point T. It is easy to see that the Nash equilibrium and the TASP are quite distinct.Further, if adaptive learning describes agents’ behaviour, the limiting frequency of Dumb is a ready indicator of whether we are in the stable or unstable case. It is also, therefore, a simple way to determine whether the predictions of learning theory hold in practice. Equilibrium theory suggests that the frequency of Dumb should be the same, equal to one half, in both games. Learning theory suggests they should be quite different.In the experiments, we find that there is a difference in play in the unstable game. The frequency of Dumb is lower and play is further from Nash, though the frequency of Dumb is always substantially greater than i “Testing the TASP: an Experimental Investigation of Learning in Games with Unstable Equilibria” by Timothy N. Cason, Daniel Friedman and Ed Hopkins, Journal of Economic Theory, November, 2010, 145, 2309-2331.i “Learning in Games with Unstable Equilibria” by Michel Benaïm, Josef Hofbauer and Ed Hopkins, Journal of Economic Theory, July, 2009, 144, 1694-1709.i “Buyer search and price dispersion: a laboratory study” by Timothy N. Cason and Daniel Friedman, Journal of Economic Theory, 2003, 112, 232-260.Fig. 1 Nash Equilibrium (N) and TASP (T) in the unstable version of the RPSD game. The frequencies of strategies 1 and 2 are on the horizontal axes and of strategy 4 on the vertical axis.Images courtesy of anankkml (cover)and Kittikun Atsawintarangkul (back)FreeDigitalPhotos.net

Ed Hopkins discusses Understanding Cyclical Behaviour in Strategic Situations

https://www.pure.ed.ac.uk/ws/files/19910911/School_FocusPaper12.pdf

Ed Hopkins discusses Understanding Cyclical Behaviour in Strategic Situations

Abstract

Similar works

Full text

Available Versions

Edinburgh Research Explorer