Cognitive load and mixed strategies: On brains and minimax

It is well-known that laboratory subjects often do not play mixed strategy equilibrium games according to the equilibrium predictions. In particular, subjects often mix with the incorrect proportions and their actions often exhibit serial correlation. However, little is known about the role of cognition in these observations. We conduct an experiment where subjects play a repeated hide and seek game against a computer opponent programmed to play either a strategy that can be exploited by the subject (a naive strategy) or designed to exploit suboptimal play of the subject (an exploitative strategy). The subjects play with either fewer available cognitive resources (under a high cognitive load) or with more available cognitive resources (under a low cognitive load). While we observe that subjects do not mix in the predicted proportions and their actions exhibit serial correlation, we do not find strong evidence these are related to their available cognitive resources. This suggests that the standard laboratory results on mixed strategies are not associated with the availability of cognitive resources. Surprisingly, we find evidence that subjects under a high load earn more than subjects under a low load. However, we also find that subjects under a low cognitive load exhibit a greater rate of increase in earnings across rounds than subjects under a high load

Cognitive load and mixed strategies: On brains and minimax

It is well-known that laboratory subjects often do not play mixed strategy equilibrium games according to the equilibrium predictions. In particular, subjects often mix with the incorrect proportions and their actions often exhibit serial correlation. However, little is known about the role of cognition in these observations. We conduct an experiment where subjects play a repeated hide and seek game against a computer opponent programmed to play either a strategy that can be exploited by the subject (a naive strategy) or designed to exploit suboptimal play of the subject (an exploitative strategy). The subjects play with either fewer available cognitive resources (under a high cognitive load) or with more available cognitive resources (under a low cognitive load). While we observe that subjects do not mix in the predicted proportions and their actions exhibit serial correlation, we do not find strong evidence these are related to their available cognitive resources. This suggests that the standard laboratory results on mixed strategies are not associated with the availability of cognitive resources. Surprisingly, we find evidence that subjects under a high load earn more than subjects under a low load. However, we also find that subjects under a low cognitive load exhibit a greater rate of increase in earnings across rounds than subjects under a high load

Cognitive load and mixed strategies: On brains and minimax

It is well-known that laboratory subjects often do not play mixed strategy equilibria games according to the theoretical predictions. However, little is known about the role of cognition in these strategic settings. We conduct an experiment where subjects play a repeated hide and seek game against a computer opponent. Subjects play with either fewer available cognitive resources (high cognitive load treatment) or with more available cognitive resources (low cognitive load treatment). Surprisingly, we find some evidence that subjects in the high load treatment earn more than subjects in the low treatment. However, we also find that subjects in the low treatment exhibit a greater rate of increase in earnings across rounds, thus suggesting more learning. Our evidence is consistent with subjects in the low load treatment over-experimenting. Further, while we observe that subjects do not mix in the predicted proportions and that their actions exhibit serial correlation, we do not find strong evidence these are related to their available cognitive resources. This suggests that the standard laboratory deviations from the theoretical predictions are not associated with the availability of cognitive resources. Our results shed light on the extent to which cognitive resources affect (and do not affect) behavior in games with mixed strategy equilibria

Many-agent Reinforcement Learning

Multi-agent reinforcement learning (RL) solves the problem of how each agent should behave optimally in a stochastic environment in which multiple agents are learning simultaneously. It is an interdisciplinary domain with a long history that lies in the joint area of psychology, control theory, game theory, reinforcement learning, and deep learning. Following the remarkable success of the AlphaGO series in single-agent RL, 2019 was a booming year that witnessed significant advances in multi-agent RL techniques; impressive breakthroughs have been made on developing AIs that outperform humans on many challenging tasks, especially multi-player video games. Nonetheless, one of the key challenges of multi-agent RL techniques is the scalability; it is still non-trivial to design efficient learning algorithms that can solve tasks including far more than two agents ($N \gg 2$), which I name by \emph{many-agent reinforcement learning} (MARL\footnote{I use the world of ``MARL" to denote multi-agent reinforcement learning with a particular focus on the cases of many agents; otherwise, it is denoted as ``Multi-Agent RL" by default.}) problems. In this thesis, I contribute to tackling MARL problems from four aspects. Firstly, I offer a self-contained overview of multi-agent RL techniques from a game-theoretical perspective. This overview fills the research gap that most of the existing work either fails to cover the recent advances since 2010 or does not pay adequate attention to game theory, which I believe is the cornerstone to solving many-agent learning problems. Secondly, I develop a tractable policy evaluation algorithm -- $\alpha^\alpha$-Rank -- in many-agent systems. The critical advantage of $\alpha^\alpha$-Rank is that it can compute the solution concept of $\alpha$-Rank tractably in multi-player general-sum games with no need to store the entire pay-off matrix. This is in contrast to classic solution concepts such as Nash equilibrium which is known to be $PPAD$-hard in even two-player cases. $\alpha^\alpha$-Rank allows us, for the first time, to practically conduct large-scale multi-agent evaluations. Thirdly, I introduce a scalable policy learning algorithm -- mean-field MARL -- in many-agent systems. The mean-field MARL method takes advantage of the mean-field approximation from physics, and it is the first provably convergent algorithm that tries to break the curse of dimensionality for MARL tasks. With the proposed algorithm, I report the first result of solving the Ising model and multi-agent battle games through a MARL approach. Fourthly, I investigate the many-agent learning problem in open-ended meta-games (i.e., the game of a game in the policy space). Specifically, I focus on modelling the behavioural diversity in meta-games, and developing algorithms that guarantee to enlarge diversity during training. The proposed metric based on determinantal point processes serves as the first mathematically rigorous definition for diversity. Importantly, the diversity-aware learning algorithms beat the existing state-of-the-art game solvers in terms of exploitability by a large margin. On top of the algorithmic developments, I also contribute two real-world applications of MARL techniques. Specifically, I demonstrate the great potential of applying MARL to study the emergent population dynamics in nature, and model diverse and realistic interactions in autonomous driving. Both applications embody the prospect that MARL techniques could achieve huge impacts in the real physical world, outside of purely video games

International Climate Finance and its Influence on Fairness and Policy

27 p.Besides costs and benefits, fairness aspects tend to influence negotiating parties’ willingness to join an international agreement on climate change mitigation. Fairness is largely considered to improve the prospects of success of international negotiations and hence measures raising fairness perception might – in turn – help to bring about effective cooperative international climate change mitigation. We consider the influences present international support of climate policy in developing countries exerts on fairness perception and how this again might affect international negotiations. In doing so, we distinguish between fairness perception which is based on historical experiences and perception which is based on conjectures about opponents’ intentions. By identifying beneficial components of current support schemes, lessons can be learnt for designing new schemes like the Green Climate Fund

Strategic behavior in risky competitive settings

In the first chapter of this dissertation, I propose a novel and tractable structural model for ascending auctions with both common and private value components in which heterogeneous bidders exhibit loss aversion. Importantly, I find that loss averse bidders bid noticeably lower than risk neutral ones. I also consider a more general framework in which bidders incorporate into their strategies the information of those bidders who are present but decide not to participate after observing the item put up for auction. This results in bidders reducing the aggressiveness of their bids even further. To empirically assess my model, I use data from storage locker auctions in the popular cable TV show Storage Wars, finding that the behavior of most of its bidders is consistent with loss aversion. Thus, I document for the first time the presence of loss aversion in actual ascending auctions. In Chapter 2, I report the results of a (quasi) field experiment in the training grounds of a professional soccer team to check if individuals, when repeatedly facing the same opponents, satisfy the main mixed strategy equilibrium predictions in soccer penalty kicks, a real-life example of strategic play. This is the first time that the implications of mixed strategy equilibria are tested in the field using repeated observations on specific heterogeneous pairs of players, a situation that rarely repeats in real life. In this respect, I also study the effects of the usual practice of treating heterogeneous rivals as if they all came from the same pair because of the lack of repeated observations for specific pairs. In particular, I show that false rejections may arise when heterogeneous pairs are treated as homogeneous and suggest valid aggregate tests that combine statistics from different opponents. My empirical results suggests that the behavior of most soccer players, when repeatedly facing the same opponents, is consistent with equal scoring probabilities across strategies except for the least professional kickers, as well as with serial independence of player's actions. However, I find dependence between the kicker's and goalkeeper's actions. I also find that the least professional goalkeepers tend to replicate each other's actions. In contrast, players do not seem to follow a reinforcement learning model. In the third chapter, I prove the numerical equivalence for general categorical variables between many seemingly unrelated independent tests. Specifically, I prove that the Pearson's independence test in a contingency table is numerically equivalent to the Lagrange Multiplier test in several popular linear and non-linear regression models: the multivariate linear probability model, the conditional and unconditional multinomial model, the multinomial logit and probit models; as well as the overidentifying restrictions test in GMM. Therefore, different researchers using different econometric procedures will reach exactly the same conclusions if they use any of those tests. Additionally, I show that the asymptotically equivalent Likelihood Ratio tests in the non-linear regression models are numerically identical, and that the heteroskedasticity-robust Wald tests in the multivariate linear probability model and GMM coincide with the Wald test in the conditional multinomial model. All these equivalences also apply to tests of serial independence in a discrete Markov chain, which can be regarded as a time series analogue of the multinomial model. Finally, I use these tests to analyze if professional soccer players follow optimal mixed strategies in penalty kicks. For some players, my empirical results are not consistent with equal scoring probabilities across strategies. In contrast, I find that player's actions are serially independent

Regulating competitive balance in sport

Sport is an integral part of popular culture, capturing the hearts and minds of millions of people from every section of society 1. Maintaining competitive balance in sport is essential in preserving its long term survival. A higher uncertainty of outcome in contests is likely to result in greater spectator demand and increased revenues; whereas an imbalanced competition will eventually lead to a decline in attendance and the overall quality of the sport. The regulation of competitive balance in individual sports is difficult to implement, as success relies largely on the athlete’s individual skill and ability 2. In contrast, team sports can be influenced by a number of regulatory measures such as player drafts, salary caps, transfer windows, and player quotas. Analysis into Europe and North America’s regulation of competitive balance reveals a variety of measures used to foster greater equality of opportunity between teams competing with one another in sport. Unfortunately European sports are restricted somewhat, as access to sport is considered to be a social advantage and importance for the integration of citizens, and therefore cannot remain outside the scope of the fundamental principles of free movement. Although the European Union accepts the ‘specificity of sport’, it grants no exception from European Union law for the industry; a principle that has recently been reaffirmed in the decisions of Bosman 3 and Meca-Medina 4. Sports governing bodies must therefore ensure new rules and regulatory measures comply with European Union law; reducing the number of measures available to them

A Restless Embrace of the Past?

The Baltic Defence College’s “Conference on Russia Papers 2022: A Restless Embrace of the Past?” begs a question. How will Russia’s past shape its present and future, both at home and abroad? This volume’s chapters include a wide range of Russian-related topics organized into four main subject groups. The first of these categories is Russian views, with a focus on tendencies toward militarism, Russian understandings of international order, and the effects of COVID-19 on policy. The following subject is about power dynamics and perceptions, with a focus on Russia’s contingent power structures, Russian narratives, and future projections. The third theme centers on the Baltic region’s connection with Russia, investigating Russia’s influence and information warfare from many angles. Finally, the concluding section examines Russian interests around the world, analyzing the position of Belarus', Russia’s options globally, and the potential of a grand vision of Russian foreign policy. The volume concludes by highlighting the challenges of maintaining dialogue in light of recent trends, particularly in the last half decade and especially in the last several months

Radio Resource Allocation in Wireless OFDM Systems

Ph.DDOCTOR OF PHILOSOPH

A Restless Embrace of the Past?

The Baltic Defence College’s “Conference on Russia Papers 2022: A Restless Embrace of the Past?” begs a question. How will Russia’s past shape its present and future, both at home and abroad? This volume’s chapters include a wide range of Russian-related topics organized into four main subject groups. The first of these categories is Russian views, with a focus on tendencies toward militarism, Russian understandings of international order, and the effects of COVID-19 on policy. The following subject is about power dynamics and perceptions, with a focus on Russia’s contingent power structures, Russian narratives, and future projections. The third theme centers on the Baltic region’s connection with Russia, investigating Russia’s influence and information warfare from many angles. Finally, the concluding section examines Russian interests around the world, analyzing the position of Belarus', Russia’s options globally, and the potential of a grand vision of Russian foreign policy. The volume concludes by highlighting the challenges of maintaining dialogue in light of recent trends, particularly in the last half decade and especially in the last several months