Randomization tests for peer effects in group formation experiments

Measuring the effect of peers on individual outcomes is a challenging problem, in part because individuals often select peers who are similar in both observable and unobservable ways. Group formation experiments avoid this problem by randomly assigning individuals to groups and observing their responses; for example, do first-year students have better grades when they are randomly assigned roommates who have stronger academic backgrounds? Standard approaches for analyzing these experiments, however, are heavily model-dependent and generally fail to exploit the randomized design. In this paper, we extend methods from randomization-based testing under interference to group formation experiments. The proposed tests are justified by the randomization itself, require relatively few assumptions, and are exact in finite samples. First, we develop procedures that yield valid tests for arbitrary group formation designs. Second, we derive sufficient conditions on the design such that the randomization test can be implemented via simple random permutations. We apply this approach to two recent group formation experiments

The Augmented Synthetic Control Method

The synthetic control method (SCM) is a popular approach for estimating the impact of a treatment on a single unit in panel data settings. The "synthetic control" is a weighted average of control units that balances the treated unit's pre-treatment outcomes as closely as possible. A critical feature of the original proposal is to use SCM only when the fit on pre-treatment outcomes is excellent. We propose Augmented SCM as an extension of SCM to settings where such pre-treatment fit is infeasible. Analogous to bias correction for inexact matching, Augmented SCM uses an outcome model to estimate the bias due to imperfect pre-treatment fit and then de-biases the original SCM estimate. Our main proposal, which uses ridge regression as the outcome model, directly controls pre-treatment fit while minimizing extrapolation from the convex hull. This estimator can also be expressed as a solution to a modified synthetic controls problem that allows negative weights on some donor units. We bound the estimation error of this approach under different data generating processes, including a linear factor model, and show how regularization helps to avoid over-fitting to noise. We demonstrate gains from Augmented SCM with extensive simulation studies and apply this framework to estimate the impact of the 2012 Kansas tax cuts on economic growth. We implement the proposed method in the new augsynth R package

A Statistical View of Learning in the Centipede Game

In this article we evaluate the statistical evidence that a population of students learn about the sub-game perfect Nash equilibrium of the centipede game via repeated play of the game. This is done by formulating a model in which a player's error in assessing the utility of decisions changes as they gain experience with the game. We first estimate parameters in a statistical model where the probabilities of choices of the players are given by a Quantal Response Equilibrium (QRE) (McKelvey and Palfrey, 1995, 1996, 1998), but are allowed to change with repeated play. This model gives a better fit to the data than similar models previously considered. However, substantial correlation of outcomes of games having a common player suggests that a statistical model that captures within-subject correlation is more appropriate. Thus we then estimate parameters in a model which allows for within-player correlation of decisions and rates of learning. Through out the paper we also consider and compare the use of randomization tests and posterior predictive tests in the context of exploratory and confirmatory data analyses

Improved Error Bounds Based on Worst Likely Assignments

Error bounds based on worst likely assignments use permutation tests to validate classifiers. Worst likely assignments can produce effective bounds even for data sets with 100 or fewer training examples. This paper introduces a statistic for use in the permutation tests of worst likely assignments that improves error bounds, especially for accurate classifiers, which are typically the classifiers of interest.Comment: IJCNN 201