Exchangeable Variable Models

A sequence of random variables is exchangeable if its joint distribution is invariant under variable permutations. We introduce exchangeable variable models (EVMs) as a novel class of probabilistic models whose basic building blocks are partially exchangeable sequences, a generalization of exchangeable sequences. We prove that a family of tractable EVMs is optimal under zero-one loss for a large class of functions, including parity and threshold functions, and strictly subsumes existing tractable independence-based model families. Extensive experiments show that EVMs outperform state of the art classifiers such as SVMs and probabilistic models which are solely based on independence assumptions.Comment: ICML 201

Exogenous impact and conditional quantile functions

An exogenous impact function is defined as the derivative of a structural function with respect to an endogenous variable, other variables, including unobservable variables held fixed. Unobservable variables are fixed at specific quantiles of their marginal distributions. Exogenous impact functions reveal the impact of an exogenous shift in a variable perhaps determined endogenously in the data generating process. They provide information about the variation in exogenous impacts across quantiles of the distributions of the unobservable variables that appear in the structural model. This paper considers nonparametric identification of exogenous impact functions under quantile independence conditions. It is shown that, when valid instrumental variables are present, exogenous impact functions can be identified as functionals of conditional quantile functions that involve only observable random variables. This suggests parametric, semiparametric and nonparametric strategies for estimating exogenous impact functions.

Farm-level Acreage Allocation under Risk

We model the area allocation decision problem for a fixed size crop farm under random yields and prices for a risk-averse farmer. We assume that in the short run, the variable input expenses are fixed per hectare and per crop (an assumption that is motivated by our data). Therefore the cost function depends only on the non-stochastic area allocation. The first order conditions of the model involve integrals across functions of random variables that do not in general have closed form solutions. Numerical simulation techniques are used to calibrate the parameters of the cost function. The two sources of randomness, price and yield, are combined into a single random variable, the yield-in-value. Based on examination of panels of yield-in-value data, we assume independence across the yield-in-value distributions and that the farmers know these distributions. We have modeled the sugar quota constraint, the Common Agricultural Policy subsidies and set-aside, and one Agri-Environmental Measure called "buffer zone".Risk and Uncertainty,

Exogenous impact and conditional quantile functions

An exogenous impact function is defined as the derivative of a structural function with respect to an endogenous variable, other variables, including unobservable variables held fixed. Unobservable variables are fixed at specific quantiles of their marginal distributions. Exogenous impact functions reveal the impact of an exogenous shift in a variable perhaps determined endogenously in the data generating process. They provide information about the variation in exogenous impacts across quantiles of the distributions of the unobservable variables that appear in the structural model. This paper considers nonparametric identification of exogenous impact functions under quantile independence conditions. It is shown that, when valid instrumental variables are present, exogenous impact functions can be identified as functionals of conditional quantile functions that involve only observable random variables. This suggests parametric, semiparametric and nonparametric strategies for estimating exogenous impact functions.endogeneity, quantile regression, identification, structural models, instrumental variables, quantile independence

The distribution of the variance of primes in arithmetic progressions

Hooley conjectured that the variance V(x;q) of the distribution of primes up to x in the arithmetic progressions modulo q is asymptotically x log q, in some unspecified range of q\leq x. On average over 1\leq q \leq Q, this conjecture is known unconditionally in the range x/(log x)^A \leq Q \leq x; this last range can be improved to x^{\frac 12+\epsilon} \leq Q \leq x under the Generalized Riemann Hypothesis (GRH). We argue that Hooley's conjecture should hold down to (loglog x)^{1+o(1)} \leq q \leq x for all values of q, and that this range is best possible. We show under GRH and a linear independence hypothesis on the zeros of Dirichlet L-functions that for moderate values of q, \phi(q)e^{-y}V(e^y;q) has the same distribution as that of a certain random variable of mean asymptotically \phi(q) log q and of variance asymptotically 2\phi(q)(log q)^2. Our estimates on the large deviations of this random variable allow us to predict the range of validity of Hooley's Conjecture.Comment: 26 pages; Modified Definition 2.1, the error term for the variance in Theorem 1.2 and its proo

Generalized Instrumental Variable Models

This paper develops characterizations of identified sets of structures and structural features for complete and incomplete models involving continuous or discrete variables. Multiple values of unobserved variables can be associated with particular combinations of observed variables. This can arise when there are multiple sources of heterogeneity, censored or discrete endogenous variables, or inequality restrictions on functions of observed and unobserved variables. The models generalize the class of incomplete instrumental variable (IV) models in which unobserved variables are singlevalued functions of observed variables. Thus the models are referred to as generalized IV (GIV) models, but there are important cases in which instrumental variable restrictions play no significant role. Building on a definition of observational equivalence for incomplete models the development uses results from random set theory that guarantee that the characterizations deliver sharp bounds, thereby dispensing with the need for case-by-case proofs of sharpness. The use of random sets defined on the space of unobserved variables allows identification analysis under mean and quantile independence restrictions on the distributions of unobserved variables conditional on exogenous variables as well as under a full independence restriction. The results are used to develop sharp bounds on the distribution of valuations in an incomplete model of English auctions, improving on the pointwise bounds available until now. Application of many of the results of the paper requires no familiarity with random set theory

Quicksort, Largest Bucket, and Min-Wise Hashing with Limited Independence

Randomized algorithms and data structures are often analyzed under the assumption of access to a perfect source of randomness. The most fundamental metric used to measure how "random" a hash function or a random number generator is, is its independence: a sequence of random variables is said to be $k$-independent if every variable is uniform and every size $k$ subset is independent. In this paper we consider three classic algorithms under limited independence. We provide new bounds for randomized quicksort, min-wise hashing and largest bucket size under limited independence. Our results can be summarized as follows. -Randomized quicksort. When pivot elements are computed using a $5$-independent hash function, Karloff and Raghavan, J.ACM'93 showed $O ( n \log n)$ expected worst-case running time for a special version of quicksort. We improve upon this, showing that the same running time is achieved with only $4$-independence. -Min-wise hashing. For a set $A$, consider the probability of a particular element being mapped to the smallest hash value. It is known that $5$-independence implies the optimal probability $O (1 /n)$. Broder et al., STOC'98 showed that $2$-independence implies it is $O(1 / \sqrt{|A|})$. We show a matching lower bound as well as new tight bounds for $3$- and $4$-independent hash functions. -Largest bucket. We consider the case where $n$ balls are distributed to $n$ buckets using a $k$-independent hash function and analyze the largest bucket size. Alon et. al, STOC'97 showed that there exists a $2$-independent hash function implying a bucket of size $\Omega ( n^{1/2})$. We generalize the bound, providing a $k$-independent family of functions that imply size $\Omega ( n^{1/k})$.Comment: Submitted to ICALP 201

A Hybrid Asymptotic Expansion Scheme: an Application to Long-term Currency Options ( Revised in April 2008, January 2009 and April 2010; forthcoming in "International Journal of Theoretical and Applied Finance". )

This paper develops a general approximation scheme, henceforth called a hybrid asymptotic expansion scheme for the valuation of multi-factor European path-independent derivatives. Specifically, we apply it to pricing long-term currency options under a market model of interest rates and a general diffusion stochastic volatility model with jumps of spot exchange rates. Our scheme is very effective for a type of models in which there exist correlations among all the factors whose dynamics are not necessarily affine nor even Markovian so long as the randomness is generated by Brownian motions. It can also handle models that include jump components under an assumption of their independence of the other random variables when the characteristic functions for the jump parts can be analytically obtained. Moreover, the hybrid scheme develops Fourier transform method with an asymptotic expansion to utilize closed-form characteristic functions obtainable in parts of a model. Our scheme also introduces a characteristic-function-based Monte Carlo simulation method with the asymptotic expansion as a control variable in order to make full use of analytical approximations by the asymptotic expansion and of closed-form characteristic functions. Finally, a series of numerical examples shows the validity of our scheme.