Sung Jae Jun

Tony Lancaster

English

Research Papers in Economics

Bayesian Quantile Regression
Tony Lancaster1 and Sung Jae Jun
Brown University
December 2005
1. Introduction:
Recent work by Schennach(2005) has opened the way to a Bayesian treat-
ment of quantile regression. Her method, called Bayesian exponentially tilted
empirical likelihood (BETEL), provides a likelihood for data y subject only to a
set of m moment conditions of the form Eg(y,θ)=0where θ is a k dimensional
parameter of interest and k may be smaller, equal to or larger than m. The
method may be thought of as construction of a likelihood supported on the n
data points that is minimally informative, in the sense of maximum entropy,
subject to the moment conditions. Speciﬁcally the probabilities {pi} attached
to the n data points are chosen to solve
max
p Σn
i=1 − pi logpi subject to Σn
i=1pi =1and Σn
i=1pig(yi,θ)=0 (1)
The solutions of this problem, well known in the maximum entropy litera-
ture, e.g. Jaynes(2003, p.357), take the form
pi(θ)=
exp{λ(θ)0g(yi,θ)}
Σn
j=1 exp{λ(θ)0g(yi,θ)}
(2)
where m vector λ, dependent on θ, satisﬁes
λ(θ)=a r gm i n
η n−1Σn
i=1 exp{η0g(yi,θ)}. (3)
The {λi} are the Lagrange multipliers corresponding to the m constraints in
the problem (1). For every θ (3) is a convex minimization problem and compu-
tationally straightforward.
The resulting likelihood for i.i.d data is Πn
i=1pi(θ) and this may be combined
with a prior density on θ to yield the posterior density
p(θ|Y )=p(θ)Πn
i=1pi(θ) (4)
on a support such that 0 is in the convex hull of the g(yi,θ).
In this paper we explore the application of this method to the case where
the moments g(yi,θ) correspond to those of quantiles or quantile regression
functions.
We ﬁrst consider the quantiles of a random variable and give an explicit
form for the posterior distributions and a comparison with the Bayesian boot-
strap posterior. We then consider posterior distributions for quantile regres-
sion functions. Finally we consider the model studied by Chernozhukov and
1Correspondence to ﬁrst author, Department of Economics, Brown University, Providence
RI 02912; Tony_Lancaster@brown.edu
1Hansen(2004) of quantile regression with endogenous regressors. We show that
the Schennach approach leads to an alternative way to do inference about struc-
tural quantile functions than that proposed by these authors. For each class of
models we consider how to construct marginal posterior densities.
2. Quantiles:
Consider ﬁrst Bayesian inference about quantiles. The τ0th quantile θτ sat-
isﬁes the moment restriction E(g)=0where
g =1 ( y ≤ θτ) − τ
where 1(.) is the indicator function. Given a random sample of size n and for any
θ in [miny,maxy) the Lagrange multiplier λ solving the problem (3) satisﬁes
the equation
Σn
i=1gieλgi =0
with solution
eλ(θ) =
τn0
(1 − τ)n1
where n1 = ](yi ≤ θτ) and n0 = n − n1.
Substituting this solution into the expression for the posterior density (4) and
assuming a uniform prior gives
p(θτ|y) ∝
φ
n1
n
n1
1 n
n0
0
, φ =
τ
1 − τ
. (5)
This is a piecewise constant density supported on [min(yi),max(yi)).
The density (5) may be sampled as follows. There are n − 1 pieces, if the
observations are all distinct, forming a partition of the interval from min(yi) to
max(yi). Sample each piece according to its probabilitity. If the sampled piece
is bounded by y(j) and y(j+1) sample a random variable uniformly distributed
on this interval.
Figure 1 shows the posterior density of the median from a random sample
of size n =4 0using a uniform prior. The step function (red) is (5) and the
continuous curve (blue) is the posterior density corresponding to a double
exponential density of the form
f(y|θ) ∝ exp{−|y − θ|}.
The vertical line shows the sample median. Both curves have been normalized
to integrate to one. Posterior distributions are always piecewise constant, as in
the ﬁgure. Highest posterior density intervals with (possibly approximate) 95%
etc. probability content are straightforward to construct.
3. Comparison with the Bayesian Bootstrap:
The Bayesian bootstrap of Rubin(1981) takes the data to be iid multinomial
with probabilities {pi}. An improper Dirichlet prior on these probabilities leads
2Figure 1:
3to a Dirichlet posterior that assigns positive probability only to the distinct
sample observations and in this respect is similar to BETEL. Paremeters such
as quantiles that can be expressed as functionals of the data distribution have
posterior distributions that can be calculated by repeatedly simulating from the
posterior distribution of the {pi} and calculating the parameter of interest. As
Chamberlain and Imbens(2003) point out, the posterior distribution of quantile
regression parameters can be simulated by repeatedly solving the problem
θ =a r gm i n
t Σn
i=1cτ(Viyi − t0Vixi), {Vi} ∼ iid E(1). (6)
Here cτ is the check function deﬁned by cτ(u)=2 u(τ − I(u<0)).T h i s
problem may be solved using Koenker’s quantile regression R function rq(y~x,
tau, weights=..) by letting the weights be n unit exponentials.
To compare the Bayesian bootstrap and BETEL (uniform prior) posteriors
we consider inference about the median — τ =0 .5 —u s i n gas a m p l en = 500
standard normal variates. To get the BB posterior we solved the problem (6)
10,000 times and drew the histogram of the realizations. This is shown in ﬁgure
(2). Note the sparsity of the Bayeian bootstrap distribution which reﬂects the
fact that there were only 162 distinct realizations among the 10,000 draws even
though the sample size was 500. This arises because the criterion function in (6)
is a piecewise linear function with knots at the data points so solutions of the
problem will always lie at one of the data points2. Hence there can be at most
n points of support for the Bayesian bootstrap distribution and with n = 500
most of these will have probability so low that they will not occur in a sample of
10,000 realizations. By contrast, the BETEL distibution, shown in red, provides
positive probability density over the relevant interval.3
It seems that the Bayesian bootstrap provides a less appealing posterior than
Schennach’s method in this application. This is in addition to to its other well
known drawbacks.
4. Regression Quantiles:
The τ0th quantile regression is such that
Pr(Y ≤ α(τ)+β(τ)X|X)=τ
and so satisﬁes the moment conditions
E(1(Y ≤ α(τ)+β(τ)X) − τ|X)=0
E(X1(Y ≤ α(τ)+β(τ)X) − τ|X)=0
If we now deﬁne
g1i =1 ( yi ≤ α + βxi) − τ
and g2i = xi(1(yi ≤ α + βxi) − τ)
2This assumes that if there is a ﬂat section at the minimum the solution is chosen as one
of the two data points between which the function is ﬂat.
3These BB realizations were computed in R according to the program g <- rexp(n); g <-
g/sum(g); coef(rq(y ~1, weights = g)). The sample median was 0.078.
4Figure 2:
5we may compute the Lagrange multipliers λ1 and λ2 by
λ =a r gm i n
η Σn
i=1 exp{η1g1i + η2g2i}
and then calculate the posterior density according to (2).
Figure 3 plots the joint posterior density of α(0.5) and β(0.5) from a sample
of n = 111 observations and under a uniform prior. The data are Graddy’s(1995)
ﬁsh market observations with Y as log quantity traded and X as log price. To
construct ﬁgure 3 the density was evaluated on a 100 × 100 grid of α,β values.
As the ﬁgure shows the density consists of adjoining ﬂat surfaces. The marginal
densities of α and β may be calculated by summing this grid across rows or
across columns. Figure (4) shows the marginal density of the price elasticity of
demand at the 0.5 quantile found in this way. The quantile regression estimate
of β(0.5) using the R program previously mentioned was −0.414 which can be
seen to be close to the marginal posterior mode.
4. Quantiles With Endogenous Covariates.
Quantile regression applied to the observations on price and quantity neglects
the simultaneity of these variables when the market is in equilibrium. This can
be surmounted by use of instrumental variables.
Recall that if the τ0th quantile is denoted α(τ) then we can represent our
random variable as
Y = α(U),U ∼ Uniform(0,1), α(.) strictly increasing,
since
Pr(Y ≤ α(τ)) = Pr(α(U) ≤ α(τ)) = Pr(U ≤ τ)=τ.
and so α(τ) is the quantile function.
Similarly, a regression quantile can be represented by
Y = α(U)+β(U)X, U|X ∼ Uniform(0,1), α(τ)+β(τ)X strictly increasing in τ.
with τ0th conditional quantile equal to α(τ)+β(τ)X.
Following Chernozhukov and Hansen(2004) consider the model
Y = D0α(U)+X0β(U),U |X,Z ∼ Uniform(0,1)
in which D is statistically dependent on U, D0α(τ)+X0β(τ) is strictly increasing
in τ, and Z is a set of instrumental variables that are independent of U but
statistically dependent on D.T h e nD0α(τ)+X0β(τ) is the τ0th quantile of Y
conditional on X,Z. That is,
Pr(Y ≤ D0α(τ)+X0β(τ)|X,Z)=τ (7)
The expression
D0α(τ)+X0β(τ)
4This was computed using rq(Q~P) in R, where Q and P are logarithms of quantity and
price.
6Figure 3:
7Figure 4:
8is what Chernozhukov and Hansen refer to as the structural quantile function.
The fact (6) then leads to conditional moments of which the simplest are of the
form
X0
i(1(yi ≤ D0
iα(τ)+X0
iβ(τ)) − τ)
Z0
i(1(yi ≤ D0
iα(τ)+X0
iβ(τ)) − τ)
Example 1: Simulated Data: In the following example the data were
generated with X null and eight instruments which are the columns of Z. The
Y data was generated by
yi = α0(U)+Diα1(U),
Di = γ0 + Ziγ1 + Vi,
α0(U),V ∼ N([0,0],[1,0.8,0.8,1]
τ was set equal 0.25;n = 500; and for the ﬁrst experiment the eight elements of
γ1 were set equal to 1 and in the second they were set equal to 0.1 The latter
choice was intended to represent weak instruments. Figures (5) and (6) show the
joint posterior densities of α0(0.25) and α1(0.25), the slope and intercept of the
strucrtural quantile function at the 0.25 quantile. Figure (4), with γ1 =1shows
a well behaved joint posterior density centered round the truth. Figure (5), with
weaker instruments shows a thicker tailed distribution with apparent multiple
modes. Further experiments not shown here show that as the coeﬃcients on the
instruments approach zero the joint density shows many modes.
Example 2: Demand for Fish: In this example we again use Graddy’s
data, also used by Chernozhukov and Hansen. Speciﬁcally, we use 111 observa-
tions on quantities of ﬁsh traded and their price. We also use observations on
two weather variables which might be supposed to aﬀect the supply of ﬁsh but
not the demand. These are called ”stormy” and ”mixed”.
The tau’th structural quantile function is
Q = α0(τ)+α1(τ)P
and this is estimated using three moment equations corresponding to stormy,
mixed and the unit variable. Figure (6 ) shows the joint posterior density of
α0(0.5) and α1(0.5). The density was evaluated on a 40×40 grid. It can be seen
that there is only limited evidence of weak instruments in the suggestion of a
secondary mode. The marginal densities can be found by summing over rows
or columns and renormalising. The marginal density of the slope — elasticity of
demand — is shown in ﬁgure (7). An approximate 95% highest posterior density
interval runs from 0.1 to -2.5 and is marked in red. Chernozhukov and Hansen
report a point estimate of −0.9 (marked in blue) using these same instruments
with a 95% conﬁdence interval running from 0 to −1.8.N o t et h em i n o rm o d e
in the marginal posterior density.
5. Conclusions
9Figure 5:
10Figure 6:
11Figure 7:
12Figure 8:
13Schennach’s method represents a way of doing Bayesian inference about
quantile regression models without using a potentially restrictive parametric
likelihood. When the number of parameters of interest is small, as in the il-
lustrations used in this paper, marginal posterior distributions can be easily
evaluated by evaluating the joint posterior density on a grid, summing over the
unwanted dimensions and renormalizing if necessary. When the dimensionality
of the parameter of interest is larger it is likely to be preferable to use MCMC
methods. In initial experiments not reported here we have used a Metropo-
lis/Hasting sampler with a proposal dist r i b u t i o nf o r m e df r o ma no v e r d i s p e r s e d
version of the asymptotic multivariate normal posterior density. This appears to
be satisfactory except when identiﬁcation is very weak and the posterior density
has multiple modes. Results on this point will be given in future versions of this
paper.
14References
Chamberlain, G. and G. W. Imbens Nonparametric applications of Bayesian
inference, Journal of Business and Economic Statistics, 2003, 21, 1, 12-18.
Chernozhukov, V. and C. Hansen Instrumental variable quantile regression,
unpublished ms, version of December 2004.
Graddy, K. Testing for imperfect competition in the Fulton ﬁsh market,R a n d
Journal of Economics, 1995 26(1),75-92.
Jaynes, E. T. Probability theory:the logic of science Cambridge University
Press 2003.
Rubin, D. The Bayesian bootstrap, The Annals of Statistics, 1981, 9, 130-
134.
Schennach, S. M. Bayesian exponentially tilted empirical likelihood,B i o -
metrika 2005 92 (1), 31-46.
15

Baysian Quantile Regression

http://www.brown.edu/Departments/Economics/Papers/2006/2006-05_paper.pdf

Baysian Quantile Regression

Abstract

Similar works

Full text

Available Versions

Research Papers in Economics