Osiewalski, J.

English

Research Papers in Economics

C~~~R Discussion
for a er Economic Research
CBM
R
8414
1989
7
I'~IINIIIIIIIIIIIIIIIIIIIIIUIIUIIININIUIiIIINo. 8907
POSTERIOR DENSITIES FOR NONLINEAR
REGRESSION WITH EQUICORRELATED
ERRORS
by J. Osiewalski
February , 1989Posterior Densities for Nonlinear Regression with Equicorrelated Errors
by
Jacek Osiewalski
Academy of Economics
ul. Rakowicka 27
31-510 Kraków
Poland
JEL classification: 211
Keywords: nonlinear models, Bayesian analysis, equicorrelation
Abstract.
For a nonlinear regression model with a constant term, it is shown that -
under diffuse priors of the constant term and of the error precision - the
assumption of equicorrelated errors (instead of uncorrelated ones) has no
new consequences on Bayesian estimation of the (nonlinear) regression
parameters (except for the constant term).
February 1989
Acknowledgement: Comments by Mark Steel are gratefully acknowledged.1
1. Introduction
Main non-Bayesian results concerníng linear regression with an
intercept and with equicorrelated observations were obtained long time ago.
Assuming joint normality of observations, Halperin (1951) showed that
certain estimators and tests of significance used in regression analysis
when observations are independent are equally valid in the case of
equicorrelated observations. McElroy (1969) proved that, in a linear
regression model with an intercept, OLS estimators are BLU if and only if
the errors have the same variances and the same nonnegative coefficient of
correlation between each pair; see also Balestra (1970).
Bayesian results for a linear model with equicorrelated disturbances
are presented in Osiewalski (1987); it is shown that under diffuse priors of
all the regression parameters and of the error variance:
1. the posterior of the correlation parameter is equal to its prior,
2. the marginal posterior of the regression parameters (except for
the constant term) is the same as in the case of uncorrelated
disturbances,
3. the posterior mean of the intercept is unaffected by
equicorrelation, but its posterior variance is seriously affected
and can be infinite for some priors.
The purpose of this paper is to extend these results to a nonlinear
model with a constant term and to general prior assumptions on (nonlinear)
regression parameters other than the constant term.
1.1 Notation
Throughout this note p(.) denotes a probability density function (PDF)
with special notation for PDF's of gamma, normal and t distributions. For
xeRk, pN(xlc,W) denotes a k-variate normal PDF with a mean vector c and a
covariance matrix W, and pS(xlT,c,T) denotes a k-variate Student t PDF with
T degrees of freedom, a noncentrality vector (a mean vector, if T) 1) c and
1
a precision matrix T. For wER~,2
a
PG(w~a.b) - C a wa-1 exp(-bw).
that is a gamma PDF with parameters a) 0, b~ 0.
2. Posterior densities under uncorrelated and equicorrelated errors - a
comparison
We consider two cases of the following (nonlinear) regression model
with normal errors:
Y- h(Z;9) t be t N, y~ ~ N(D.u-1V), (1)
where yERn is observed; gEpCRk-1, SER1 and WER} are unknown parameters; e is
a column vector of ones (thus b is a constant term); Z is a known
(nonrandom) matrix of (nxt) fixed values of T explanatory variables; h(Z;g)
is a known vector function (without a constant term). We assume that the
vector function h(as a function of g only, given Z) is sufficiently well-
behaved to ensure the possibility of integration with respect to g. In order
to save space, we will write hs instead of h(Z;g).
Model ( 1) with a scalar covariance matrix ( i.e. with V-I ) is the
n
first case considered here. Model (1) with equicorrelated errors, that is
with
V - V - (1-p)In t ~oee~ . ~oe ~D~1)~
~
constitutes the second case. In both cases we assume diffuse priors on the
constant term (S) and the precision parameter (u).
A. If V- In, then the conditional data density p(yl Z, g, b, W)is the
following n-variate normal density:
p(YIZ. 8. b. W) - pN (Y~hs 4 be. ~ lIn)-3
Under the prior
P(8. b. u) a u-iP(8). 6E0. bERl. ~eR;,
which uses Jeffreys' rule to express vague prior knowledge about u, we
obtain the following posterior:
n -1
P(8. b. ulY. Z) ~ P(8) W2 exp[- 2(Y-hg-be)'(Y-hg-be)~.
Denoting
d~ - n e' (Y-hD).
sg - (Y-he)'(ln - n ee')ÍY-hg),
we can write the identity
(Y-hg-be)' ÍY-h9-be) - sg ~ n(b-dg)2.
Now it is possible to present the joint posterior density in the form:
n-1
PÍ9. b. ulY. Z) a P(9) s9 2 PU(w1n21. SZ) PN (b~dg. ~~). (2)
From (2) we easily obtain the following marginal and conditional posterior
densities:
n-1
P(9IY.2) oc P(9) sg 2.
P(uIY.Z,9) - PU(m~ nZl. 2R).
P(S~Y.Z.9.w) - PN(b~dg. n~),
P(bIY.Z,B) - oJm P(b~Y,Z.B.c~) P(uIY,Z.B) dc~ - PS(bin-1, dg. n nsl)-
94
In the case of a linear model with diffuse prior, that is when
T- k-1, O- Rk-1, hg - Zg, p(g) - const,
one obtains the well-known posterior results for ~ - [b g']', as in Zellner
(1971) pp. 66-69.
B. If we assume equicorrelated disturbances, then we have one more unknown
parameter, ~[0,1), and the conditional data density p(yIZ, g, b, u, p) is
the following normal density:
P(YIZ. 8, b. u. V) - PN (Y~hg ' be. ~,-1 V~),
V~o - (1-p) In t pee'.
Under the prior
P(g, b. u, ~n) oc u-1 P(g.~) - u-1P(g)P(tolg). ge0. bERl, c~eR~. y,e[G,1).
where p(pIg) is proper (for every gE0). we obtain the following joint
posterior density
n -1
P(g. b, c~. S~IY. Z) a P(g) P(VIg)IVpI-~ u2 exp[- 2(Y-hg-be)~V~-1
(Y-hg-be)].
Now let us take into account that
-1 1 ,
V~ - 1-~ [ In- 1. n-1 ~ ee ],
see e.g. Graybill (1969) p. 172; we have
(Y-hg-be)' V~-1 (y-hg-Se) -
- 11~ { (y-hg-be)' (Y-hg-be) - lt n-1 ~, [(Y-hg-be)'e]2} -- llp sg } 1. n-1 ~ (b-dg)Z.
5
The joint posterior density can be presented as
n -1 ~s
P(9, b. u. pIY. Z) a P(PI9) P(9)IVpI-~ w2 exP[-
2 lgy ~
exp[- 2 lt n~l ~ Íb-dg)2~.
Since the determinant of V~p takes the form
IVf~I - [1 . (n-1)S~~ (1-p)n-1
see e.g. Graybill (1969) p. 1~2, we can write
n-1
P(g. b. u. ~vIY. Z) a P(FIg) P(g) sg 2 PG(uIn21' 2 lgp )
Pl (b d , lt(n-1)S~).
N I 9 nu
or equivalently
P(9.b.u.wlY.Z) - P(9IY.Z) PÍpIY.Z.9) P(~rIY.Z.9.p) P(bIY.Z.9.p,ca)
where
n-1
P(gIY.Z) a P(9) sg Z.
PÍyIY.Z.g) - P(5~~9).
P(t~IY.Z.9,p) - PG(wIn21. 2 s9-~ ).
P(bIY.Z.9.q.~) - P1 (b d 1 } (n-1)F).
N ~ g' n~6
First, let us notice that the conditional posterior of p given g is
identical to the conditional prior of .p (given g); we can learn from the
data about ~ only through prior dependence between ~ and g:
P(PIY. Z) - J~ P(p~9) PÍBIY. Z) dg.
In the case of prior independence, i.e. if p(~olg) - p(p), we also have
posterior independence between ~ and g, and we cannot learn about ~ from the
data at all. Let us also notice that the marginal posterior of g is the same
as in the case of uncorrelated errors, so the form of correlation assumed
here has no influence on the estimation of g, provided, of course, that we
use the same (margínal) prior p(g) in both cases.
The presence of unknown pE [0,1) affects only the posterior densities
of the precision parameter ~ and the constant term b.
In the case of uncorrelated disturbances we have p(c~Iy, Z, g) -
- pG(wIn21, 2 s9- ) and thus, for example, E(WIy, Z, g) - nsl.
~P~ g
In the case of equicorrelation disturbances we have
P(wIY. Z. g) - gfl PGÍwIn21. 2 sg-~ ) P(P~g) dP~
E(WIY. Z. g) ' nsl [1 - E(S~I9)~.
g
The assumption of equicorrelation influences the posterior of b, but not its
mean; we have
P(SIY. Z. 9. P) - oJ~ P(bIY. Z. 8. V. u) PÍwIY. Z. 9. P) dc~ -
- PS (bIn-1, dg, n nsg' l.ln-1 ~).
P(bIY, Z. 8) - gJl PS (bin-1. dg, n nsl' ltln-1 ~) P(íolg) dp.
g
E(bIY. Z. 8) - dg - E(bIY. Z, g, P).where the conditional posterior mean dg is the same as in the case of
uncorrelated errors. The higher moments of b, however, are different and can
be infinite. For example:
E(52~Y. Z, g) - pJl Ln ng3 . 1'in~l)~ t dg] P(plg) dp -
dg } nsn-3 ~OJ~ pl-~8) dp f(n-1) UJ1 1~P(~~g) d~].
and for p(plg) - 1 we have
UJ1 ~ - tm, UJ1 ~ dp - .m.
~ P
Usually, however, the precision ~ and the constant term b are treated as
nuisance parameters. Whenever the model with equicorrelated observations
seems appropriate and the elements of g are the only parameters of interest,
we can rely on the posterior results under assumption of uncorrelated
observations. The same conclusions were reached in Osiewalski (198~), but
only for the case of a linear model with diffuse prior, that is for
hg - Z9. P(g.F) a P(P).
3. Concluding remarks
For a(nonlinear) regression model with a constant term b, with
equicorrelated errors and with diffuse priors of b and of the error
precision W, it is shown that the marginal posterior of g, the vector of the
(nonlinear) regression parameters other than b, is exactly the same as in
the case of uncorrelated errors.
Since normality of errors and nonrandomness of explanatory variables
(Z matrix) were assumed, let us note that both these assumptions can be
relaxed. We could consider random Z and under the assumption of a Baysesian
cut our results would remain wholly valid; for the definition of a Bayesian
cut see Florens and Mouchart (1985). If we assumed, more generally,
elliptical errors instead of normal ones, then only the posterior of the8
error precision ~ would be affected by this change, but the (marginal)
posterior of the rest of the parameters would remain unchanged; see
Osiewalski (1988). On the other hand, the presence of the constant term (b)
in the model and the form of the priors of b and ~(diffuse!) seem to be the
crucial assumptions for obtaining the results presented in the paper.
References
Balestra, P., 1970, On the efficiency of ordinary least-squares in
regression models, Journal of the American Statistical Association 65,
1330-1337.
Florens, J.-P. and M. Mouchart, 1985, Conditioning in dynamic models,
Journal of Time Series Analysis 6, 15-34.
Graybill, F.A., 1969, Introduction to Matrices with Applications in
Statistics (Wadsworth, Belmont).
Halperin, M., 1951, Normal regression theory in the presence of intra-class
correlation, Annals of Mathematical Statistics 22, 573-579.
McElroy, F.W., 1967, A necessary and sufficient condition that ordinary
least-squares estimators be best linear unbiased, Journal of the
American Statistical Association 62, 1302-1304.
Osiewalski, J., 1987, Regresja liniowa z jednakowo skorelowanymi skladnikami
losowymi - ujecie bayesowskie (Linear regression with equicorrelated
errors - Bayesian approach), Przeglad Statystyczny 34, 233-242.
Osiewalski, J., 1988, A note on Bayesian estimation and prediction in
nonlinear regressions with elliptical errors, mimeo.
Zellner, A., 1971, An Introduction to Bayesian Inference in Econometrics
(Wiley, New York).Discussion Paper Series, CentER, Tilburg University, The Netherlands:
No. Author(s) Title
8801 Th. van de Klundert Fiscal Policy and Finite Lives in Interde-
and F. van der Ploeg pendent Economies with Real and Nominal Wage
Rigidity
8802 J.R. Magnus and The Bias oF Forecasts from a First-order
H. Pesaran Autoregression
8803 A.A. Weber The Credibility of Monetary Policies, Policy-
makers' Reputation and the EMS-Hypothesis:
Empirical Evidence from 13 Countries
8804 F. van der Ploeg and Perfect Equilibrium in a Model of Competitive
A.J. de Zeeuw Arms Accumulation
8805 M.F.J. Steel Seemingly Unrelated Regression Equation
Systems under Diffuse Stochastic Prior
Information: A Recursive Analytical Approach
8806 Th. Ten Raa and Secondary Products and the Measurement of
E.N. Wolff Productivity Growth
8807 F. van der Ploeg Monetary and Fiscal Policy in Interdependent
Economies with Capital Accumulation, Death
and Population Growth
8901 Th. Ten Raa and The Choice of Model in the Construction of
P. Kop Jansen Input-Output Coefficient Matrices
8902 Th. Nijman and F. Palm Generalized Least Squares Estimation of
Linear Models Containing Rational Future
Expectations
8903 A. van Soest, Labour Supply, Income taxes and Hours
I. Woittiez, A. Kapteyn Restrictions in The Netherlands
8904 F. van der Ploeg Capital Accumulation, Inflation and Long-
Run Conflict in International Objectives
8905 Th. van de Klundert and Unemployment Persistence and Loss of
A. van Schaik Productive Capacity: a Keynesian approach
8906 A.J. Markink and Dynamic Policy of Linear Models with
F. van der Ploeg Rational Expectations of Future Events:
A Computer Package
890~ J. Osiewalski Posterior Densities for Nonlinear Regression
with Equicorrelated ErrorsP.O. Rnu an~~~ ~nnn i~ Tii RI IR~ TuG nl~Tl-JGRI qNDS
Bibliotheek K. U. Brabant
Niiwi~~~~i~~u~~ir~iu~iii

and Dynamic Policy of Linear Models with F. van der Ploeg Rational Expectations of Future Events: A Computer Package

and The Bias oF Forecasts from a First-order H. Pesaran Autoregression

Conditioning in dynamic models,

de Klundert and Unemployment Persistence and Loss of A. van Schaik Productive Capacity: a Keynesian approach

de Klundert Fiscal Policy and Finite Lives in Interdeand F. van der Ploeg pendent Economies with Real and Nominal Wage Rigidity

der Ploeg and Perfect Equilibrium in a Model of Competitive A.J. de Zeeuw Arms Accumulation

der Ploeg Capital Accumulation, Inflation and LongRun Conflict

der Ploeg Monetary and Fiscal Policy

Generalized Least Squares Estimation of Linear Models Containing Rational Future Expectations

Labour Supply, Income taxes and Hours

On the efficiency of ordinary least-squares in regression models,

Osiewalski Posterior Densities for Nonlinear Regression with Equicorrelated ErrorsP.O. Rnu an~~~ ~nnn i~

Seemingly Unrelated Regression Equation Systems under Diffuse Stochastic Prior Information: A Recursive Analytical Approach

Ten Raa and The Choice of Model in the Construction of P.

The Credibility of Monetary Policies, Policymakers' Reputation and the EMS-Hypothesis: Empirical Evidence from 13 Countries

Posterior Densities for Nonlinear Regression with Equicorrelated Errors

Posterior Densities for Nonlinear Regression with Equicorrelated Errors

Abstract

Similar works

Full text

Available Versions

Research Papers in Economics