In a recent paper Yang and Stufken [Ann. Statist. 40 (2012a) 1665-1685] gave
sufficient conditions for complete classes of designs for nonlinear regression
models. In this note we demonstrate that there is an alternative way to
validate this result. Our main argument utilizes the fact that boundary points
of moment spaces generated by Chebyshev systems possess unique representations.Comment: Published in at http://dx.doi.org/10.1214/13-AOS1108 the Annals of
  Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical
  Statistics (http://www.imstat.org

Dette, Holger

Schorning, Kirsten

English

arXiv

Holger Dette

Kirsten Schorning

CiteSeerX

Complete classes of designs for nonlinear regression models and principal representations of moment spaces

In a recent paper Yang and Stufken (2012a) gave sufficient conditions for complete
classes of designs for nonlinear regression models. In this note we demonstrate that
their result is a simple consequence of the fact that boundary points of moment spaces
generated by Chebyshev systems possess unique representations

Eldorado - Repository of the TU Dortmund

SFB 823 Complete classes of designs for nonlinear regression models and principal representations of moment spaces Discussion Paper  Holger Dette, Kirsten Schorning      Nr. 55/2012        Complete classes of designs for nonlinear regressionmodels and principal representations of moment spacesHolger Dette, Kirsten SchorningRuhr-Universita¨t BochumFakulta¨t fu¨r Mathematik44780 Bochum, Germanye-mail: holger.dette@rub.deNovember 16, 2012AbstractIn a recent paper Yang and Stufken (2012a) gave sufficient conditions for completeclasses of designs for nonlinear regression models. In this note we demonstrate thattheir result is a simple consequence of the fact that boundary points of moment spacesgenerated by Chebyshev systems possess unique representations.Keywords and Phrases: locally optimal design; admissible design; Chebyshev system; prin-ciple representations; moment spaces; complete classes of designsAMS Subject Classification: 62K051 IntroductionThe construction of locally optimal designs for nonlinear regression models has found con-siderable interest in recent years [see for example He et al. (1996), Dette et al. (2006), Khuriet al. (2006), Fang and Hedayat (2008), Yang and Stufken (2012b) among others]. Whilemost of the literature focuses on specific models or specific optimality criteria, general resultscharacterizing the structure of locally optimal designs are extremely difficult to obtain dueto the complicated structure of the corresponding nonlinear optimization problems. In aseries of remarkable papers Yang and Stufken (2009), Yang (2010), Dette and Melas (2011)and Yang and Stufken (2012a) derived several complete classes of designs with respect to theLoewner Ordering of the information matrices. The first paper in this direction of Yang andStufken (2009) investigates nonlinear regression models with two parameters. These results1were generalized by Yang (2010) and Dette and Melas (2011) to identify small completeclasses for nonlinear regression models with more than two parameters. The most generalcontribution is the recent paper of Yang and Stufken (2012a), which provides a sufficientcondition for a complete class of designs and is applicable to most of the commonly usedregression models. The proof of this statement is complicated and requires several auxiliaryresults.The purpose of the present paper is to demonstrate that conditions of this type are inti-mately related to the characterization of boundary points of moment spaces associated witha nonlinear regression model. Our main tool is a Chebyshev system [Karlin and Studden(1966)] appearing in (a transformation of) the Fisher Information matrix of a given design.The complete class of designs can essentially be characterized as the set of measures cor-responding to the lower and upper principal representation of the boundary points of thecorresponding moment spaces. With this insight the main result in the paper of Yang andStufken (2012a) is a simple consequence of the fact that a representation of a boundary pointof a k + 1-dimensional moment space associated with a Chebyshev system depends only onthe first k functions which are used to generate the moment space.In Section 2 we present some facts on moment spaces associated with Chebyshev systemswhich are of general interest for constructing admissible designs. The design problem andTheorem 1 of Yang and Stufken (2012a) are stated in Section 3, where we also present ouralternative proof. We finally note that the paper of Yang and Stufken (2012a) containsnumerous interesting examples and provided a further result which are not discussed in thisnote for the sake of brevity.2 Chebyshev systems and associated moment spacesA set of k real valued functions Ψ0, . . . ,Ψk−1 : [A,B] → R is called Chebychev system onthe interval [A,B] if and only if it fulfills the inequalitydet Ψ0(x0) . . . Ψ0(xk−1)... . . . ...Ψk−1(x0) . . . Ψk−1(xk−1) > 0for any points x0, . . . , xk−1 with A ≤ x0 < x1 . . . < xk−1 ≤ B. The moment space associatedwith a Chebychev system is defined byMk−1 ={c = (c0, . . . , ck−1)T∣∣∣ ci = ∫ BAΨi(x)dσ(x) , i = 0, . . . , k−1 , σ ∈ P([A,B])}, (2.1)where P([A,B]) denotes the set of all finite measures on the interval [A,B]. It can becharacterized as the smallest convex cone containing the curveCk−1 ={(Ψ0(t), . . . ,Ψk−1(t))T∣∣∣ t ∈ [A,B]}2[see Karlin and Studden (1966)]. By Caratheodory’s theorem any point of Mk−1 can bedescribed as a linear combination of at most k + 1 points in Ck−1, where the coefficientsare positive. Moment spaces can be defined for any set of linearly independent functions,but if the functions {Ψ0, . . . ,Ψk−1} generate a Chebychev system, the moment space hasseveral additional interesting properties. In particular, less points of Ck−1 are required forthe representation of points in Mk−1. To be precise, we define for a point c0 ∈ Mk−1 itsindex I(c0) as the minimal number of points in Ck−1 which are required to represent c0,where the points (Ψ0(A), . . . ,Ψk−1(A))T and (Ψ0(B), . . . ,Ψk−1(B))T corresponding to theboundary of the interval [A,B] are counted by 1/2. The index of a finite measure σ on [A,B]is defined as the index of the point c =∫ BA(Ψ0(x), . . . ,Ψk−1(x))Tdσ(x). The measure σ isalso called representation of the point c ∈Mk−1.With this convention it follows that the point c0 ∈Mk−1 is a boundary point ofMk−1 if andonly if its index satisfies I(c0) < k2. Similarly, c0 is in the interior ofMk−1 if its index is k2 ork+12. Following Karlin and Studden (1966) we denote a representation of an interior point c0as principal, if I(c0) = k2. These authors proved that for each interior point c0 ∈Mk−1 thereexist exactly two principal representations. The first is called upper principal representationand contains the point (Ψ0(B), . . . ,Ψk−1(B))T corresponding to the right boundary of theinterval [A,B], whereas the second is called lower principal representation and does not usethis point. The corresponding measures are denoted by σ+ and σ−. If k is odd the lower andupper principal representation have k+12support points. On the other hand, if k is even thelower and upper principal representation have k2and k+22support points respectively. Thefollowing Lemma is crucial in the following investigations and a direct consequence of thediscussion on page 55-56 in Karlin and Studden (1966).Lemma 2.1 Let Ψj : [A,B] → R (j = 0, . . . , k − 1); Ω : [A,B] → R denote real valuedfunctions and assume that the systems {Ψ0, . . . ,Ψk−1} and {Ψ0, . . . ,Ψk−1,Ω} are Chebychevsystems on the interval [A,B]. If c0 = (c01, . . . , c0k−1)T ∈ Mk−1, then the upper and lowerprincipal representation σ+ and σ− of c0 are uniquely determined and satisfymax{∫ BAΩ(t)dσ(t)∣∣∣ σ ∈ P([A,B]), c0i = ∫ BAΨi(t)dσ(t), i = 0, . . . , k − 1}=∫ BAΩ(t)dσ+(t),min{∫ BAΩ(t)dσ(t)∣∣∣ σ ∈ P([A,B]), c0i = ∫ BAΨi(t)dσ(t), i = 0, . . . , k − 1}=∫ BAΩ(t)dσ−(t).In particular both representation do not depend on the function Ω : [A,B]→ R.3 A complete class of designs for regression modelsConsider the common non linear regression modelE[Y |x] = η(x, θ) (3.1)3where θ ∈ Rp is the vector of unknown parameters, x denotes a real valued covariate fromthe design space [A,B] ⊂ R and different observations are assumed to be independent withvariance σ2. The function η is called regression function [see Seber and Wild (1989) orRatkowsky (1990)] and assumed to be continuous and differentiable with respect to the vari-able θ. A design is defined as a probability measure ξ on the interval [A,B] with finitesupport [see Kiefer (1974)]. If the design ξ has masses wi at the points xi (i = 1, . . . , l) andn observations can be made by the experimenter, this means that the quantities win arerounded to integers, say ni, satisfying∑li=1 ni = n, and the experimenter takes ni observa-tions at each location xi (i = 1, . . . , l). If the design ξ contains l support points x1, . . . , xlsuch that the vectors ∂∂θη(x1, θ), . . . ,∂∂θη(xl, θ) are linearly independent and observations aretaken according to this procedure it follows from Jennrich (1969) that the covariance matrixof the non-linear least squares estimator is approximately (if n→∞) given byσ2nM−1(ξ, θ) =σ2n(∫ BA(∂∂θη(x, θ))(∂∂θη(x, θ))Tdξ(x))−1, (3.2)An optimal design maximizes an appropriate functional of the matrix nσ2M(ξ, θ) and numer-ous criteria have been proposed in the literature to discriminate between competing designs[see Pukelsheim (2006)]. Note that the matrix (3.2) depends on the unknown parameter θand following Chernoff (1953) we call the maximizing designs locally optimal designs. Thesedesigns require an initial guess of the unknown parameters in the model and are used asbenchmarks for many commonly used designs or for the construction of more sophisticatedoptimality criteria which require less information regarding the parameters of the model[Chaloner and Verdinelli (1995) and Dette (1997)].Most of the available optimality criteria are positively homogeneous, that is Φ(nσ2M(ξ, θ))=nσ2Φ(M(ξ, θ)) [Pukelsheim (2006)]. Therefore it is sufficient to consider maximization of func-tions of the matrix M(ξ, θ), which is called information matrix in the literature. Moreover,the commonly used optimality criteria also satisfy a monotonicity property with respect tothe Loewner ordering, that is Φ(M(ξ1, θ)) ≥ Φ(M(ξ2, θ)), whenever M(ξ1, θ) ≥ M(ξ2, θ),where the parameter θ is fixed, ξ1, ξ2 are two competing designs on the interval [A,B] and Φdenotes an information function in the sense of Pukelsheim (2006). Throughout this paper wecall a design ξ admissible if there does not exist any design ξ1, such that M(ξ1, θ) 6= M(ξ, θ)andM(ξ1, θ) ≥M(ξ, θ). (3.3)Yang and Stufken (2012a) derive a complete class theorem in this general context whichcharacterizes the class of designs, which cannot be improved with respect to the Loewnerordering of their information matrices. For the sake of completeness and because of itsimportance we will state this result here again. In particular, we demonstrate that thecomplete class specified by these authors corresponds to upper and principal representationsof a moment space generated by the regression functions. For this purpose we denote by P (θ)4a regular p× p matrix, which does not depend on the design ξ, such that the representationM(ξ, θ) = P (θ)C(ξ, θ)P T (θ) (3.4)holds, where the p× p matrix C(ξ, θ) is defined byC(ξ, θ) =∫ BAΨ11(x) . . . Ψ1p(x)... . . . ...Ψp1(x) . . . Ψpp(x) dξ(x) = ∫ BA(C11(x) CT21(x)C21(x) C22(x))dξ(x)and C11(x) ∈ Rp−p1×p−p1 , C21(x) ∈ Rp1×p−p1 , C22(x) ∈ Rp1×p1 are appropriate block matrices(1 ≤ p1 ≤ p). Obviously, P (θ) could be chosen as identity matrix, but in concrete appli-cations other choices might be advantageous [see Yang and Stufken (2012b), Section 4, fornumerous interesting examples]. A similar comment applies to the choice of p1 which is usedto represent the matrix C in a 2× 2 block matrix. Note that the inequality (3.3) is satisfiedif and only if the inequalityC(ξ1, θ) ≥ C(ξ, θ) (3.5)holds. Following Yang and Stufken (2012a) we define Ψ0(x) = 1, denote the different ele-ments among {Ψij|1 ≤ i ≤ p, j ≤ p − p1} in the matrices C11(x) and C21(x) which are notconstant by Ψ1, . . . ,Ψk−1 and define for any vector Q ∈ Rp1 \ {0} the functionΨQk (x) = QTC22(x)Q. (3.6)We are now in apposition to state and prove the main result of this paper.Theorem 3.1 [Yang and Stufken (2012a)]1. If {Ψ0, . . . ,Ψk−1} and {Ψ0, . . . ,Ψk−1,ΨQk } are Chebychev systems for every non-zerovector Q, then for any design ξ there exists a design ξ+ with at most k+22support points,such that M(ξ+, θ) ≥M(ξ, θ).If the index of ξ satisfies I(ξ) < k2, then the design ξ+ is uniquely determined in theset {η∣∣∣ ∫ BAΨi(x)dη(x) =∫ BAΨi(x)dξ(x), i = 1, . . . , k − 1}(3.7)and coincides with the design ξ.If the index of ξ satisfies I(ξ) ≥ k2, then the following cases are discriminated:(a) If k is odd, then the design ξ+ has at most k+12support points and it can be chosensuch that B is a support point of the design ξ+.(b) If k is even, then the design ξ+ has at most k+22support points and it can bechosen such that A and B are support points of the design ξ+.52. If {Ψ0, . . . ,Ψk−1} and {Ψ0, . . . ,Ψk−1,−ΨQk } are Chebychev systems for every non-zerovector Q, then for any design ξ there exists a design ξ− with at most k+12support points,such that M(ξ−, θ) ≥M(ξ, θ).If the index of ξ satisfies I(ξ) < k2, then the design ξ− is uniquely determined in theset of measures satisfying (3.7) and coincides with the design ξ.If the index of ξ satisfies I(ξ) ≥ k2, then the following cases are discriminated:(a) If k is odd, then the design ξ− has at most k+12support points and it can be chosensuch that A is a support point of the design ξ−.(b) If k is even, then the design ξ− has at most k2support points.Proof. We only present the proof of the first part of the theorem, the second part followsby similar arguments. Yang and Stufken (2012a) showed that a design ξ1 satisfies (3.3) ifthe conditions ∫ BAΨi(x)dξ1(x) =∫ BAΨi(x)dξ(x) i = 1, . . . , k − 1 (3.8)∫ BAΨQk (x)dξ1(x) ≥∫ BAΨQk (x)dξ(x)are satisfied for all vectors Q 6= 0. Consequently an improvement of the design ξ is obtainedby maximizing the “k-th moment”∫ BAΨQk (x)dξ1(x) in the set of all designs satisfying (3.8).If I(ξ) < k2, then this set is a singleton and the maximizing design ξ+Q coincides with ξ.Otherwise, by Lemma 2.1 the maximizing measure ξ+Q corresponds to the upper principalpresentation of the moment point (∫ BAΨ0(x)dξ(x), . . . ,∫ BAΨk−1(x)dξ(x))T , which does notdepend on the vector Q. Finally, assertion 1(a) or 1(b) of Theorem 3.1 follow from thediscussion regarding the number of support points of principal representations given at theend of Section 2.2Acknowledgements. The authors thank Martina Stein, who typed parts of this manuscriptwith considerable technical expertise. This work has been supported in part by the Collab-orative Research Center “Statistical modeling of nonlinear dynamic processes” (SFB 823,Teilprojekt C2) of the German Research Foundation (DFG).ReferencesChaloner, K. and Verdinelli, I. (1995). Bayesian experimental design: A review. StatisticalScience, 10(3):273–304.Chernoff, H. (1953). Locally optimal designs for estimating parameters. Annals of Mathe-matical Statistics, 24:586–602.6Dette, H. (1997). Designing experiments with respect to “standardized” optimality criteria.Journal of the Royal Statistical Society, Ser. B, 59:97–110.Dette, H., Melas, V., and Wong, W. K. (2006). Locally D-optimal designs for exponentialregression models. Statistica Sinica, 16:789–803.Dette, H. and Melas, V. B. (2011). A note on the de la Garza phenomenon for locally optimaldesigns. Annals of Statistics, 39(2):1266–1281.Fang, X. and Hedayat, A. S. (2008). Locally D-optimal designs based on a class of composedmodels resulted from blending Emax and one-compartment models. Annals of Statistics,36:428–444.He, Z., Studden, W. J., and Sun, D. (1996). Optimal designs for rational models. Annals ofStatistics, 24:2128–2142.Jennrich, R. I. (1969). Asymptotic properties of non-linear least squares estimators. Annalsof Mathematical Statistics, 40:633–643.Karlin, S. and Studden, W. J. (1966). Tchebysheff Systems: With Application in Analysisand Statistics. Wiley, New York.Khuri, A., Mukherjee, B., Sinha, B., and Ghosh, M. (2006). Design issues for generalizedlinear models. Statistical Science, 21(3):376–399.Kiefer, J. (1974). General equivalence theory for optimum designs (approximate theory).Annals of Statistics, 2:849–879.Pukelsheim, F. (2006). Optimal Design of Experiments. SIAM, Philadelphia.Ratkowsky, D. A. (1990). Handbook of Nonlinear Regression Models. Dekker, New York.Seber, G. A. F. and Wild, C. J. (1989). Nonlinear Regression. John Wiley and Sons Inc.,New York.Yang, M. (2010). On the de la Garza phenomenon. Annals of Statistics, 38(4):2499–2524.Yang, M. and Stufken, J. (2009). Support points of locally optimal designs for nonlinearmodels with two parameters. Annals of Statistics, 37:518–541.Yang, M. and Stufken, J. (2012a). Identifying locally optimal designs for nonlinear models:A simple extension with profound consequences. Annals of Statistics, 40:1665–1685.Yang, M. and Stufken, J. (2012b). On locally optimal designs for generalized linear modelswith group effects. Statistica Sinica, 22:1765–1786.7   

In a recent paper Yang and Stufken (2012a) gave sufficient conditions for complete classes of designs for nonlinear regression models. In this note we demonstrate that their result is a simple consequence of the fact that boundary points of moment spaces generated by Chebyshev systems possess unique representations

SFB 
823 
Complete classes of designs 
for nonlinear regression 
models and principal 
representations of moment 
spaces 
D
iscussion P
aper 
 
Holger Dette, Kirsten Schorning 
 
 
 
 
 
Nr. 55/2012 
 
 
 
 
 
 
 
Complete classes of designs for nonlinear regression
models and principal representations of moment spaces
Holger Dette, Kirsten Schorning
Ruhr-Universität Bochum
Fakultät für Mathematik
44780 Bochum, Germany
e-mail: holger.dette@rub.de
November 16, 2012
Abstract
In a recent paper Yang and Stufken (2012a) gave sufficient conditions for complete
classes of designs for nonlinear regression models. In this note we demonstrate that
their result is a simple consequence of the fact that boundary points of moment spaces
generated by Chebyshev systems possess unique representations.
Keywords and Phrases: locally optimal design; admissible design; Chebyshev system; prin-
ciple representations; moment spaces; complete classes of designs
AMS Subject Classification: 62K05
1 Introduction
The construction of locally optimal designs for nonlinear regression models has found con-
siderable interest in recent years [see for example He et al. (1996), Dette et al. (2006), Khuri
et al. (2006), Fang and Hedayat (2008), Yang and Stufken (2012b) among others]. While
most of the literature focuses on specific models or specific optimality criteria, general results
characterizing the structure of locally optimal designs are extremely difficult to obtain due
to the complicated structure of the corresponding nonlinear optimization problems. In a
series of remarkable papers Yang and Stufken (2009), Yang (2010), Dette and Melas (2011)
and Yang and Stufken (2012a) derived several complete classes of designs with respect to the
Loewner Ordering of the information matrices. The first paper in this direction of Yang and
Stufken (2009) investigates nonlinear regression models with two parameters. These results
1
were generalized by Yang (2010) and Dette and Melas (2011) to identify small complete
classes for nonlinear regression models with more than two parameters. The most general
contribution is the recent paper of Yang and Stufken (2012a), which provides a sufficient
condition for a complete class of designs and is applicable to most of the commonly used
regression models. The proof of this statement is complicated and requires several auxiliary
results.
The purpose of the present paper is to demonstrate that conditions of this type are inti-
mately related to the characterization of boundary points of moment spaces associated with
a nonlinear regression model. Our main tool is a Chebyshev system [Karlin and Studden
(1966)] appearing in (a transformation of) the Fisher Information matrix of a given design.
The complete class of designs can essentially be characterized as the set of measures cor-
responding to the lower and upper principal representation of the boundary points of the
corresponding moment spaces. With this insight the main result in the paper of Yang and
Stufken (2012a) is a simple consequence of the fact that a representation of a boundary point
of a k + 1-dimensional moment space associated with a Chebyshev system depends only on
the first k functions which are used to generate the moment space.
In Section 2 we present some facts on moment spaces associated with Chebyshev systems
which are of general interest for constructing admissible designs. The design problem and
Theorem 1 of Yang and Stufken (2012a) are stated in Section 3, where we also present our
alternative proof. We finally note that the paper of Yang and Stufken (2012a) contains
numerous interesting examples and provided a further result which are not discussed in this
note for the sake of brevity.
2 Chebyshev systems and associated moment spaces
A set of k real valued functions Ψ0, . . . ,Ψk−1 : [A,B] → R is called Chebychev system on
the interval [A,B] if and only if it fulfills the inequality
det
 Ψ0(x0) . . . Ψ0(xk−1)... . . . ...
Ψk−1(x0) . . . Ψk−1(xk−1)
 > 0
for any points x0, . . . , xk−1 with A ≤ x0 < x1 . . . < xk−1 ≤ B. The moment space associated
with a Chebychev system is defined by
Mk−1 =
{
c = (c0, . . . , ck−1)
T
∣∣∣ ci = ∫ B
A
Ψi(x)dσ(x) , i = 0, . . . , k−1 , σ ∈ P([A,B])
}
, (2.1)
where P([A,B]) denotes the set of all finite measures on the interval [A,B]. It can be
characterized as the smallest convex cone containing the curve
Ck−1 =
{
(Ψ0(t), . . . ,Ψk−1(t))
T
∣∣∣ t ∈ [A,B]}
2
[see Karlin and Studden (1966)]. By Caratheodory’s theorem any point of Mk−1 can be
described as a linear combination of at most k + 1 points in Ck−1, where the coefficients
are positive. Moment spaces can be defined for any set of linearly independent functions,
but if the functions {Ψ0, . . . ,Ψk−1} generate a Chebychev system, the moment space has
several additional interesting properties. In particular, less points of Ck−1 are required for
the representation of points in Mk−1. To be precise, we define for a point c0 ∈ Mk−1 its
index I(c0) as the minimal number of points in Ck−1 which are required to represent c0,
where the points (Ψ0(A), . . . ,Ψk−1(A))
T and (Ψ0(B), . . . ,Ψk−1(B))
T corresponding to the
boundary of the interval [A,B] are counted by 1/2. The index of a finite measure σ on [A,B]
is defined as the index of the point c =
∫ B
A
(Ψ0(x), . . . ,Ψk−1(x))
Tdσ(x). The measure σ is
also called representation of the point c ∈Mk−1.
With this convention it follows that the point c0 ∈Mk−1 is a boundary point ofMk−1 if and
only if its index satisfies I(c0) < k
2
. Similarly, c0 is in the interior ofMk−1 if its index is k2 or
k+1
2
. Following Karlin and Studden (1966) we denote a representation of an interior point c0
as principal, if I(c0) = k
2
. These authors proved that for each interior point c0 ∈Mk−1 there
exist exactly two principal representations. The first is called upper principal representation
and contains the point (Ψ0(B), . . . ,Ψk−1(B))
T corresponding to the right boundary of the
interval [A,B], whereas the second is called lower principal representation and does not use
this point. The corresponding measures are denoted by σ+ and σ−. If k is odd the lower and
upper principal representation have k+1
2
support points. On the other hand, if k is even the
lower and upper principal representation have k
2
and k+2
2
support points respectively. The
following Lemma is crucial in the following investigations and a direct consequence of the
discussion on page 55-56 in Karlin and Studden (1966).
Lemma 2.1 Let Ψj : [A,B] → R (j = 0, . . . , k − 1); Ω : [A,B] → R denote real valued
functions and assume that the systems {Ψ0, . . . ,Ψk−1} and {Ψ0, . . . ,Ψk−1,Ω} are Chebychev
systems on the interval [A,B]. If c0 = (c01, . . . , c
0
k−1)
T ∈ Mk−1, then the upper and lower
principal representation σ+ and σ− of c0 are uniquely determined and satisfy
max
{∫ B
A
Ω(t)dσ(t)
∣∣∣ σ ∈ P([A,B]), c0i = ∫ B
A
Ψi(t)dσ(t), i = 0, . . . , k − 1
}
=
∫ B
A
Ω(t)dσ+(t),
min
{∫ B
A
Ω(t)dσ(t)
∣∣∣ σ ∈ P([A,B]), c0i = ∫ B
A
Ψi(t)dσ(t), i = 0, . . . , k − 1
}
=
∫ B
A
Ω(t)dσ−(t).
In particular both representation do not depend on the function Ω : [A,B]→ R.
3 A complete class of designs for regression models
Consider the common non linear regression model
E[Y |x] = η(x, θ) (3.1)
3
where θ ∈ Rp is the vector of unknown parameters, x denotes a real valued covariate from
the design space [A,B] ⊂ R and different observations are assumed to be independent with
variance σ2. The function η is called regression function [see Seber and Wild (1989) or
Ratkowsky (1990)] and assumed to be continuous and differentiable with respect to the vari-
able θ. A design is defined as a probability measure ξ on the interval [A,B] with finite
support [see Kiefer (1974)]. If the design ξ has masses wi at the points xi (i = 1, . . . , l) and
n observations can be made by the experimenter, this means that the quantities win are
rounded to integers, say ni, satisfying
∑l
i=1 ni = n, and the experimenter takes ni observa-
tions at each location xi (i = 1, . . . , l). If the design ξ contains l support points x1, . . . , xl
such that the vectors ∂
∂θ
η(x1, θ), . . . ,
∂
∂θ
η(xl, θ) are linearly independent and observations are
taken according to this procedure it follows from Jennrich (1969) that the covariance matrix
of the non-linear least squares estimator is approximately (if n→∞) given by
σ2
n
M−1(ξ, θ) =
σ2
n
(∫ B
A
(
∂
∂θ
η(x, θ)
)(
∂
∂θ
η(x, θ)
)T
dξ(x)
)−1
, (3.2)
An optimal design maximizes an appropriate functional of the matrix n
σ2
M(ξ, θ) and numer-
ous criteria have been proposed in the literature to discriminate between competing designs
[see Pukelsheim (2006)]. Note that the matrix (3.2) depends on the unknown parameter θ
and following Chernoff (1953) we call the maximizing designs locally optimal designs. These
designs require an initial guess of the unknown parameters in the model and are used as
benchmarks for many commonly used designs or for the construction of more sophisticated
optimality criteria which require less information regarding the parameters of the model
[Chaloner and Verdinelli (1995) and Dette (1997)].
Most of the available optimality criteria are positively homogeneous, that is Φ
(
n
σ2
M(ξ, θ)
)
=
n
σ2
Φ(M(ξ, θ)) [Pukelsheim (2006)]. Therefore it is sufficient to consider maximization of func-
tions of the matrix M(ξ, θ), which is called information matrix in the literature. Moreover,
the commonly used optimality criteria also satisfy a monotonicity property with respect to
the Loewner ordering, that is Φ(M(ξ1, θ)) ≥ Φ(M(ξ2, θ)), whenever M(ξ1, θ) ≥ M(ξ2, θ),
where the parameter θ is fixed, ξ1, ξ2 are two competing designs on the interval [A,B] and Φ
denotes an information function in the sense of Pukelsheim (2006). Throughout this paper we
call a design ξ admissible if there does not exist any design ξ1, such that M(ξ1, θ) 6= M(ξ, θ)
and
M(ξ1, θ) ≥M(ξ, θ). (3.3)
Yang and Stufken (2012a) derive a complete class theorem in this general context which
characterizes the class of designs, which cannot be improved with respect to the Loewner
ordering of their information matrices. For the sake of completeness and because of its
importance we will state this result here again. In particular, we demonstrate that the
complete class specified by these authors corresponds to upper and principal representations
of a moment space generated by the regression functions. For this purpose we denote by P (θ)
4
a regular p× p matrix, which does not depend on the design ξ, such that the representation
M(ξ, θ) = P (θ)C(ξ, θ)P T (θ) (3.4)
holds, where the p× p matrix C(ξ, θ) is defined by
C(ξ, θ) =
∫ B
A
Ψ11(x) . . . Ψ1p(x)... . . . ...
Ψp1(x) . . . Ψpp(x)
 dξ(x) = ∫ B
A
(
C11(x) C
T
21(x)
C21(x) C22(x)
)
dξ(x)
and C11(x) ∈ Rp−p1×p−p1 , C21(x) ∈ Rp1×p−p1 , C22(x) ∈ Rp1×p1 are appropriate block matrices
(1 ≤ p1 ≤ p). Obviously, P (θ) could be chosen as identity matrix, but in concrete appli-
cations other choices might be advantageous [see Yang and Stufken (2012b), Section 4, for
numerous interesting examples]. A similar comment applies to the choice of p1 which is used
to represent the matrix C in a 2× 2 block matrix. Note that the inequality (3.3) is satisfied
if and only if the inequality
C(ξ1, θ) ≥ C(ξ, θ) (3.5)
holds. Following Yang and Stufken (2012a) we define Ψ0(x) = 1, denote the different ele-
ments among {Ψij|1 ≤ i ≤ p, j ≤ p − p1} in the matrices C11(x) and C21(x) which are not
constant by Ψ1, . . . ,Ψk−1 and define for any vector Q ∈ Rp1 \ {0} the function
ΨQk (x) = Q
TC22(x)Q. (3.6)
We are now in apposition to state and prove the main result of this paper.
Theorem 3.1 [Yang and Stufken (2012a)]
1. If {Ψ0, . . . ,Ψk−1} and {Ψ0, . . . ,Ψk−1,ΨQk } are Chebychev systems for every non-zero
vector Q, then for any design ξ there exists a design ξ+ with at most k+2
2
support points,
such that M(ξ+, θ) ≥M(ξ, θ).
If the index of ξ satisfies I(ξ) < k
2
, then the design ξ+ is uniquely determined in the
set {
η
∣∣∣ ∫ B
A
Ψi(x)dη(x) =
∫ B
A
Ψi(x)dξ(x), i = 1, . . . , k − 1
}
(3.7)
and coincides with the design ξ.
If the index of ξ satisfies I(ξ) ≥ k
2
, then the following cases are discriminated:
(a) If k is odd, then the design ξ+ has at most k+1
2
support points and it can be chosen
such that B is a support point of the design ξ+.
(b) If k is even, then the design ξ+ has at most k+2
2
support points and it can be
chosen such that A and B are support points of the design ξ+.
5
2. If {Ψ0, . . . ,Ψk−1} and {Ψ0, . . . ,Ψk−1,−ΨQk } are Chebychev systems for every non-zero
vector Q, then for any design ξ there exists a design ξ− with at most k+1
2
support points,
such that M(ξ−, θ) ≥M(ξ, θ).
If the index of ξ satisfies I(ξ) < k
2
, then the design ξ− is uniquely determined in the
set of measures satisfying (3.7) and coincides with the design ξ.
If the index of ξ satisfies I(ξ) ≥ k
2
, then the following cases are discriminated:
(a) If k is odd, then the design ξ− has at most k+1
2
support points and it can be chosen
such that A is a support point of the design ξ−.
(b) If k is even, then the design ξ− has at most k
2
support points.
Proof. We only present the proof of the first part of the theorem, the second part follows
by similar arguments. Yang and Stufken (2012a) showed that a design ξ1 satisfies (3.3) if
the conditions ∫ B
A
Ψi(x)dξ1(x) =
∫ B
A
Ψi(x)dξ(x) i = 1, . . . , k − 1 (3.8)∫ B
A
ΨQk (x)dξ1(x) ≥
∫ B
A
ΨQk (x)dξ(x)
are satisfied for all vectors Q 6= 0. Consequently an improvement of the design ξ is obtained
by maximizing the “k-th moment”
∫ B
A
ΨQk (x)dξ1(x) in the set of all designs satisfying (3.8).
If I(ξ) < k
2
, then this set is a singleton and the maximizing design ξ+Q coincides with ξ.
Otherwise, by Lemma 2.1 the maximizing measure ξ+Q corresponds to the upper principal
presentation of the moment point (
∫ B
A
Ψ0(x)dξ(x), . . . ,
∫ B
A
Ψk−1(x)dξ(x))
T , which does not
depend on the vector Q. Finally, assertion 1(a) or 1(b) of Theorem 3.1 follow from the
discussion regarding the number of support points of principal representations given at the
end of Section 2.
2
Acknowledgements. The authors thank Martina Stein, who typed parts of this manuscript
with considerable technical expertise. This work has been supported in part by the Collab-
orative Research Center “Statistical modeling of nonlinear dynamic processes” (SFB 823,
Teilprojekt C2) of the German Research Foundation (DFG).
References
Chaloner, K. and Verdinelli, I. (1995). Bayesian experimental design: A review. Statistical
Science, 10(3):273–304.
Chernoff, H. (1953). Locally optimal designs for estimating parameters. Annals of Mathe-
matical Statistics, 24:586–602.
6
Dette, H. (1997). Designing experiments with respect to “standardized” optimality criteria.
Journal of the Royal Statistical Society, Ser. B, 59:97–110.
Dette, H., Melas, V., and Wong, W. K. (2006). Locally D-optimal designs for exponential
regression models. Statistica Sinica, 16:789–803.
Dette, H. and Melas, V. B. (2011). A note on the de la Garza phenomenon for locally optimal
designs. Annals of Statistics, 39(2):1266–1281.
Fang, X. and Hedayat, A. S. (2008). Locally D-optimal designs based on a class of composed
models resulted from blending Emax and one-compartment models. Annals of Statistics,
36:428–444.
He, Z., Studden, W. J., and Sun, D. (1996). Optimal designs for rational models. Annals of
Statistics, 24:2128–2142.
Jennrich, R. I. (1969). Asymptotic properties of non-linear least squares estimators. Annals
of Mathematical Statistics, 40:633–643.
Karlin, S. and Studden, W. J. (1966). Tchebysheff Systems: With Application in Analysis
and Statistics. Wiley, New York.
Khuri, A., Mukherjee, B., Sinha, B., and Ghosh, M. (2006). Design issues for generalized
linear models. Statistical Science, 21(3):376–399.
Kiefer, J. (1974). General equivalence theory for optimum designs (approximate theory).
Annals of Statistics, 2:849–879.
Pukelsheim, F. (2006). Optimal Design of Experiments. SIAM, Philadelphia.
Ratkowsky, D. A. (1990). Handbook of Nonlinear Regression Models. Dekker, New York.
Seber, G. A. F. and Wild, C. J. (1989). Nonlinear Regression. John Wiley and Sons Inc.,
New York.
Yang, M. (2010). On the de la Garza phenomenon. Annals of Statistics, 38(4):2499–2524.
Yang, M. and Stufken, J. (2009). Support points of locally optimal designs for nonlinear
models with two parameters. Annals of Statistics, 37:518–541.
Yang, M. and Stufken, J. (2012a). Identifying locally optimal designs for nonlinear models:
A simple extension with profound consequences. Annals of Statistics, 40:1665–1685.
Yang, M. and Stufken, J. (2012b). On locally optimal designs for generalized linear models
with group effects. Statistica Sinica, 22:1765–1786.
7
 
 
 


http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=BFDCA5AC2391BAAEBB8737165ECFB94B?doi=10.1.1.1074.8029&rep=rep1&type=pdf

Complete classes of designs for nonlinear regression models and principal representations of moment spaces

Abstract

Similar works

Full text

Available Versions

CiteSeerX

Eldorado - Repository of the TU Dortmund

CiteSeerX