A smooth testing approach has been used to develop a test of the distributional assumption for generalized linear models. Application of this test to help assess Poisson and logistic regression models is discussed in this paper and power is compared to some common tests

Rayner, J. C. W.

Rippon, Paul

The smooth testing approach described in [2] has been used to develop a test of the distributional assumption for generalized linear models. Application of the test to help assess Poisson and logistic regression models is discussed. Power is compared to other common tests

Rayner, John

Research Online

University of Wollongong Research Online Applied Statistics Education and Research Collaboration (ASEARC) - Conference Papers Faculty of Engineering and Information Sciences 2011 Assessing Poisson and Logistic Regression Models Using Smooth Tests Paul Rippon University of Newcastle John Rayner University of Newcastle Follow this and additional works at: https://ro.uow.edu.au/asearc Recommended Citation Rippon, Paul and Rayner, John, "Assessing Poisson and Logistic Regression Models Using Smooth Tests" (2011). Applied Statistics Education and Research Collaboration (ASEARC) - Conference Papers. 21. https://ro.uow.edu.au/asearc/21 Research Online is the open access institutional repository for the University of Wollongong. For further information contact the UOW Library: research-pubs@uow.edu.au Assessing Poisson and Logistic Regression Models Using Smooth Tests Abstract The smooth testing approach described in [2] has been used to develop a test of the distributional assumption for generalized linear models. Application of the test to help assess Poisson and logistic regression models is discussed. Power is compared to other common tests. Keywords generalized linear models, goodness of fit, logistic regression, Poisson regression Publication Details Rippon, Paul; and Rayner, John, Assessing Poisson and Logistic Regression Models Using Smooth Tests, Proceedings of the Fourth Annual ASEARC Conference, 17-18 February 2011, University of Western Sydney, Paramatta, Australia. This conference paper is available at Research Online: https://ro.uow.edu.au/asearc/21 Assessing Poisson and Logistic Regression Models Using Smooth TestsPaul Rippon, J.C.W. RaynerThe University of Newcastle, Callaghan, NSW, 2308, AUSTRALIAAbstractThe smooth testing approach described in [2] has been used to develop a test of the distributional assumption forgeneralized linear models. Application of the test to help assess Poisson and logistic regression models is discussed.Power is compared to other common tests.Key words: generalized linear models, goodness of fit, logistic regression, Poisson regression1. IntroductionThe concept of smooth testing originally proposed in[1] has been developed in [2] to provide goodness of fittests for a wide range of distributions. In [3], these ideashave been applied to the generalized linear modellingframework, where the variables are no longer identicallydistributed, to derive a test of the distributional assump-tion. Section 2 describes the test, Section 3 commentson its application and Section 4 discusses the resultsof simulation studies examining the power of this testwhen applied to Poisson and logistic regression.2. A Smooth Test of the Distributional Assumptionin Generalized Linear ModelsThe generalized linear modelling structure comprisesa linear combination of predictor variables related via alink function to the mean of the response distributionselected from the exponential family of distributions.In commonly used notation, independent response vari-ables, Y1, . . . ,Yn, are distributed with density functionf (y j; θ j) = exp[y jθ j − b(θ j)a(φ j)+ c(y j, φ j)]from the exponential family with canonical parametersθ j to be estimated and dispersion parameters φ j assumedto be known; a, b and c are known functions. Using g(·)to represent the link function:g(µ j) = η j = xTj β = x j1β1 + . . . + x jpβpwhere µ j = E[Y j] = b′(θ j) for j = 1, . . . , n. To simplifysubscripting, an explicit intercept term, β0, is not shown.There is no loss of generality as β1 can become an in-tercept term by setting all x j1 = 1 in the first column ofX.To test the distributional assumption, the assumed re-sponse variable density, f (y j; θ j), is embedded within amore complex alternative density functionfk(y j; τ, θ j) = C(τ, θ j) exp k∑i=1τihi(y j; θ j) f (y j; θ j).This structure allows for ‘smooth’ departures from theassumed distribution controlled by the vector parame-ter, τ = [τ1, . . . , τk]T acting on the elements of the set,{hi(y; θ)}, of polynomials up to order k which are or-thonormal on the assumed distribution. The normaliz-ing constant, C(τ, θ j), simply ensures that fk(y j; τ, θ j) iscorrectly scaled to provide a valid probability densityfunction.When τ = 0, this smooth alternative collapses to theoriginal response variable distribution. Thus a test ofH0 : τ = 0 against HA : τ , 0 can reasonably beconsidered a test of the distributional assumption in ageneralized linear model.In [3], a score test statistic has been derived that canbe expressed as a sum of squares of several contributingcomponents:Ŝ k =V̂21ω̂2+ V̂22 + . . . + V̂2kwhereV̂i =1√nn∑j=1hi(y j; θ̂ j).The ith component involves the sum over the data ofthe ith order polynomial from the orthonormal sequenceFourth Annual ASEARC Conference February 17–18, 2011, Parramatta, AustraliaFourth Annual ASEARC Conference February 17–18, 2011, Parramatta, Australiaused in the construction of the smooth alternative distri-bution. The first component also contains a termω2 = 1 −1T H1nwhich is related to the hat matrix, H, obtained from themodel estimation process.Large values of Ŝ k provide evidence against H0.Asymptotically, the components V̂21/ω̂2, V̂22 , etc caneach be expected to follow the χ2(1) distribution and Ŝ kthe χ2(k) distribution. In practice this has not proved agood enough approximation for common sample sizesand so a parametric bootstrap process is recommendedto estimate p-values.3. Applying the Smooth TestIn deriving this test of the distributional assumption,the linear predictor and the link function are assumedto be correctly specified. If this is not true then a largevalue of the test statistic may be caused by a mismatchbetween the data and these other components of the gen-eralized linear model rather than an inappropriate re-sponse distribution. Similar issues arise with other teststhat are used to assess generalized linear models. Forexample, the well-known deviance statistic is derivedas a likelihood ratio test statistic comparing the fittedmodel with a saturated model having a linear predic-tor with as many parameters as there are covariate pat-terns. This provides the best possible fit to the observeddata – assuming that the specified response distributionand link function are correct. If this is not true, then alarge value of the deviance statistic may indicate a prob-lem with the assumed distribution or link function ratherthan the linear predictor. Similarly, a model that “fails”a goodness-of-link test may really have a problem withthe assumed distribution or linear predictor and not thelink function.Can we ever truly diagnose the problem with a poorlyfitting model? Clearly all such tests need to be carefullyinterpreted. There are many different ways that a modelcan be misspecified, some of which are very difficultto distinguish from each other. The smooth testing ap-proach is not a panacea. In addition to providing a re-liable test of the distributional assumption however, theindividual components can be considered as test statis-tics in their own right. This can provide useful diag-nostic information about the nature of any lack of fitdetected.0.0 0.2 0.4 0.6 0.8 1.00.00.20.40.60.81.0β2PowerSmooth Deviance Link0.0 0.2 0.4 0.6 0.8 1.00.00.20.40.60.81.0β2PowerSmooth Deviance LinkV̂12ω̂2V̂22V̂32Figure 1: Power to detect a misspecified linear predictor in simulatedlogistic regression data.4. Power Study4.1. Logistic RegressionFigure 1 shows the results of a simulation study forlogistic regression with a misspecified linear predic-tor. In this example, the fitted model waslog(π1 − π)= β0 + β1x1but the true model used to simulate the data waslog(π1 − π)= β0 + β1x1 + β2x2.A fixed covariate pattern was used for each sim-ulation with 25 groups corresponding to x1 tak-ing values −1,−0.5, 0, 0.5, 1 and x2 taking values−1.2,−0.7,−0.2, 0.3, 0.8. There were m = 30 trials ineach group. These two models coincide when β2 = 0.The misspecification increases as β2 increases (horizon-tal axis).5000 simulations were conducted for β2 = 0 to char-acterize the null distribution of each test statistic and1000 simulations for each of the other β2 values to char-acterize the alternative distributions. The α = 5% crit-ical value from the null distribution was used to definethe rejection region and thus determine the probabilityof the null hypothesis being rejected (power to detect themisspecification) which is plotted on the vertical axis.Three test statistics have been considered here: thedeviance statistic, the smooth test statistic of order 3and a link test statistic (see Appendix A). For all statis-tics used, the powers were based on simulated distribu-tions and not on approximate sampling distributions. In2Fourth Annual ASEARC Conference February 17–18, 2011, Parramatta, Australia0.0 0.1 0.2 0.3 0.40.00.20.40.60.81.0aPowerSmooth Deviance Link0.0 0.1 0.2 0.3 0.40.00.20.40.60.81.0aPowerSmooth Deviance LinkV̂12ω̂2V̂22V̂32Figure 2: Power to detect a misspecified link function in simulatedlogistic regression data.this first example, the deviance performs best in detect-ing this particular kind of misspecification of the linearpredictor. But the smooth test still performs reasonablywell and the link test is essentially useless here. The per-formance of the Ŝ k statistic is a compromise betweenthe performance of the individual components whichcan also be considered separately. In this case: thefirst component is almost exactly matching the perfor-mance of the goodness of link test; the second compo-nent has good power and drives the performance of theoverall test statistic and the third component is not par-ticularly useful. The components correspond roughlyto moments and so the second component is indicatingthat the variance in the data is not well modelled. Thismakes sense. A covariate is missing and so the stochas-tic part of the model is trying to cope with additionalvariation that should really have been explained by thelinear predictor.Figure 2 shows the results for a misspecified linkfunction where the fitted model wasπ(η) =eη1 + eηlog(π1 − π)= η = β0 + β1x1but the data was simulated using a generalization of thelogit link function (see Appendix B):π(η) =eh(η;a)1 + eh(η;a). (1)The parameter a plotted along the horizontal axis con-trols the amount of misspecification with zero again rep-resenting no misspecification. Other simulation detailsare the same as in the first example.Unsurprisingly, it is the goodness of link test thatperforms best here as this is the kind of problem it is0.00 0.01 0.02 0.03 0.040.00.20.40.60.81.0τPowerSmooth Deviance Link0.00 0.01 0.02 0.03 0.040.00.20.40.60.81.0τPowerSmooth Deviance LinkV̂12ω̂2V̂22V̂32Figure 3: Power to detect a misspecified response distribution in sim-ulated logistic regression data.designed to detect. However, the smooth test still per-forms well. Looking at the individual components, thefirst component is again matching the performance ofthe goodness of link test and is driving the performanceof the overall test statistic in detecting this kind of mis-specified model. The first component is correctly indi-cating that the problem is in how the mean of the datais being modelled. The second and third componentsaren’t useful in this case.Figure 3 shows the results for a misspecified re-sponse distribution where a binomial distribution isspecified when fitting the model but the data was sim-ulated using a beta-binomial distribution where theresponses Y j are B(m j, π∗j) for π∗j independently dis-tributed as beta random variables on (0, 1) with E[π∗j] =π j and Var(π∗j) = τπ j(1 − π j).Again the parameter plotted along the horizontal axis,τ in this case, controls the amount of misspecificationwith zero representing no misspecification. The de-viance test performs best in detecting this particular typeof misspecification, with the smooth test again perform-ing reasonably well and the goodness of link test poorly.The story with the components is again similar with thefirst component matching the performance of the good-ness of link test and the second component indicatingcorrectly that the variance is not being modelled cor-rectly in this example.4.2. Poisson RegressionIn Figure 4, the simulation scenario is the same as forFigure 1 except that the linear predictor is set to log µwhere Y j ∼ P(µ j). The performance of the smooth teststatistic and components in detecting this type of mis-specified linear predictor in Poisson regression can be3Fourth Annual ASEARC Conference February 17–18, 2011, Parramatta, Australia0.0 0.1 0.2 0.3 0.4 0.50.00.20.40.60.81.0β2PowerSmooth Deviance Link0.0 0.1 0.2 0.3 0.4 0.50.00.20.40.60.81.0β2PowerSmooth Deviance LinkV̂12ω̂2V̂22V̂32Figure 4: Power to detect a misspecified linear predictor distributionin simulated Poisson regression data.0.0 0.2 0.4 0.6 0.80.00.20.40.60.81.0τPowerSmooth Deviance Link0.0 0.2 0.4 0.6 0.80.00.20.40.60.81.0τPowerSmooth Deviance LinkV̂12ω̂2V̂22V̂32Figure 5: Power to detect a misspecified response distribution in sim-ulated Poisson regression data.seen to be very similar to that already discussed for lo-gistic regression.In Figure 5, a Poisson distribution is specified whenfitting the model but the data was simulated using a neg-ative binomial distribution with log µ j = η j and varianceµ j + τµ2j . As in the similar logistic regression example,the deviance is more powerful in detecting the misspeci-fication but the smooth test performs reasonably and thesecond component correctly indicates that the problemis in how the variance of the data is being modelled.5. ConclusionsA smooth test for assessing the distributional assump-tion in generalized linear models has been derived andapplied to Poisson and logistic regression models fittedto simulated data. While not always the most powerfultest, it appears to perform quite well in detecting lack offit even when the misspecification is in the link functionor the linear predictor rather than the response distribu-tion. Interpretation of the components provides addi-tional diagnostic information.A. Goodness of Link TestThere are a number of tests described in the literaturefor testing the adequacy of the link function in a gen-eralized linear model. Many of these are specific to aparticular link function. The goodness of link test usedin this paper is more generally applicable and is equiva-lent to the linktest function provided in STATA [4].The η̂ = Xβ̂ term from the fitted model and a η̂2 termare used as the predictors of the original response vari-ables in a new model. The η̂ term contains all the ex-planatory information of the original model. If there isa misspecified link the relationship between η̂ and g(y)will be non-linear and the η̂2 term is likely to be sig-nificant. The difference in deviance between these twomodels has been used as the link test statistic in thisstudy.B. Generalized Logit FunctionExpressed as an inverse link function, a generaliza-tion of the logit function is described by [5] in the sameform as Eq. (1) but using a function h(η;α1, α2) wherethe two shape parameters, α1 and α2, separately controlthe left and right tails. α1 = α2 gives a symmetric prob-ability curve π(η) with the logistic model as the specialcase α1 = α2 = 0. The function h(η; a) used in Eq. 1corresponds to a = −α1 = α2. This gives an asymmetricprobability curve that according to [5] corresponds to aBox-Cox power transform.References[1] J. Neyman. Smooth tests for goodness of fit. Skandinavisk Aktu-arietidskrift, 20:149–199, 1937.[2] J. C. W. Rayner, O. Thas, and D. J. Best. Smooth tests of goodnessof fit: Using R. Oxford University Press, 2nd edition, 2009.[3] Paul Rippon. Application of smooth tests of goodness of fit togeneralized linear models. Unpublished PhD thesis., 2011.[4] StataCorp. Stata Base Reference Manual, Release 9. Stata press,2005.[5] Therese A. Stukel. Generalized logistic models. Journal of theAmerican Statistical Association, 83(402):426–431, 1988.4

Assessing Poisson and logistic regression models using smooth tests

Abstract

Similar works

Full text

Available Versions

Research Online

University of Newcastle's Digital Repository