This paper presents methods to analyze and detect non-MCAR processes that lead to missing covariate values in linear regression models. First, the data situation and the problem is sketched. The next section provides an overview of the methods that deal with missing covariate values. The idea of using outlier methods to detect non-MCAR processes is described in section 3. Section 4 uses these ideas to introduce a graphical method to visualize the problem. Possible extensions conclude the presentation

Fieger, A.

Toutenburg, Helge

Albanian

Open Access LMU

Toutenburg, Fieger:Using diagnostic measures to detect non-MCARprocesses in linear regression models with missingcovariatesSonderforschungsbereich 386, Paper 204 (2000)Online unter: http://epub.ub.uni-muenchen.de/ProjektpartnerUsing diagnostic measures to detect nonMCARprocesses in linear regression models with missingcovariatesH Toutenburg A FiegeryJune  AbstractThis paper presents methods to analyze and detect nonMCAR processes thatlead to missing covariate values in linear regression models First the data situationand the problem is sketched The next section provides an overview of the methodsthat deal with missing covariate values The idea of using outlier methods to detectnonMCAR processes is described in section  Section  uses these ideas to introduce a graphical method to visualize the problem Possible extensions conclude thepresentation  Data and problemWe consider the classical linear regression modelyn    Xn  p p    n  with missing data in the n   p covariate matrix X  Reorganization of the n rows of thedata matrix X  the corresponding elements of the response y and the error term  leadsto the following structure ycymis XcXmis  cmisThe index c indicates the completely observed submodel whereas the index mis corre	sponds to the submodel with missing values in the covariate matrix Xmisnote that ymisis completely observedUsing the missing data indicator matrix R introduced by Rubin Rij if Zijis observed if Zijis missingwith data Zij yi Xij the missing mechanism can be characterized by the conditionaldistribution fRjZ  of R given the data Z and an unknown parameter  The n pmatrix Z consists of observed data Zobsand unobserved values ZmisThe data are said to be missing completely at random MCAR if the distribution ofR given Z and  only depends on the unknown parameter  for any Z iefRjZ   fRj Z If the conditional distribution of R depends on Z only via the observed values Zobsfor all Zmis iefRjZ   fRjZobs  Zmisthe data are called missing at random MAR Institute of Statistics LudwigMaximilians University of Munich Germany email toutenbstatunimuenchendeyInstitute of Statistics LudwigMaximilians University of Munich Germany email andreasstatunimuenchendeThe optimal estimator is the Gauss	Markov estimator b applied to the data in b  XcXmis  XcXmis XcXmis  ycymis X cXcX misXmisX cycX misymis  Due to the unknown values in Xmis of course this estimator can not be used directlyThere are a variety of methods that deal with this problem Dealing with missing valuesA simple and often used method is to discard all the information available in ymis Xmisand to use the completely observed data in yc Xc onlybc X cXcX cycUsing the available information in Z by estimating xxxywhere the elements ofxxandxyare formed by using all jointly observed pairs xij xi j and xij yi is called the available case methodMaximum likelihood procedures address the missing data problem by factoring thejoint distributionfZRj   fZjfRjZ  Integration over the missing data ZmisyieldsfZobs Rj  ZfZRj  dZmisZfRjZ fZj dZmisIf fRjZ depends only on the observed data Zobs ie the MAR assumption holds wehavefZobs Rj   fRjZobs ZfZj dZmis fRjZobs fZobsj which is why the missing data mechanism is also called ignorable in this caseImputation procedures form a dierent approach to the problem Here the missingvalues in Xmisare replaced by new values Having done this by some procedure theestimator  with Xmisreplaced by XRwith XRas described below becomes operationalTo replace the unknown values in Xmisa variety of imputation methods exist meanimputation or zero order regression ZOR replaces an unknown value xijby the meanxj either formed of the complete cases in Xcor the available cases in Xcand XmisConditional mean imputation or rst order regression FOR uses auxilliary regressionsto nd replacements for the missing values Regressing the covariate with missing valueson the remaining covaraites with parameters estimates based on the complete casesyields predictions of the missing values that are used as substitutes If the response yis also used in these regressions a stochastic element is introduced see Buck  orToutenburg and Shalabh Multiple Imputation Rubin  Schafer  repeats the imputation step andaverages the results While a single imputation is too smooth the dierences between theindividual imputation steps can be properly used to estimate the variance as the sum ofthe average variance within the imputed data sets and the between imputation varianceThis strategy reects the uncertainty about the imputation process which is ignored in asingle imputation strategyBy replacing a missing value by xR the model  becomes the mixed model ycyR XcXR  	 cRwhere 	 addresses the dierence between the true but unknown values in Xmisand theirreplacements in XR Using the mixed estimator Theil and Goldberger  we haveb  XcXR  XcXR XcXR  ycy X cXcX RXRX cycX Ry The weighted	mixed	estimator introduced by Scharin and Toutenburg  uses aweight    for the values in ymis XRb  X cXc X RXRX cyc X RyR  This estimator may be interpreted as the familiar mixed estimator in the model ycpy XcpXR  cp MCAR diagnosis with outlier measuresPopular diagnostics to detect non	MCAR processes contain the comparison of correlationor covariance matrices the comparison of means ycvs ymis or a more general test asdescribed by Little  For the situation with only one column aected by missingvalues Simon and Simono  present diagnostic plots where envelopes are comparedThe idea rst presented by Simono  combines the missing data problem withstatistics that derive from the oulier detection eld A comparison of the values of astatistic computed with and without imputation is the comparison of the sub	samples Zcand ZmisIf the imputation of values can be considered appropriate under MCAR and we reallyhave MCAR which is the null hypothesis H the statistics should be more or less thesame If we have something other than MCAR the statistics should reect this by havingdierent valuesSimono  uses Cooks distance which is based on the condence ellipsoidC cX Xcpsthe residual sum of squares DRSS Andrews and Pregibon DRSS RSSRSScnmisRSScnc nmis p and the determinant of the X X matrix DXX Andrews and Pregibon DXX jX cXcjjX XjFor the construction of tests the distribution of the measures under His neededAs this distribution also depends on the X values Monte Carlo methods are used todetermine it by rst computing the complete case statistics and imputing missing valuesunder MCAR	assumption The generation of new response valuesyMCmisXmisc MCwith MC N sI generates a new data set where missing values are drawn fromusing an MCAR mechanismAfter applying the imputation procedure to these data the diagnostic measures arecomputed Repeating the data deletion and imputation steps a null distribution of thediagnostic measure is generated Finally the measure can be applied to the imputedoriginal data and the resulting value can be compared to the null distribution Graphical diagnosis of the missing mechanismAnimated residualplots are presented in Cook and Weisberg  In a stepwise proce	dure weights between  and  are used to include one case into the regression The plotsthus represent the inuence of that single case Park Kim and Toutenburg  presenta similar approach to visualize the inclusion of a further variable into a regression modelThe adaption to the situation with missing data which shows a close relationship tothe procedures of the preceeding section is described in   Like in the above proceduresimputation is performed under an MCAR assumption Having lled the gaps in Xmis theweighted	mixed	estimator  is computed for certain values     Again the ideasis that if we really have MCAR there should not be any tendency in the residual plotwhen stepwise including ZRin the model by increasing the weight from  to Figure  shows a small program that visualizes the following procedurefor       step fcompute regression parameterscompute estimated residualsdisplay residual	plotgFigure  Java program for visualization on computer screen Reads data foreach frame of the animation and draws the single plots of the animation SeehttpwwwstatunimuenchendeandreasThe residual plot in gure  shows an example of an animated plot of y an the X	axisversus  on the Y	axis for a model y  xix  with missing data generatedby a non MAR process where P Ri  a value xiis missing dependes on xAn increasing  gives higher weight to the imputed data in the estimation of theregression parameters The center of the residual plot in gure  shifts towards theorigin For    the imputed data have the same weight as the complete data and biasedestimated result    the complete case esitmator on the other hand gives consistentestimates as the missing process is independent of the response y Possible extensionsThe ideas of animated residual plots could be extended in various ways Imagine a si	multaneous plot of  vs y vs Xiin dierent windows where the windows are linked Bybrushing selected points of the plot could be highlighted in all windows and their locationor movement can be studied while the value of  changesUnivariate plots y vs Xjfor all j together with the estimated regression lineiXiwhere the points are static as the imputation does not depend on the weight  and theestimated regression line is dynamic Again these plots could be linked as describedaboveCreation of a null plot where missing values are created articially by a known MCARmechanism This plot can be used as a means of comparison in order to have an idea ofwhat the plot should look like under MCAR-15 -10 -5 0 5 10 15-10-50510 0.•••••••••••••••••••••••••• •♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦-15 -10 -5 0 5 10 15-10-50510 0.1.••••••••••••••••••••••••••••♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦-15 -10 -5 0 5 10 15-10-50510 0.2.••••••••••••••••••••••••• •••♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦-15 -10 -5 0 5 10 15-10-50510 0.3.••••••••••••••••••••••••• •••♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦-15 -10 -5 0 5 10 15-10-50510 0.4.••••••••••••••••••••••••• •••♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦-15 -10 -5 0 5 10 15-10-50510 0.5.••••••••••••••••••••••••• •••♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦-15 -10 -5 0 5 10 15-10-50510 0.6.••••••••••••••••••••••••• •••♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦-15 -10 -5 0 5 10 15-10-50510 0.7.••••••••••••••••••••••••• •••♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦-15 -10 -5 0 5 10 15-10-50510 0.8.••••••••••••••••••••••••• •••♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦-15 -10 -5 0 5 10 15-10-50510 0.9.•••••••••••••••••••••• ••• •••♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦-15 -10 -5 0 5 10 15-10-50510 1.•••••••••••••••••••••• ••• •••♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦Figure  Residualplot of y X	axis versus  Y	axis for a model y  xixwith a non MAR process where P Ri  a value xiis missing dependes on x from  top left to  bottom rightReferencesAndrews D F and Pregibon D  Finding outliers that matter Journal of theRoyal Statistical Society Series B   Buck S F  A method of estimation of missing values in multivariate data suitablefor use with an electronic computer Journal of the Royal Statistical Society SeriesB   Cook R D and Weisberg S  Regression diagnostics with dynamic graphicsTechnometrics   Park S H Kim Y H and Toutenburg H  Regression diagnostics for removingan observation with animating graphics Statistical Papers   Rubin D B  Inference and missing data Biometrika   Rubin D B  Multiple Imputation for Nonresponse in Sample Surveys WileyNew YorkSchafer J L  Analysis of Incomplete Multivariate Data Chapman and HallLondonScharin B and Toutenburg H  Weighted mixed regression Zeitschrift fur Angewandte Mathematik und Mechanik   Simon G A and Simono J S  Diagnostic plots for missing data in least squaresregression Journal of the American Statistical Association 	  Simono J S  Regression diagnostics to detect nonrandom missingness in linearregression Technometrics   Theil H and Goldberger A S  On pure and mixed estimation in econometricsInternational Economic Review   Toutenburg H and Shalabh  Prediction of response values in linear regressionmodels from replicated experiments to be published 

Using diagnostic measures to detect non-MCAR processes in linear regression models with missing covariates

Universität München: Elektronischen Publikationen

English

https://epub.ub.uni-muenchen.de/1594/1/paper_204.pdf

Using diagnostic measures to detect non-MCAR processes in linear regression models with missing covariates

Abstract

Similar works

Full text

Available Versions

Open Access LMU

Universität München: Elektronischen Publikationen