Fienberg, Stephen E.

Meyer, Michael M.

1 online resource (PDF, 6 pages)Fienberg, Stephen E.; Meyer, Michael M.. (1979). Alternative Computational Methods for Estimation of Logit Model Parameters. Retrieved from the University Digital Conservancy, https://hdl.handle.net/11299/199349

University of Minnesota Digital Conservancy

' .. ALTERNATIVE COMPUTATIONAL METHODS FOR ESTIMATION OF LOGIT MODEL PARAMETERS by Stephen E. Fienberg and Michael M. Meyer University of Minnesota and G.W. Stewart University of Maryland Technical Report No. 348 June 1979 .. ··.,."" ALTERNATIVE COMPtrrATIONAL METHODS FOR ESTIMATION OF LOGIT MODEL PARAMETERS Stephen E. Fienberg and Michael M. Meyer School of Statistics, University of Minnesota St. Paul, Minnesota, U.S.A. and G. W. Stewart Department of Computer Science, University of Maryland College Park, Maryland, U.S.A. INTRODUCTION The analysis of cross-classified categorical data involves statistical problems where both the explanatory variables (or factors) and response variables are categorical. Loglinear and legit models are now widely-used.in the analysis of such data (e.g. see Bishop, Fienberg, and Holland, 1975; Fienberg 1977; Haberman, 1974, 1978). Since logit models are loglinear models, the computational methods for the analysis of cross-classifications via loglinear models can also be used for analyses involving the use of· legit models. In this paper we.compare three different computational approaches for maximum likelihood estimation in logit situations: (a) iterative proportional fitting, (b) iteratively reweighted least squares as implemented in GLIM (see Nelder and Wedderburn, 1972), (c) a variant of Newton's method, as developed by Fienberg and Stewart (1979), applied in a somewhat different form for loglinear and logit formulations. Additional comparisons can be ~ade with the Newton-Raphson algorithm of Haberman (1978), but they are not included here. LOGLINEAR AND LOGIT MODELS For simplicity, consider a problem involving two explanatory variables with dimensions I and J, and a response variable of dimension K. Thus the data are counts in the form of an IxJxK table where the totals in the IxJ margin corresponding to the explanatory variables are taken as fixed. We assume that the sampling model for the counts is product-multinomial (Bishop, Fienberg, and Holland, 1975). Logit models (involving K-1 simultaneous legit equations) for this problem are equivalent to loglinear models that treat the three variables as if they are responses, and that include u-terms corresponding to the main effects and interaction for the explanatory variables. (Fienberg, 1977, Chapter 6). Thus the iterative proportional fitting (IPF) algorithm used for loglinear models, can be used directly for logit models in this problem. 2 NEWTON'S METHOD Fienberg and Stewart (1979) have used a variant of Newton's method for analyzing both loglinear and logit models. Their first algorithm treats the logit model parameters as loglinear model parameters, and adjusts for the required conditioning at the end of the computation. A second algorithm proc~eds by initially conditioning on the explanatory variables, and then using a somewhat different set of computations. These algorithms involve the construction of the upper half of a pxp weighted cross-product matrix, and take full advantage of the sparseness of the nxp design matrix without actually con-structing it. The algorithms proceed via Newton's method with variable step length, using a Cholesky decomposition with pivoting. Further details will be reported in the near future. GLIM The GLIM algorithm, as developed by Nelder and his colleagues, is designed to handle the product multinomial sampling model for binary response structures, as well as other sampling models not considered here. For further details see Nelder and Wedderburn (1972). In order to handle a K-level response variable in GLIM, the user needs to treat it in an asymmetric fashion, eg. via the use of continuation ratios (see Fienberg, 1977, Chapter 6). COMPARISONS We have summarized the qualitative aspects of the four algorithms in question in Table 1. Because the Fienberg-Stewart algorithms have opted for economy of storage over efficiency of operation, the comparisons of storage requirements between their algorithms and GLIM is misleading. In practice GLIM cannot handle as large problems as can be handled by Fienberg-Stewart algorithms. Although we have not made direct comparisons here on speed of convergence, we note that IPF has linear ~onvergence properties while the other algorithms have quadratic convergence. We expect that the special features in the Fienberg-Stewart algorithms should allow for convergence at a slightly faster rate than GLIM, but this should not be a serious difference between these methods. More important is the issue of numerical stability of the algorithms, where again we anticipate the superiority of the Fienberg-Stewart ones. Whether one should use the legit or loglinear version of the Fienberg-Stewart algorithms depends on the size of the marginal array corresponding to the explanatory variables. When this margin is small some advantages may accrue to the loglinear approach. ,;>· .. ACKNOWLEDGEMENT This research was supported by Office of Naval Research Contracts N00014-78-C-0151 and N00014-78-C-0600 to the University of Minnesota. Reproduction in whole or in part is permitted for any purpose of the United States Government. REFERENCES 3 Bishop, Y.M.M., Fienberg, S.E. and Holland, P.W. (1975). Discrete Multivariate Analysis: Theory and Practice. Cambridge, Mass., The MIT Press. Fienberg, S.E. (1977). The Analysis of Cross-classified Categorical Data. Cambridge, Mass., The MIT Press. Fienberg, S.E. and Stewart, G.W. (1979). "The numerical analysis of contingency tables." Unpublished manuscript. Haberman, S.J. (1974). The Analysis of Frequency Data. Chicago, Ill., University of Chicago Press. Haberman, S.J. (1978). Analysis of Qualitative Data 2 Volume I 2 Intro-ductory Topics. New York, Academic Press. Nelder, J.A. and Wedderburn, R.W.M. (1972). Generalized linear models. J. Roy. Statist. Soc. A 135, 370-384. SUMMARY Several algorithms have been proposed for the computation of maximum likelihood estimates for contingency tables. Since logit models can be treated as special versions of loglinear models, many of ·these same techniques can be used for logit models as well. In this paper, we compare in a quali~ative fashion the relative merits of (i) the widely-used method of iterative proportional fitting, (ii) GLIM as developed by Nelder and Wedderburn, and (iii) two variants of Newton's method developed by Fienberg and Stewart. SOMMAIRE Plusieurs algorithmes ont ete proposes afin de calculer les evaluations de probabilite maximum pour les tables d'eventualite. Puisque les modeles "logit" pouvent etre traites comme un cas parti-culier des modeles logarithmiques et lineaires, la plupart des tech-niques qui s'appliquent aces demiers peuvent s'appliquer dont aussi bien aux modeles "logit". Dans cet article, nous comparons de £aeon qualitative les merits relatifs des methodes suivantes: (i) la methode frequemment utilisee de l'ajustement proportionnel iteratif, (ii) la methode GLIM, developpeepar Nelder et Wedderburn, et (iii) la methode des deux variants de Newton, developpee par Fienberg et Stewart. 1. 2. 3. 4. 5. 6. 7. 8. 9. 1 2 Table 1: General Comparison of Methods. (i) (ii) Fienberg-StewartFienberg-Stewart Criterion Loglinear _Lo git Utilizes conditioning No implied by logit model Handles Polytomous Yes response structures directly Can easily be extended to No general logisticregression· Storage requirements 1 SSP matrix. Detects non- Yes existence of MLE's Handles structural zeros Yes Produces parameter Yes estimates Produces Covariance Yes estimates Detects Aliasing Yes, via pivoted Cholesky decomposition. SSP. matrix is for all loglinear parameters. SSP matrix is for logit parameters only. Yes Yes Yes 2 SSP matrix. Yes Yes Yes Yes Yes, via pivoted Cholesky decomposition• {iii) GLIM Yes No Yes SSP 2 matrix. No, only by non-convergence. No Yes Yes Yes, via small diagonal elements in non-pivoting Gaussian elimination. (iv) IPF No Yes No Data array, plus margins. No, only by slow convergence to zero Yes Only for complete tables No Not applicable. t I 5 c;ECUl'.ITY CLASSI Fl CATION OF THIS PAGE (Mien Data Ente.,ed) REPORT DOCUMENTATION PAGE READ INSTRUCTIONS BEFORE COMPLETING FORM 1. REPORT NUMBER Technical Report No. 348 12· GOVT ACCESSION NO. 3. RECIPIENT'S CATALOG NUMBER 4. TITLE (and Subtitle) ALTERNATIVE COMPUTATIONAL METHODS FOR ESTIMATION OF LOGIT MODEL PARAMETERS 7, AUTHOR(•) Stephen E. Fienberg, Michael M. Meyer, G. W. Stewart 9. PERFORMING ORGANIZATION NAME ANO ADDRESS Department of Applied Statistics School of Statistics, University of Minnesota 1994 Buford Avenue, St. Paul, MN 55108 I I. CONTROLLING OFFICE NAME ANO ADDRESS Office of Naval Research 800 N. Quincy Street Arlington, VA 22217 14. MONITORING AGENCY NAME 6 A00RESS(II dllletent ftom Controllln,i Otllee) 16, DISTRIBUTION STATEMENT (ot tide Repott) 5. TYPE OF REPORT Iii PERIOD COVERED 6. PERFORMING ORG. REPORT NUMBER 8. CONTRACT OR GRANT NUMBER(•) N00014-78-C-0600 10, PROGRAM ELEMENT, PROJECT, TASK AREA & WORK UNIT NUMBERS 12. REPORT DATE June 1979 13. NUMBER OF PAGES 4 15. SECURITY CLASS, (ol thl• tepott) Unclassified 15•, DECLASSIFICATION/DOWNGRADING SCHEDULE Approved for public release: Distribution unlimited. 17. DISTRIBUTION STATEMENT (ol the •bet,act entered In BlocJc 20, U dlllerent &om Repon) 18, SUPPLEMENTARY NOTES 19. KEY WORDS (Continue on tentrH •Id• II nee•••_,. and Identity bJ' block number) Categorical data; logit models; loglinear models; iterative proportional fitting; GLIM; Newton's method. 20. ABSTRAC:T (Continue on t•v••• •Id• II nece••_,. and Identity by block numbet) DD In this paper we compare three different computational approaches for maximum likelihood estimation in legit situations: FORM (a) iterative proportional fitting, {b) iteratively reweighted least squares as implemented in GLL.'1. (see Nelder and Wedderburn, 1972), (c) a variant of Newton's method, as developed by Fienberg and Stewart (1979), .~pplied in a somewhat different form for loglinear and logit formulations. 1 JAN 73 1473 EDITION OF I NOV S5 IS O&SOLETE SIN 0102-LF-014-6601 SECURITY CLASSIFICATION OF THIS P-'G£ (lftteft Det• Bnler*') 

Alternative Computational Methods for Estimation of Logit Model Parameters

https://conservancy.umn.edu/bitstream/handle/11299/199349/Technical%20Report%20348%20Alternative%20Computational%20Methods%20for%20Estimation%20of%20Logit%20Model%20Parameters%20Feinberg%20_%20Meyer.pdf?sequence=1&isAllowed=y

Alternative Computational Methods for Estimation of Logit Model Parameters

Abstract

Similar works

Full text

Available Versions

University of Minnesota Digital Conservancy