Hatzinger, Reinhold

Rusch, Thomas

Wurzer, Marcus

English

Elektronische Publikationen der Wirtschaftsuniversität Wien

Chain Graph Models in R:Implementing the Cox-WermuthProcedureSLIDE 1 Psychoco 2013, 14-02-2013Outline1 Brief Introduction to Graphical Models2 The coxwer function: Fitting Chain Graph Models via theCox-Wermuth Heuristic3 Illustration: Contraceptive Method Choice4 Conclusion and OutlookThis is joint work with Marcus Wurzer and Reinhold Hatzinger.SLIDE 2 Psychoco 2013, 14-02-2013Graphical Models: GeneralGraphical models (GM) allow multivariate analysis of complexdependency structuresThey are probability distributions over a multidimensional spaceencoded by graphs (as a set of vertices/variables, V, and a set ofedges/relationships between variables, E)Different types: undirected GM (e.g., Markov random fields),directed GM (e.g., Bayesian Networks, DAG), Chain GMGM represent multivariate dependencies by conditionaldependence and independence statementsThus they can help in reducing overall complexity and allowmodel formulation, identification and selectionSLIDE 3 Psychoco 2013, 14-02-2013Graphical Models: ExampleA simple graphical model (a Markov random field):In GM the Markov property of graphs allows to factorize thedistribution FV into a set of conditional distributions, e.g., forV = {A,B,C,D} by way of densities: fV = fA|B × fB|C × fC|D × fDThus the problem of fitting graphical models effectively reducesto estimating a series of conditional distributionsSLIDE 4 Psychoco 2013, 14-02-2013Chain Graph Models: GeneralChain graph models (CGM) are a mixture of directed andundirected graphical modelsThey are particularly interesting for social and behavioralsciences (observational studies, complex multivariatedependencies, existing substantive knowledge)In CGM, all variables are assigned to boxes (disjoint variablesubsets Vt,V =⋃t Vt) by theory or substantive knowledgeBetween boxes exist directed edges, within boxes the edges areundirectedTwo types of CGM:Univariate recursive regression graph model (URRG; one variableper block)Joint response chain graph model (JRCG; more than one variableper block)SLIDE 5 Psychoco 2013, 14-02-2013Chain Graph Models: FactorizationA joint response chain graph model:In CGM factorization happens at least recursively betweenblocks: fV = fVT |VT−1,...,V1 × fVT−1|VT−2,...,V1 × · · · × fV1 .Possibly additional conditional independence by missing edges,e.g., for the above graphfV = fF|C,E,D,A,B × fC,E,D|A,B × fA,B = fF|C,E × fC,D|A,B × fE|B × fA,BSLIDE 6 Psychoco 2013, 14-02-2013Chain Graph Models: EstimationFor CGM there are no theoretical restrictions on the form of theconditional distributions (though usually conditional Gaussiandistributions; Lauritzen & Wermuth, 1989)In particular variable types can be of mixed type within andbetween boxes (discrete and continuous components)General algorithms for computing estimates in every CGM underevery possible variable type specification are not yet availableFitting the conditional distributions of the factorization with aseries of multiple univariate conditional regressions is feasible(Wermuth & Cox, 2001)Cox & Wermuth (1996; see also Caputo et al., 1997) lay out ideasfor a data-driven, heuristic selection strategy to approximate theCGM by univariate conditional regressionsSLIDE 7 Psychoco 2013, 14-02-2013The coxwer Functionality in RWe implemented an algorithm based on the ideas of the Cox-Wermuthheuristic in R for approximate fitting of JRCG and URRG models.Currently, there are the following functions intended for the user:cw-class S3 class for objects from a Cox-Wermuth fitcoxwer Fit a JRCG or a URRG via Cox-Wermuthselection strategyprep_coxwer Setup of variable frame, block membership andvariable type (interactive)summary, print S3 methods for class cwplot, predictadjmatrix Extracts the adjacency matrixwrite_cw Writes and saves the graph in igraph formatSLIDE 8 Psychoco 2013, 14-02-2013Using the coxwer Function Icoxwer arguments are a variable frame and an observations ×variables data frame.The variable frame defines the block and type of a variable. Itmust have the same row names as the data frame has columnnames.type blockage cont 5wifeEdu ord 4husbEdu ord 4nrChild count 1wifeRel bin 4wifeWork bin 4husbOcc categ 4solIndex ord 3mediaExp bin 2contraceptive categ 1The prep_coxwer function allows to define the variable frameinteractively.SLIDE 9 Psychoco 2013, 14-02-2013Using the coxwer Function IIFurther arguments to coxwer are:adjfile: Save the adjacency matrix to this file.autodetect: Automatically assign the data type to the variables inthe data frame according to variable type in the variable frame.pen, signif: Parameters for screening and model selection. pen isthe penalty for the information criterion used in stepAIC andsignif the significance level when screening for higher-ordereffects and non-linearities.contrasts: The contrasts to be used for categorical predictors.Defaults to dummy coding for ordered and unordered factors.silent: Flag for whether model fitting progress should be printed.SLIDE 10 Psychoco 2013, 14-02-2013The coxwer Selection AlgorithmOur algorithm is roughly the following (cf. Caputo et. al., 1997):1 Start in the block with the lowest number2 Take one variable from that block. Fit main effects model with allthe variables in the same block or higher block.3 Screen for quadratic effects (metric variables) and two-wayinteractions by adding of single terms. Retain the ones with anassociated p-value < signif.4 Fit the model with main and retained effects.5 Use backward selection to reduce the model.6 Re-enter interactions for the terms that remain in the model.7 Use backward selection.8 Re-enter quadratic terms for remaining effects.9 Use backward selection.10 If other variables in the same block: Repeat for them. Else: jumpto next block and repeat.SLIDE 11 Psychoco 2013, 14-02-2013Univariate Models used by coxwerFor binary targets: binomial logistic modelsstats::glm(...,family=binomial,link=logit)For unrestricted continuous targets: OLS/Gaussian linear modelsstats::glm(...,family=gaussian,link=identity)For positive continuous targets: gamma or inverse Gaussian GLMstats::glm(...,family=Gamma,link=inverse)stats::glm(...,family=inverse.gaussian,link=1/mu2)For count targets: Poisson/negative binomial loglinear modelsMASS::glm.nb(...,link=log)For categorical targets: multinomial logistic modelsnnet::multinom(...,link=logit)For ordinal targets: proportional odds logistic modelsMASS::polr(...,link=logit)SLIDE 12 Psychoco 2013, 14-02-2013CMC: DataFor illustration we fit a JRCGM for contraceptive methods choice(CMC) in a subset of the 1987 National Indonesia ContraceptivePrevalence Survey (Lim et. al., 1999)Overall we have 1473 observations of married women on 10variables.Age (age; continuous)Education (wifeEdu; ordinal 1=low, 2, 3, 4=high)Husband’s education (husbEdu; ordinal 1=low, 2, 3, 4=high)Number of children ever born (nrChild; count)Religion (wifeRel; binary; 0=Non-Islam 1=Islam)Wife’s now working? (wifeWork; binary 0=Yes, 1=No)Husband’s occupation (husbOcc; categorical 1, 2, 3, 4)Standard-of-living index (soliNdex; ordinal 1=low, 2, 3, 4=high)Media exposure (mediaExp; binary 0=Good, 1=Not good)Contraceptive method used (contraceptive; categorical 1=No-use2=Long-term 3=Short-term)SLIDE 13 Psychoco 2013, 14-02-2013CMC: BlocksBlocksBlock 1 - Dependent variables: contraceptive, nrChildBlock 2 - Intermediate variable: mediaExpBlock 3 - Intermediate variable: solIndexBlock 4 - Intermediate variables: wifeEdu, husbEdu, wifeRel,wifeWork, husbOccBlock 5 - Purely explanatory variable: ageSLIDE 14 Psychoco 2013, 14-02-2013CMC: coxwer Results> cmc_prep <- prep_coxwer(cmc)> res.cmc <- coxwer(cmc_prep, cmc)TARGET: nrChild (poisson loglinear model)TARGET: contraceptive (multinomial logit model)TARGET: mediaExp (binomial logit model)TARGET: solIndex (proportional odds logit model)TARGET: wifeEdu (proportional odds logit model)TARGET: husbEdu (proportional odds logit model)TARGET: wifeRel (binomial logit model)TARGET: wifeWork (binomial logit model)TARGET: husbOcc (multinomial logit model)> print(res.cmc)Adjacency Matrix:1 2 3 4 5 6 7 8 9 101 age 0 1 1 1 1 0 1 1 1 12 wifeEdu 0 0 1 1 1 0 1 1 1 13 husbEdu 0 1 0 0 0 0 1 1 0 04 nrChild 0 0 0 0 0 0 0 0 0 15 wifeRel 0 1 1 1 0 0 1 1 0 06 wifeWork 0 1 0 1 0 0 0 0 0 07 husbOcc 0 1 1 0 1 0 0 1 0 08 solIndex 0 0 0 0 0 0 0 0 1 09 mediaExp 0 0 0 0 0 0 0 0 0 010 contraceptive 0 0 0 1 0 0 0 0 0 0SLIDE 15 Psychoco 2013, 14-02-2013CMC: Joint Response Chain Graph> plot(res.cmc)agewifeEduhusbEdunrChildwifeRelwifeWorkhusbOccsolIndexmediaExpcontraceptiveSLIDE 16 Psychoco 2013, 14-02-2013CMC: Target “nrChild”> plot(res.cmc)agewifeEduhusbEdunrChildwifeRelwifeWorkhusbOccsolIndexmediaExpcontraceptiveSLIDE 17 Psychoco 2013, 14-02-2013CMC: Model for “nrChild”> summary(res.cmc,target=c("nrChild","contraceptive"))---------- Summary for dependent variable: nrChild ----------Call:stats::glm(formula = y ~ age + wifeEdu + wifeRel + wifeWork +contraceptive + I(poly(age, 2)[, 2]), family = curr.family,data = dat)Deviance Residuals:Min 1Q Median 3Q Max-3.3620 -0.6483 -0.1031 0.5343 3.5907Coefficients:Estimate Std. Error z value Pr(>|z|)(Intercept) -1.228343 0.110211 -11.145 < 2e-16 ***age 0.058168 0.002117 27.480 < 2e-16 ***wifeEdu2 0.012220 0.050068 0.244 0.807wifeEdu3 -0.075736 0.049643 -1.526 0.127wifeEdu4 -0.351352 0.049615 -7.082 1.42e-12 ***wifeRel1 0.263919 0.044373 5.948 2.72e-09 ***wifeWork1 0.171091 0.035053 4.881 1.06e-06 ***contraceptive2 0.334047 0.039516 8.454 < 2e-16 ***contraceptive3 0.348241 0.035753 9.740 < 2e-16 ***I(poly(age, 2)[, 2]) -5.163229 0.622035 -8.301 < 2e-16 ***---Signif. codes: 0 âA˘Ÿ***âA˘Z´ 0.001 âA˘Ÿ**âA˘Z´ 0.01 âA˘Ÿ*âA˘Z´ 0.05 âA˘Ÿ.âA˘Z´ 0.1 âA˘Ÿ âA˘Z´ 1(Dispersion parameter for poisson family taken to be 1)Null deviance: 2529.0 on 1472 degrees of freedomResidual deviance: 1452.1 on 1463 degrees of freedomAIC: 5529.9Number of Fisher Scoring iterations: 5SLIDE 18 Psychoco 2013, 14-02-2013CMC: Target “contraceptive”> plot(res.cmc)agewifeEduhusbEdunrChildwifeRelwifeWorkhusbOccsolIndexmediaExpcontraceptiveSLIDE 19 Psychoco 2013, 14-02-2013CMC: Model for “contraceptive”> summary(res.cmc,target=c("nrChild","contraceptive"))---------- Summary for dependent variable: contraceptive ----------Call:nnet::multinom(formula = y ~ age + wifeEdu + nrChild + I(poly(nrChild,2)[, 2]), data = dat, Hess = TRUE, trace = FALSE, MaxNWts = 5000)Coefficients:(Intercept) age wifeEdu2 wifeEdu3 wifeEdu4 nrChild2 -2.292873 -0.04835992 0.8820847 1.8373202 3.096257 0.35782423 1.745353 -0.11908511 0.2365778 0.6442521 1.337352 0.3558117I(poly(nrChild, 2)[, 2])2 -25.603743 -26.44224Std. Errors:(Intercept) age wifeEdu2 wifeEdu3 wifeEdu4 nrChild2 0.5138863 0.01211590 0.4047368 0.3869659 0.3816910 0.044443983 0.3756312 0.01136707 0.2482052 0.2452609 0.2461524 0.04057962I(poly(nrChild, 2)[, 2])2 3.5704543 3.223996Residual Deviance: 2708.166AIC: 2736.166SLIDE 20 Psychoco 2013, 14-02-2013ConclusionApplicabilityThe procedure allows to explore multivariate dependencies andapproximate the real CGMNeglects some information in the multivariate structure (loss ofefficiency)Validity of equivalence of Markovian properties for the wholegraph is not ensuredProgramIntended to further broaden the availability and applicability ofalgorithms for graphical models in R.Provides a unified, user-friendly way of approximately fitting CGMwith mixed variable types.Implementation can be used as a building block in even morecomplicated computational tasks, e.g., Wurzer & Hatzinger (2013).The coxwer procedure is not very fast and computing timeincreases massively for a large number of variables.SLIDE 21 Psychoco 2013, 14-02-2013OutlookCurrent future plansRelease it (look for gRchain or chaingraphs on R-Forge)Extend support to other variable typesFormula interface, normalizing of inputs and standardized effectsNew screening option that does not rely on p valuesNew model selection option by L1-regularizationNew way of treating within-block associationUnified model summaryAdd support for model diagnostics and interpretationLeverage/use/embed functionality offered in packages such asggraph, gRBase, igraph,...Incorporate measurement models/latent variablesSLIDE 22 Psychoco 2013, 14-02-2013References ICaputo, A., Heinicke, A. & Pigeot, I. (1997). A graphical chain model derivedfrom a model selection strategy for the sociologists graduates study.Collaborative Research Center 386, Discussion Paper 73.Cox, D. & Wermuth, N. (1996). Multivariate Dependencies: Models, Analysis,Interpretation. Florida:Chapman&Hall/CRC.Cox, D. & Wermuth, N. (2001). Joint response graphs and separation inducedby triangular systems. Research Report, Australian National University.Lauritzen, S. & Wermuth, N. (1989). Graphical models for associations betweenvariables, some of which are qualitative and some quantitative. The Annals ofStatistics, 31–57.Lim, T.-S., Loh, W.-Y. & Shih, Y.-S. (1999). A Comparison of Prediction Accuracy,Complexity, and Training Time of Thirty-three Old and New ClassificationAlgorithms.Machine Learning, 40, 203–238.SLIDE 23 Psychoco 2013, 14-02-2013References IIWermuth, N. (1998). Graphical Markov models. Encyclopedia of StatisticalScience. Update, 2.Wermuth, N. (2003). Analysing social science data with graphical Markovmodels. Oxford Statistical Science Series, 33–38.Wurzer, M. & Hatzinger, R. (2013). Using Graphical Models in Microsimulation.In C. O’Donoghue (ed.), New Pathways In Microsimulation. Ashgate Publishing,forthcoming.SLIDE 24 Psychoco 2013, 14-02-2013Thank you for your AttentionThomas RuschCenter for Empirical Research Methodsemail: thomas.rusch@wu.ac.atURL: http://wu.ac.at/methods/en/hum/truschWU Vienna University of Economics and BusinessAugasse 2–6, A-1090 ViennaSLIDE 25 Psychoco 2013, 14-02-2013

Chain Graph Models in R: Implementing the Cox-Wermuth Procedure

Abstract

Similar works

Full text

Available Versions

Elektronische Publikationen der Wirtschaftsuniversität Wien