Wiering (2005, Statistics and Probability Letters, 75, 211-218) provides conditions

for the identifiability of a class of latent models. Here we derive an alternative more

general method of proving this result, which is based on standard identi¯ability

methods involving forming Jacobians

Cole, Diana J.

English

Kent Academic Repository

A note on the identifiability of certain latentclass modelsDiana ColeIMSAS, University of Kent, Canterbury, UKAbstractWiering (2005, Statistics and Probability Letters, 75, 211-218) provides conditionsfor the identifiability of a class of latent models. Here we derive an alternative moregeneral method of proving this result, which is based on standard identifiabilitymethods involving forming Jacobians.Key words: Computer algebra, identifiability, Latent class model, Maple, symbolicalgebra1 IntroductionWieringen (2005) examines the identifiability of a certain class of latent mod-els using a set of results that are specific to this type of problem. Here weshow how a general symbolic method, similar in concept to that used byGoodman (1974) for latent class models, can be used to determine the iden-tifiability of this class of latent models. A parameterisation is identifiable ifit is one-to-one. Identifiability can be further classified as locally and globallyidentifiable, where global refers to the identifiability holding for the whole pa-rameter domain and local identifiability refers to the identifiability holding fora neighbourhood of the parameter domain.The latent class of models of Wieringen (2005) is defined by the unconditionalprobability of manifest random variable Xi = [Xi1, . . . , Xin] occurring:P (Xi = xi; Ψ)= θ¯m∏j=1 ljxij p¯ilj−xijj,0 pixijj,0 + θ m∏j=1 ljxij p¯ilj−xijj,1 pixijj,1 (1)Email address: d.j.cole@kent.ac.uk (Diana Cole).Preprint submitted to Elsevier 26 August 2009with z¯ = 1−z and where the parameters are Ψ = [θ, pi1,1, . . . , pim,1, pi1,0, . . . , pim,0].Wieringen (2005) show that the class of latent model is globally identifiable if−1 +m∏j=1(lj + 1) ≥ 2m+ 1, (2)with the condition pij,0 6= pij,1 for any j. Here we provide an alternative proof.The alternative approach involves comparing the rank of a Jacobian matrix.This was first considered for latent models in Goodman (1974) although theorigins of this method can be attributed to Rothenberg (1971). Here an ex-tended version of this approach is used, where a Jacobian matrix is formed bydifferenting an exhaustive summary of the model with respect to the param-eters, with an exhaustive summary defined as a vector that uniquely definesthe model. Regardless of the exhaustive summary used, the model will be non-identifiable if the rank of the Jacobian is less than the number of parameters(Cole &Morgan, 2009a). An exhaustive summary for this class of latent modelsare the P (Xi = xi; Ψ) given by equation (1). This exhaustive summary consistsof∏mj=1(lj+1) elements. Due to the constraint∑P (Xi = xi; Ψ) = 1, a simplerexhaustive summary can be created by removing one of the P (Xi = xi; Ψ). Asthere are 2m+1 parameters and only∏mj=1(lj+1)−1 elements in this simplerexhaustive summary, it is obvious that a model would be non-identifiable ifinequality (2) does not hold (Wieringen, 2005, Remark 5). The alternativeproof that the class of latent models is actually globally identifiable if inequal-ity (2) holds is given in Section 2. The symbolic algebra involved in the proofis executed using the symbolic algebra package Maple; the Maple worksheetis available as supplementary material.2 Extended Jacobian MethodProving global identifiability is achieved by first finding a simpler exhaustivesummary for specific values of m and li using reparameterisation of the origi-nal model (Cole & Morgan, 2009a) and then generalising these results for anym and li using the extension theorem (Catchpole & Morgan, 1997, Theorem6). Consider the case when m = 3 and lj = 1 for all j. This particular caseis considered as a model for a naive Bayesian network (Whiley, 1999, Chap-ter3). Using reparameterisation Cole & Morgan (2009a) show that a simplerexhaustive summary iss =[θ, θ¯pi1,0, θpi1,1, θ¯pi1,0pi2,0, θpi1,1pi2,1, θ¯pi1,0pi2,0pi3,0, θpi1,1pi2,1pi3,1]T. (3)2Next consider changing l1 = 1 to l1. The exhaustive summary consisting ofthe P (Xi = xi; Ψ) given by equation (1) can be split into two parts. Firstconsider the exhaustive summary terms derived from xi1 = lj and xi1 =0. These terms are reparameterised in terms of si from equation (3). Thisreparameterised part of the exhaustive summary is differentiated with respectto the 7 si and the resulting derivative matrix is found to have rank 7. Therest of the exhaustive summary terms form the second part, as know new siparameters are added this is a trivial application of the extension theorem(Catchpole & Morgan, 1997, remark 7) and therefore the extended model alsohas full rank 7 (see Maple worksheet for details). Similar results would holdfor changing a different lj or for changing two or more lj. Therefore (3) is aexhaustive summary for any lj.As m increases the exhaustive summary can be extended tose =θ(1− θ)pi1,0θpi1,1(1− θ)pi1,0pi2,0θpi1,1pi2,1...(1− θ)pi1,0pi2,0 . . . pim,0θpi1,1pi2,1 . . . pim,1. (4)as long as pii,0 6= pii,1 for all i. This can also be shown to be true using theextension theorem. (The details for which are given in the Maple worksheet).The exhaustive summary given by equation (4) can then be differentiatedwith respect to the original parameters to form a Jacobian matrix. If therank of this Jacobian matrix is less than the number of parameters then thelatent model is non-identifiable. This is confirmed to be true for m ≥ 3 inthe Maple worksheet. Theorem 7 of Cole & Morgan (2009a) states a modelis globally identifiable if there is a unique solution to se = k (where se is anexhaustive summary the same length as the number of parameters). Here theunique solution is of the form θ = k1, pi1,0 =k21−k1 , pi1,1 =k3k1,. . ., pim,1 =k2m+1k2m−1.Therefore this latent model class is always globally identifiable for m ≥ 3. Thecases m = 2 and m = 1 are show in a similar vein in the Maple worksheet.33 DiscussionThe concept of being able to use an exhaustive summary as the basis for de-termining identifiability is very general and is applicable to any parametricmodel. The long-established Jacobian method for testing for local identifia-bility certainly more useable for simpler exhaustive summaries, such as thosethat result from the reparameterisation method. Global identifiability can alsobe determined easily from any simple exhaustive summary that is the samelength as the number of parameters. Here we have shown how the identifiabil-ity of a particular class of latent models can be determined using this generalmethod. It can be seen that the application of the extension theorem is veryuseful for generalising identifiability results.This Jacobian method also has the advantage that it is easy to generalise toother families of latent class models such as those with covariates. Examplesof latent class models with covariates are given in Huang & Bandeen-Roche(2004) and Forcina (2008). The extension of this Jacobian method to includecovariates is examined in Cole & Morgan (2009b).ReferencesCatchpole, E. A. and Morgan, B. J. T., Detecting parameter redundancy.Biometrika, 1997, 84, 187-196.Cole, D.J., & Morgan, B.J.T, Determining the Parametric Structureof Non-Linear Models, 2009a. University of Kent Technical Report.UKC/IMS/09/005 www.kent.ac.uk/ims/personal/djc24/para.pdfCole, D.J., & Morgan, B.J.T, Parameter Redundancy with Covari-ates. 2009b University of Kent Technical Report. UKC/IMS/07www.kent.ac.uk/ims/personal/djc24/cova.pdfForcina, A. Identifiability of extended latent class models with individual co-variates, 2008, Computational Statistics and Data Analysis, 52, 52635268Huang G., Bandeen-Roche, K., Building an identifiable latent class model withcovariat effects on underlying and measuring variables. Psychometrika, 69,5-32Goodman, L. A., Exploratory latent structure analysis using both identifiableand unidentifiable models, 1974, Biometrika, 61, 215-231.Rothenberg, T. J., Identification in parametric models, 1971, Econometrica,39, 577-591.Whiley, M. Aspects of the Interface Between Statistics and Neural Networks,1999, Ph.D Thesis University of Glasgow, Scotland.Wieringen, W. N., On identifiability of certain latent class models, 2005,Statistics & Probability Letters, 75, 211-218.4

Aspects of the Interface Between Statistics and Neural Networks,

Detecting parameter redundancy. Biometrika,

Determining the Parametric Structure of Non-Linear Models,

Identification in parametric models,

On identifiability of certain latent class models,

Parameter Redundancy with Covariates. 2009b University of Kent Technical Report. UKC/IMS/07 www.kent.ac.uk/ims/personal/djc24/cova.pdf Forcina, A. Identifiability of extended latent class models with individual covariates,

A note on the identifiability of certain latent class models

http://kar.kent.ac.uk/41230/1/latentclass.pdf

A note on the identifiability of certain latent class models

Abstract

Similar works

Full text

Available Versions

Kent Academic Repository