Knowledge Space Theory (KST) links in several ways to Formal Concept Analysis (FCA). Recently, the probabilistic and statistical aspects of KST have been further developed by several authors. We review part of the recent results, and describe some of the open problems. The question of whether the outcomes can be useful in FCA remains to be investigated

Doignon, Jean-Paul

English

Technische Universität Dresden: Qucosa

Identifiability in Knowledge Space Theory:a survey of recent resultsJean-Paul DoignonUniversité Libre de Bruxelles,Bd du Triomphe, c.p. 216,B–1050 Bruxelles.Belgium.doignon@ulb.ac.beAbstract. Knowledge Space Theory (KST) links in several ways to For-mal Concept Analysis (FCA). Recently, the probabilistic and statisticalaspects of KST have been further developed by several authors. We re-view part of the recent results, and describe some of the open problems.The question of whether the outcomes can be useful in FCA remains tobe investigated.Keywords: knowledge space, Basic Local Independence Model, CorrectResponse Model, model identifiabilityIn Knowledge Space Theory (KST, see Doignon & Falmagne, 1999; Falmagne& Doignon, 2011), a body of knowledge is represented by a finite set, say Q, oftest items. The knowledge state of a student is identified with the collection ofitems he masters. Because of dependencies among the items, not any subset ofQ can be a knowledge state; for instance, if Q is structured by a prerequisiterelation, the states should be taken as the ideals of the transitive closure of theprerequisite relation. In general, the collection K of all possible knowledge statesforms a knowledge structure (Q,K); it is assumed ∅, Q ∈ K. The correctnessof the answer provided at a certain time by a student to any item is granted todepend only on his knowledge state, except for careless errors and lucky guesses.Because variations are routinely observed in such answers, a probabilisticextension of KST was designed. So, assume the knowledge state of a studentmay vary (around a certain time point of his apprenticeship) in K accordingto a probability distribution π on K. Moreover, for any item q in Q, let βq bethe probability of a careless error in answering q, and ηq be the probability ofa lucky guess in answering q. All the numbers π(K) (for K in K), βq and ηq(for q in Q) will be considered as parameters with (unknown) latent values. (Ofcourse, the π(K)’s are not independent parameters, because they add up to 1.)The straight case obtains when βq = ηq = 0, for any q in Q. We now proposetwo models for the probabilities of correctness of student answers (consideredas the observables). Both models are based on the latent knowledge structuretogether with the various parameters we have just introduced. The second model2 Jean-Paul Doignonis described in Doignon & Falmagne (1999) (see also Falmagne & Doignon, 2011),while the first one is only implicit there.The first model, the Correct Response Model (CRM), defines the probabilityτ(q) of a correct answer to any isolated item q. It first conditions the probabilityof a correct answer to item q on the state of the student:τ(q) =∑K∈KPr(q K) · π(K).Then, it specifies each conditional probability Pr(q K) by taking into accountthe careless error probabilities βq and the lucky guess probabilities ηq:Pr(q K) ={1− βq if q ∈ K,ηq if q /∈ K.(1)A second model, the Basic Local Independence Model (BLIM), defines theprobability of a pattern of responses. Here, a pattern is a subset of Q meant tocontain all items to which a student (at a given time) produces a correct answer.Exactly as the CRM, the BLIM conditions the pattern probability on the stateof the student. Thus, the probability of a given pattern R of responses (withR ⊆ Q) equalsρ(R) =∑K∈Kr(R,K) · π(K),where r(R,K) is specified as follows:r(R,K) =( ∏q∈K\Rβq)( ∏q∈K∩R(1− βq))( ∏q∈R\Kηq)( ∏q∈Q\(R∪K)(1− ηq)).Both of our models, the CRM and the BLIM, are instances of probabilisticmodels. On the basis of a fixed knowledge structure, they predict from anyparameter point (that is, any list of values for all parameters) some definiteprobability values for the observables (in our case, the observables are eitherindividual correct responses, or whole patterns of correct responses). We use theterm predicted distribution to designate “any distribution of probability valuesfor the observables that are predicted by the model”. The questions we willconsider are as follows (the first two are clearly stated in Bamber & van Santen,2000 for probabilistic models in general).1. Model testability: is there some distribution of probability values for theobservables that the model does not predict?2. Model identifiability: is each predicted distribution produced from at mostone parameter point?3. Model characterizability: are the predicted distributions susceptible of aneffective characterization (without reference to the underlying parametervalues)?Identifiability in Knowledge Space Theory 3Recently Spoto, Stefanutti & Vidotto (2012) have investigated the first twoquestions for the BLIM, the model for pattern probabilities. Moreover, Ste-fanutti, Heller, Anselmi & Robusto (2012) have produced additional, nice resultsabout identifiability of the BLIM, especially in its local version: local identifia-bility means identifiability when the model is restricted to some neighborhoodof any given parameter point.On our part, we consider the three types of questions for the CRM, thecorrect response model, however working mainly in the straight case. First, weare able to characterize testability of the model using a simple criterion (and alsoto reformulate a variant of it, numerical testability, in a manageable, technicalway). Second, about characterizability, we point out unavoidable difficulties inrecognizing when it holds. Third, as regards identifiability, we give a tractableequivalent, concluding that identifiability is not often met. On the positive side,we indicate how to modify the parameter domain (consisting of the knowledgestate probabilities) in order to restore identifiability while keeping the sameprediction range; nevertheless, we show that the construction works well onlyfor the knowledge structures (Q,K) which are derived from a quasi order onQ (as it is the case in the presence of a prerequisite relation). As a matter offact, the construction heavily relies on a theorem of Stanley (1986) for a convexpolytope he associates to a partial order.The results presented during the talk are taken from a manuscript underpreparation (Doignon, 2013).BibliographyBamber, D., & van Santen, J. P. H. (2000). How to assess a model’s testabilityand identifiability. J. Math. Psych., 44 , 20–40.Doignon, J.-P. (2013). A correct response model in knowledge space theory.Manuscript in preparation.Doignon, J.-P., & Falmagne, J.-C. (1999). Knowledge spaces. Berlin: Springer-Verlag.Falmagne, J.-C., & Doignon, J.-P. (2011). Learning Spaces. Berlin: Springer-Verlag.Spoto, A., Stefanutti, L., & Vidotto, G. (2012). Considerations about the iden-tification of forward- and backward-graded knowledge structures. Submitted.Stanley, R. (1986). Two poset polytopes. Discrete Comput. Geom., 1 , 9–23.Stefanutti, L., Heller, J., Anselmi, P., & Robusto, E. (2012). Assessing thelocal identifiability of probabilistic knowledge structures. Behavior ResearchMethods, 44 , 1197–1211.

Identifiability in Knowledge Space Theory: a survey of recent results

https://tud.qucosa.de/api/qucosa%3A26886/attachment/ATT-0/

Identifiability in Knowledge Space Theory: a survey of recent results

Abstract

Similar works

Full text

Available Versions

Technische Universität Dresden: Qucosa