Extending the Basic Local Independence Model to Polytomous Data

Psychometrika - A probabilistic framework for the polytomous extension of knowledge space theory (KST) is proposed. It consists in a probabilistic model, called polytomous local independence model,...     

On the empirical indistinguishability of knowledge structures

In recent years a number of articles have focused on the identifiability of the basic local independence model. The identifiability issue usually concerns two model parameter sets predicting an identical probability distribution on the response patterns. Both parameter sets are applied to the same knowledge structure. However, nothing is known about cases where different knowledge structures predict the same probability distribution. This situation is referred to as ʻempirical indistinguishabilityʼ between two structures and is the main subject of the present paper. Empirical indistinguishability is a stronger form of unidentifiability, which involves not only the parameters, but also the structural and combinatorial properties of the model. In particular, as far as knowledge structures are concerned, a consequence of empirical indistinguishability is that the existence of certain knowledge states cannot be empirically established. Most importantly, it is shown that model identifiability cannot guarantee that a certain knowledge structure is empirically distinguishable from others. The theoretical findings are exemplified in a number of different empirical scenarios.     

An iterative procedure for extracting skill maps from data

The methodologies for the construction of a knowledge structure mainly refer to the query to experts, the skill maps, and the data-driven approaches. This last method is of growing interest in recent literature. In this paper, an iterative procedure for building a skill map from a set of data is introduced. This procedure is based on the minimization of the distance between the knowledge structure delineated by a given skill map and the data. The accuracy of the proposed method is tested through a number of simulation studies where the amount of noise in the data is manipulated as well as the kind of structure to be reconstructed. Results show that the procedure is accurate and that its performance tends to be sufficiently stable even with high error rates. The procedure is compared to two already-existing methodologies to derive knowledge structures from a set of data. The use of the corrected Akaike Information Criterion (AICc) as a stopping criterion of the iterative reconstruction procedure is tested against the app criterion introduced by Schrepp. Moreover, two empirical applications on clinical data are reported, and their results show the applicability of the procedure.     

On the assessment of procedural knowledge: From problem spaces to knowledge spaces

By generalizing and completing the work initiated by Stefanutti and Albert (2003, Journal of Universal Computer Science, 9, 1455), this article provides the mathematical foundations of a theoretical approach whose primary goal is to construct a bridge between problem solving, as initially conceived by Newell and Simon (1972, Human problem solving. Englewood Cliffs, NJ: Prentice-Hall.), and knowledge assessment (Doignon and Falmagne, 1985, International Journal of Man-Machine Studies, 23, 175; Doignon and Falmagne, 1999, Knowledge spaces. Berlin, Germany: Springer-Verlag.; Falmagne et al., 2013, Knowledge spaces: Applications in education. New York, NY: Springer-Verlag; Falmagne and Doignon, 2011, Learning spaces: Interdisciplinary applied mathematics. Berlin, Germany: Springer-Verlag.). It is shown that the collection of all possible knowledge states for a given problem space is a learning space. An algorithm for deriving a learning space from a problem space is illustrated. As an example, the algorithm is used to derive the learning space of a neuropsychological test whose problem space is well known: the Tower of London (TOL; Shallice, 1982, Philosophical Transactions of the Royal Society of London B: Biological Sciences, 298, 199). The derived learning space could then be used for adaptively assessing individual planning skills with the TOL.     

A class of k-modes algorithms for extracting knowledge structures from data

One of the most crucial issues in knowledge space theory is the construction of the so-called knowledge structures. In the present paper, a new data-driven procedure for large data sets is described, which overcomes some of the drawbacks of the already existing methods. The procedure, called k-states, is an incremental extension of the k-modes algorithm, which generates a sequence of locally optimal knowledge structures of increasing size, among which a “best” model is selected. The performance of k-states is compared to other two procedures in both a simulation study and an empirical application. In the former, k-states displays a better accuracy in reconstructing knowledge structures; in the latter, the structure extracted by k-states obtained a better fit.     

The assessment of knowledge and learning in competence spaces: The gain–loss model for dependent skills

The gain–loss model (GaLoM) is a formal model for assessing knowledge and learning. In its original formulation, the GaLoM assumes independence among the skills. Such an assumption is not reasonable in several domains, in which some preliminary knowledge is the foundation for other knowledge. This paper presents an extension of the GaLoM to the case in which the skills are not independent, and the dependence relation among them is described by a well‐graded competence space. The probability of mastering skill s at the pretest is conditional on the presence of all skills on which s depends. The probabilities of gaining or losing skill s when moving from pretest to posttest are conditional on the mastery of s at the pretest, and on the presence at the posttest of all skills on which s depends. Two formulations of the model are presented, in which the learning path is allowed to change from pretest to posttest or not. A simulation study shows that models based on the true competence space obtain a better fit than models based on false competence spaces, and are also characterized by a higher assessment accuracy. An empirical application shows that models based on pedagogically sound assumptions about the dependencies among the skills obtain a better fit than models assuming independence among the skills.     

A characterization of the concept of independence in knowledge structures

In knowledge space theory a knowledge structure provides a deterministic representation of the implications among the items in a given set Q. Concrete procedures for the efficient assessment of knowledge by means of a knowledge structure have been proposed by Doignon and Falmagne [Falmagne, J.-C., & Doignon, J.-P. (1988a). A class of stochastic procedures for the assessment of knowledge. British Journal of Mathematical and Statistical Psychology, 41, 1-23; Falmagne, J.-C., & Doignon, J.-P. (1988b). A markovian procedure for assessing the state of a system. Journal of Mathematical Psychology, 32, 232-258]. The primitive idea at the core of such procedures is that the (correct or wrong) answers of a student to a subset A⊆Q of items could be inferred from the answers to a subset B⊆Q of items that were previously presented to that student. Since B provides information about A, from the viewpoint of the teacher these two subsets are not independent. This idea of dependence vs. independence is formalized in this paper in terms of an independence relation on the power set of Q. A nice characterization of this relation allows to express an arbitrary knowledge structure as the combination of a number of substructures each of which is independent of each other. An algorithm is then proposed which checks for independence in a knowledge structure and decomposes this last into a collection of independent substructures.     

A discrimination–association model for decomposing component processes of the Implicit Association Test

A formal model is proposed that decomposes the Implicit Association Test (IAT) effect into three process components: stimuli discrimination, automatic association, and termination criterion. Both response accuracy and reaction time are considered. Four independent and parallel Poisson processes, one for each of the four label categories of the IAT, are assumed. The model parameters are the rate at which information accrues on the counter of each process and the amount of information that is needed before a response is given. The aim of this study is to present the model and an illustrative application in which the process components of a Coca–Pepsi IAT are decomposed.     

On the unidentifiability of a certain class of skill multi map based probabilistic knowledge structures

In the process of fitting a probabilistic knowledge structure to data, standard goodness-of-fit statistics only partially describe the correctness of the fitted model. Irrespectively of how good the fit is, a too-high value of the error rates (careless error and lucky guess probabilities) might be a symptom of a misspecification of the model. In this situation, it could be critical to interpret those values as error rates. A more reasonable solution would be to hypothesize that some modifications have to be introduced in the model. In this paper, we show that in specific cases, these modifications yield basic local independence model parameterizations that are not identifiable. The applicative consequences of the theoretical results are displayed by means of an example carried out on a set of clinical data collected through the Maudsley Obsessional-Compulsive Questionnaire.     

Uncovering the Best Skill Multimap by Constraining the Error Probabilities of the Gain-Loss Model

The Gain-Loss model is a probabilistic skill multimap model for assessing learning processes. In practical applications, more than one skill multimap could be plausible, while none corresponds to the true one. The article investigates whether constraining the error probabilities is a way of uncovering the best skill assignment among a number of alternatives. A simulation study shows that this approach allows the detection of the models that are closest to the correct one. An empirical application shows that it allows the detection of models that are entirely derived from plausible assumptions about the skills required for solving the problems.