The assessment of knowledge and learning in competence spaces: The gain–loss model for dependent skills

The gain–loss model (GaLoM) is a formal model for assessing knowledge and learning. In its original formulation, the GaLoM assumes independence among the skills. Such an assumption is not reasonable in several domains, in which some preliminary knowledge is the foundation for other knowledge. This paper presents an extension of the GaLoM to the case in which the skills are not independent, and the dependence relation among them is described by a well‐graded competence space. The probability of mastering skill s at the pretest is conditional on the presence of all skills on which s depends. The probabilities of gaining or losing skill s when moving from pretest to posttest are conditional on the mastery of s at the pretest, and on the presence at the posttest of all skills on which s depends. Two formulations of the model are presented, in which the learning path is allowed to change from pretest to posttest or not. A simulation study shows that models based on the true competence space obtain a better fit than models based on false competence spaces, and are also characterized by a higher assessment accuracy. An empirical application shows that models based on pedagogically sound assumptions about the dependencies among the skills obtain a better fit than models assuming independence among the skills.     

Positive associations primacy in the IAT: a many-facet rasch measurement analysis

Two studies investigated the different contribution of positive and negative associations to the size of the Implicit Association Test (IAT) effect. A Many-Facet Rasch Measurement analysis was applied for the purpose. Across different IATs (Race and Weight) and different groups of respondents (White, Normal weight, and Obese people) we observed that positive words increase the IAT effect whereas negative words tend to decrease it. Results suggest that the IAT is influenced by a positive associations primacy effect. As a consequence, we argue that researchers should be careful when interpreting IAT effects as a measure of implicit prejudice.     

A discrimination–association model for decomposing component processes of the Implicit Association Test

A formal model is proposed that decomposes the Implicit Association Test (IAT) effect into three process components: stimuli discrimination, automatic association, and termination criterion. Both response accuracy and reaction time are considered. Four independent and parallel Poisson processes, one for each of the four label categories of the IAT, are assumed. The model parameters are the rate at which information accrues on the counter of each process and the amount of information that is needed before a response is given. The aim of this study is to present the model and an illustrative application in which the process components of a Coca–Pepsi IAT are decomposed.     

Uncovering the Best Skill Multimap by Constraining the Error Probabilities of the Gain-Loss Model

The Gain-Loss model is a probabilistic skill multimap model for assessing learning processes. In practical applications, more than one skill multimap could be plausible, while none corresponds to the true one. The article investigates whether constraining the error probabilities is a way of uncovering the best skill assignment among a number of alternatives. A simulation study shows that this approach allows the detection of the models that are closest to the correct one. An empirical application shows that it allows the detection of models that are entirely derived from plausible assumptions about the skills required for solving the problems.     

Testing the actual equivalence of automatically generated items

If the automatic item generation is used for generating test items, the question of how the equivalence among different instances may be tested is fundamental to assure an accurate assessment. In the present research, the question was dealt by using the knowledge space theory framework. Two different ways of considering the equivalence among instances are proposed: The former is at a deterministic level and it requires that all the instances of an item template must belong to exactly the same knowledge states; the latter adds a probabilistic level to the deterministic one. The former type of equivalence can be modeled by using the BLIM with a knowledge structure assuming equally informative instances; the latter can be modeled by a constrained BLIM. This model assumes equality constraints among the error parameters of the equivalent instances. An approach is proposed for testing the equivalence among instances, which is based on a series of model comparisons. A simulation study and an empirical application show the viability of the approach.     

Assessing the local identifiability of probabilistic knowledge structures

Given a collection Q of problems, in knowledge space theory Doignon & Falmagne, (International Journal of Man–Machine Studies 23:175–196, 1985) the knowledge state of a student is the collection K ? Q of all problems that this student is capable of solving. A knowledge structure is a pair (Q, ), where is a collection of knowledge states that contains at least the empty set and Q. A probabilistic knowledge structure (PKS) is a knowledge structure (Q, , π), where π is a probability distribution on the knowledge states. The PKS that has received the most attention is the basic local independence model BLIM; Falmagne & Doignon, (British Journal of Mathematical and Statistical Psychology 41:1–23, 1988a, Journal of Mathematical Psychology 32:232–258, 1988b). To the best of our knowledge, systematic investigations in the literature concerning the identifiability of the BLIM are totally missing. Based on the theoretical work of Bamber and van Santen (Journal of Mathematical Psychology 29:443–473, 1985), the present article is aimed to present a method and a corresponding computerized procedure for assessing the local identifiability of the BLIM, which is applicable to any finite knowledge structure of moderate size.     

Assessing Parameter Invariance in the BLIM: Bipartition Models

In knowledge space theory, the knowledge state of a student is the set of all problems he is capable of solving in a specific knowledge domain and a knowledge structure is the collection of knowledge states. The basic local independence model (BLIM) is a probabilistic model for knowledge structures. The BLIM assumes a probability distribution on the knowledge states and a lucky guess and a careless error probability for each problem. A key assumption of the BLIM is that the lucky guess and careless error probabilities do not depend on knowledge states (invariance assumption). This article proposes a method for testing the violations of this specific assumption. The proposed method was assessed in a simulation study and in an empirical application. The results show that (1) the invariance assumption might be violated by the empirical data even when the model’s fit is very good, and (2) the proposed method may prove to be a promising tool to detect invariance violations of the BLIM.     

The effect of job quality on quality of life and wellbeing in later career stages: A multilevel and longitudinal analysis on older workers in Europe

Applied Research in Quality of Life - This paper uses data from four waves of the Survey of Health, Ageing and Retirement in Europe (SHARE) and investigates the effect of job quality on CASP-12,...     

Assessing the impact of replication on implicit association test effect by means of the extended logistic model for the assessment of change

The present study consists of a longitudinal investigation in which implicit association measures were collected by means of the implicit association test (IAT), with the purpose of assessing the impact of replication on the test effect with the same participants. The extended logistic model for the assessment of change (ELMAC), a procedure that belongs to Rasch modeling, was applied. By means of the ELMAC application, the existence of latent traits characteristic of the impact on the IAT effect (due to replication through time) is shown; more precisely, the model time parameters show the decrement of the IAT effect through time, due to the decrease of the implicit preference in both conditions, compatible and incompatible, associated with sweet foods and salty foods. The results confirm that in the long run, the incompatible condition latency may approach the compatible condition latency, causing a significant attenuation of the IAT effect.