The Heteroscedastic Graded Response Model with a Skewed Latent Trait: Testing Statistical and Substantive Hypotheses Related to Skewed Item Category Functions

The Graded Response Model (GRM; Samejima, Estimation of ability using a response pattern of graded scores, Psychometric Monograph No.?17, Richmond, VA: The Psychometric Society, 1969) can be derived by assuming a linear regression of a continuous variable, Z, on the trait, ??, to underlie the ordinal item scores (Takane & de Leeuw in Psychometrika, 52:393?C408, 1987). Traditionally, a normal distribution is specified for Z implying homoscedastic error variances and a normally distributed ??. In this paper, we present the Heteroscedastic GRM with Skewed Latent Trait, which extends the traditional GRM by incorporation of heteroscedastic error variances and a skew-normal latent trait. An appealing property of the extended GRM is that it includes the traditional GRM as a special case. This enables specific tests on the normality assumption of Z. We show how violations of normality in Z can lead to asymmetrical category response functions. The ability to test this normality assumption is beneficial from both a statistical and substantive perspective. In a simulation study, we show the viability of the model and investigate the specificity of the effects. We apply the model to a dataset on affect and a dataset on alexithymia.     

Is g an entity? A Japanese twin study using syllogisms and intelligence tests

Using a behavioral genetic approach, we examined the validity of the hypothesis concerning the singularity of human general intelligence, the g theory, by analyzing data from two tests: the first consisted of 100 syllogism problems and the second a full-scale intelligence test. The participants were 448 Japanese young adult twins (167 pairs of identical and 53 pairs of fraternal twins). Data were analyzed for their fit to two kinds of multivariate genetic models: a common pathway model, in which a higher-order latent variable, g, was postulated as an entity; and an independent pathway model, in which the higher-order latent variable was not posited. These analyses revealed that the common pathway model which included additive genetic and nonshared environmental factors best accounted for the three distinct mental abilities: syllogistic logical deductive reasoning, verbal, and spatial. Both the substantial g-loading for syllogism-solving, historically recognized as the symbol of human intelligence, and the emergence of g as an entity at an etiological level, that is, at the genetic and environmental factor level, provide further support for the g theory.     

A class of k-modes algorithms for extracting knowledge structures from data

One of the most crucial issues in knowledge space theory is the construction of the so-called knowledge structures. In the present paper, a new data-driven procedure for large data sets is described, which overcomes some of the drawbacks of the already existing methods. The procedure, called k-states, is an incremental extension of the k-modes algorithm, which generates a sequence of locally optimal knowledge structures of increasing size, among which a “best” model is selected. The performance of k-states is compared to other two procedures in both a simulation study and an empirical application. In the former, k-states displays a better accuracy in reconstructing knowledge structures; in the latter, the structure extracted by k-states obtained a better fit.     

The Latent Change Score Model: A More Flexible Approach to Modeling Time in Self-Regulated Learning

Purpose

This research advances understanding of empirical time modeling techniques in self-regulated learning research. We intuitively explain several such methods by situating their use in the extant literature. Further, we note key statistical and inferential assumptions of each method while making clear the inferential consequences of inattention to such assumptions.

Design/Methodology/Approach

Using a population model derived from a recent large-scale review of the training and work learning literature, we employ a Monte Carlo simulation fitting six variations of linear mixed models, seven variations of latent common factor models, and a single latent change score model to 1500 simulated datasets.

Findings

The latent change score model outperformed all six of the linear mixed models and all seven of the latent common factor models with respect to (1) estimation precision of the average learner improvement, (2) correctly rejecting a false null hypothesis about such average improvement, and (3) correctly failing to reject true null hypothesis about between-learner differences (i.e., random slopes) in average improvement.

Implications

The latent change score model is a more flexible method of modeling time in self-regulated learning research, particularly for learner processes consistent with twenty-first-century workplaces. Consequently, defaulting to linear mixed or latent common factor modeling methods may have adverse inferential consequences for better understanding self-regulated learning in twenty-first-century work.

Originality/Value

Ours is the first study to critically, rigorously, and empirically evaluate self-regulated learning modeling methods and to provide a more flexible alternative consistent with modern self-regulated learning knowledge.

     

Variables latentes et propriétés mentales : pour une épistémologie affirmée pragmatiste et réaliste

The reliability of models with latent variables is questioned by different authors: what is the ontology of mental properties? How to integrate the complex of mental processes with latent variables? All these debates raise the question of the legitimacy of latent variables as methodological approach comparatively to new approaches such as Network analysis. By clearly posing the ontological nature of mental properties as emergent properties of mental processes, and by clearly posing a theory of pragmatist and realistic knowledge, our work seeks to show that latent variables are efficient approaches for inferring mental properties.     

The dynamics of choice among multiple alternatives

We consider neurally based models for decision-making in the presence of noisy incoming data. The two-alternative forced-choice task has been extensively studied, and in that case it is known that mutually inhibited leaky integrators in which leakage and inhibition balance can closely approximate a drift-diffusion process that is the continuum limit of the optimal sequential probability ratio test (SPRT). Here we study the performance of neural integrators in n?2 alternative choice tasks and relate them to a multihypothesis sequential probability ratio test (MSPRT) that is asymptotically optimal in the limit of vanishing error rates. While a simple race model can implement this ‘max-vs-next’ MSPRT, it requires an additional computational layer, while absolute threshold crossing tests do not require such a layer. Race models with absolute thresholds perform relatively poorly, but we show that a balanced leaky accumulator model with an absolute crossing criterion can approximate a ‘max-vs-ave’ test that is intermediate in performance between the absolute and max-vs-next tests. We consider free and fixed time response protocols, and show that the resulting mean reaction times under the former and decision times for fixed accuracy under the latter obey versions of Hick's law in the low error rate range, and we interpret this in terms of information gained. Specifically, we derive relationships of the forms log(n-1), log(n), or log(n+1) depending on error rates, signal-to-noise ratio, and the test itself. We focus on linearized models, but also consider nonlinear effects of neural activities (firing rates) that are bounded below and show how they modify Hick's law.     

A joint modeling framework of responses and response times to assess learning outcomes

Abstract

A general modeling framework of response accuracy and response times is proposed to track skill acquisition and provide additional diagnostic information on the change of latent speed in a learning environment. This framework consists of two types of models: a dynamic response model that captures the response accuracy and the change of discrete latent attribute profile upon factors such as practice, intervention effects, and other latent and observable covariates, and a dynamic response time model that describes the change of the continuous response latency due to change of latent attribute profile. These two types of models are connected through a parameter, describing the change rate of the latent speed through the learning process, and a covariate defined as a function of the latent attribute profile. A Bayesian estimation procedure is developed to calibrate the model parameters and measure the latent variables. The estimation algorithm is evaluated through several simulation studies under various conditions. The proposed models are applied to a real data set collected through a spatial rotation diagnostic assessment paired with learning tools.     

A Doubly Latent Space Joint Model for Local Item and Person Dependence in the Analysis of Item Response Data

Item response theory (IRT) is one of the most widely utilized tools for item response analysis; however, local item and person independence, which is a critical assumption for IRT, is often violated in real testing situations. In this article, we propose a new type of analytical approach for item response data that does not require standard local independence assumptions. By adapting a latent space joint modeling approach, our proposed model can estimate pairwise distances to represent the item and person dependence structures, from which item and person clusters in latent spaces can be identified. We provide an empirical data analysis to illustrate an application of the proposed method. A simulation study is provided to evaluate the performance of the proposed method in comparison with existing methods.

     

Bayesian Estimation of the DINA Q matrix

Cognitive diagnosis models are partially ordered latent class models and are used to classify students into skill mastery profiles. The deterministic inputs, noisy “and” gate model (DINA) is a popular psychometric model for cognitive diagnosis. Application of the DINA model requires content expert knowledge of a Q matrix, which maps the attributes or skills needed to master a collection of items. Misspecification of Q has been shown to yield biased diagnostic classifications. We propose a Bayesian framework for estimating the DINA Q matrix. The developed algorithm builds upon prior research (Chen, Liu, Xu, & Ying, in J Am Stat Assoc 110(510):850–866, 2015) and ensures the estimated Q matrix is identified. Monte Carlo evidence is presented to support the accuracy of parameter recovery. The developed methodology is applied to Tatsuoka’s fraction-subtraction dataset.     

Nonparametric item response theory axioms and properties under nonlinearity and their exemplification with knowledge space theory

This paper investigates the dichotomous Mokken nonparametric item response theory (IRT) axioms and properties under incomparabilities among latent trait values and items. Generalized equivalents of the unidimensional nonparametric IRT axioms and properties are formulated for nonlinear (quasi-ordered) person and indicator spaces. It is shown that monotone likelihood ratio (MLR) for the total score variable and nonlinear latent trait implies stochastic ordering (SO) of the total score variable, but may fail to imply SO of the nonlinear latent trait. The reason for this and conditions under which the implication holds are specified, based on a new, simpler proof of the fact that in the unidimensional case MLR implies SO. The approach is applied in knowledge space theory (KST), a combinatorial test theory. This leads to a (tentative) Mokken-type nonparametric axiomatization in the currently parametric theory of knowledge spaces. The nonparametric axiomatization is compared with the assumptions of the parametric basic local independence model which is fundamental in KST. It is concluded that this paper may provide a first step toward a basis for a possible fusion of the two split directions of psychological test theories IRT and KST.     

A Multicomponent Latent Trait Model for Diagnosis

This paper presents a noncompensatory latent trait model, the multicomponent latent trait model for diagnosis (MLTM-D), for cognitive diagnosis. In MLTM-D, a hierarchical relationship between components and attributes is specified to be applicable to permit diagnosis at two levels. MLTM-D is a generalization of the multicomponent latent trait model (MLTM; Whitely in Psychometrika, 45:479–494, 1980; Embretson in Psychometrika, 49:175–186, 1984) to be applicable to measures of broad traits, such as achievement tests, in which component structure varies between items. Conditions for model identification are described and marginal maximum likelihood estimators are presented, along with simulation data to demonstrate parameter recovery. To illustrate how MLTM-D can be used for diagnosis, an application to a large-scale test of mathematics achievement is presented. An advantage of MLTM-D for diagnosis is that it may be more applicable to large-scale assessments with more heterogeneous items than are latent class models.     

Context Matters,but How Much? Latent state–trait analysis of cognitive ability assessments

Cognitive abilities are thought to represent temporally stable constructs, however, accumulating evidence suggests that effects of the measurement situation could affect its measurement (e.g., effects of test motivation, stress level). The present study modeled these effects explicitly in a latent variables approach. In contrast to previous studies, we investigated participants (job candidates) in repeated high‐stakes settings (N = 188). We found that cognitive ability measurements in high‐stakes settings not only reflect a stable latent trait and random measurement error, but also systematic effects of the test setting. Our results support the application of cognitive ability tests in organizational contexts but have implications for its use in applied settings such as personnel selection.     

Two Studies Examining the Error Theory Underlying the Measurement Model of the Verbal Aggressiveness Scale

Meta‐analysis indicates moderate correlations between the Verbal Aggressiveness Scale (VAS) and other self‐report measures but near‐zero correlations with behavioral measures. Accurately interpreting correlations between the VAS and other variables, however, requires an examination of the untested error theory underlying the measurement model for the VAS. In two separate studies, the results of single‐factor correlated uniqueness confirmatory factor analytic models revealed a pattern of significant error covariances indicating that VAS item scores are confounded by systematic error attributable to multiple unspecified latent effects. After pruning the item sets, we identified 4 items that were free of latent variable influences other than trait verbal aggressiveness. Implications for interpreting the verbal aggressiveness literature are discussed along with recommendations for revising the VAS.     

$${\in_K}$$: a Non-Fregean Logic of Explicit Knowledge

We present a new logic-based approach to the reasoning about knowledge which is independent of possible worlds semantics. \({\in_K}\) (Epsilon-K) is a non-Fregean logic whose models consist of propositional universes with subsets for true, false and known propositions. Knowledge is, in general, not closed under rules of inference; the only valid epistemic principles are the knowledge axiom K _i φ → φ and some minimal conditions concerning common knowledge in a group. Knowledge is explicit and all forms of the logical omniscience problem are avoided. Various stronger epistemic properties such as positive and/or negative introspection, the K-axiom, closure under logical connectives, etc. can be restored by imposing additional semantic constraints. This yields corresponding sublogics for which we present sound and complete axiomatizations. As a useful tool for general model constructions we study abstract versions of some 3-valued logics in which we interpret truth as knowledge. We establish a connection between \({\in_K}\) and the well-known syntactic approach to explicit knowledge proving a result concerning equi-expressiveness. Furthermore, we discuss some self-referential epistemic statements, such as the knower paradox, as relaxations of variants of the liar paradox and show how these epistemic “paradoxes” can be solved in \({\in_K}\). Every specific \({\in_K}\)-logic is defined as a certain extension of some underlying classical abstract logic.     

Limited‐information goodness‐of‐fit testing of diagnostic classification item response models

Despite the growing popularity of diagnostic classification models (e.g., Rupp et al., 2010, Diagnostic measurement: theory, methods, and applications, Guilford Press, New York, NY) in educational and psychological measurement, methods for testing their absolute goodness of fit to real data remain relatively underdeveloped. For tests of reasonable length and for realistic sample size, full‐information test statistics such as Pearson's X² and the likelihood ratio statistic G² suffer from sparseness in the underlying contingency table from which they are computed. Recently, limited‐information fit statistics such as Maydeu‐Olivares and Joe's (2006, Psychometrika, 71, 713) M₂ have been found to be quite useful in testing the overall goodness of fit of item response theory models. In this study, we applied Maydeu‐Olivares and Joe's (2006, Psychometrika, 71, 713) M₂ statistic to diagnostic classification models. Through a series of simulation studies, we found that M₂ is well calibrated across a wide range of diagnostic model structures and was sensitive to certain misspecifications of the item model (e.g., fitting disjunctive models to data generated according to a conjunctive model), errors in the Q‐matrix (adding or omitting paths, omitting a latent variable), and violations of local item independence due to unmodelled testlet effects. On the other hand, M₂ was largely insensitive to misspecifications in the distribution of higher‐order latent dimensions and to the specification of an extraneous attribute. To complement the analyses of the overall model goodness of fit using M₂, we investigated the utility of the Chen and Thissen (1997, J. Educ. Behav. Stat., 22, 265) local dependence statistic $X_{\mathrm{LD}}^2$ for characterizing sources of misfit, an important aspect of model appraisal often overlooked in favour of overall statements. The $X_{\mathrm{LD}}^2$ statistic was found to be slightly conservative (with Type I error rates consistently below the nominal level) but still useful in pinpointing the sources of misfit. Patterns of local dependence arising due to specific model misspecifications are illustrated. Finally, we used the M₂ and $X_{\mathrm{LD}}^2$ statistics to evaluate a diagnostic model fit to data from the Trends in Mathematics and Science Study, drawing upon analyses previously conducted by Lee et al., (2011, IJT, 11, 144).     

Combining Item Response Theory and Diagnostic Classification Models: A Psychometric Model for Scaling Ability and Diagnosing Misconceptions

Traditional testing procedures typically utilize unidimensional item response theory (IRT) models to provide a single, continuous estimate of a student’s overall ability. Advances in psychometrics have focused on measuring multiple dimensions of ability to provide more detailed feedback for students, teachers, and other stakeholders. Diagnostic classification models (DCMs) provide multidimensional feedback by using categorical latent variables that represent distinct skills underlying a test that students may or may not have mastered. The Scaling Individuals and Classifying Misconceptions (SICM) model is presented as a combination of a unidimensional IRT model and a DCM where the categorical latent variables represent misconceptions instead of skills. In addition to an estimate of ability along a latent continuum, the SICM model provides multidimensional, diagnostic feedback in the form of statistical estimates of probabilities that students have certain misconceptions. Through an empirical data analysis, we show how this additional feedback can be used by stakeholders to tailor instruction for students’ needs. We also provide results from a simulation study that demonstrate that the SICM MCMC estimation algorithm yields reasonably accurate estimates under large-scale testing conditions.     

Assessing Parameter Invariance in the BLIM: Bipartition Models

In knowledge space theory, the knowledge state of a student is the set of all problems he is capable of solving in a specific knowledge domain and a knowledge structure is the collection of knowledge states. The basic local independence model (BLIM) is a probabilistic model for knowledge structures. The BLIM assumes a probability distribution on the knowledge states and a lucky guess and a careless error probability for each problem. A key assumption of the BLIM is that the lucky guess and careless error probabilities do not depend on knowledge states (invariance assumption). This article proposes a method for testing the violations of this specific assumption. The proposed method was assessed in a simulation study and in an empirical application. The results show that (1) the invariance assumption might be violated by the empirical data even when the model’s fit is very good, and (2) the proposed method may prove to be a promising tool to detect invariance violations of the BLIM.     

Breast Cancer Risk Perceptions among Relatives of Women with Uninformative Negative <Emphasis Type="Italic">BRCA1/2</Emphasis> Test Results: The Moderating Effect of the Amount of Shared Information

The most common result of BRCA1/2 mutation testing when performed in a family without a previously identified mutation is an uninformative negative test result. Women in these families may have an increased risk for breast cancer because of mutations in non-BRCA breast cancer predisposition genes, including moderate- or low-risk genes, or shared environmental factors. Genetic counselors often encourage counselees to share information with family members, however it is unclear how much information counselees share and the impact that shared information may have on accuracy of risk perception in family members. We evaluated 85 sisters and daughters of women who received uninformative negative BRCA1/2 results. We measured accuracy of risk perception using a latent variable model where accuracy was represented as the correlation between perceived risk (indicators = verbal and quantitative measures) and calculated risk (indicators = Claus and BRCAPRO). Participants who reported more information was shared with them by their sister or mother about her genetic counseling session had greater accuracy of risk perception (0.707, p?=?0.000) than those who reported little information was shared (0.326, p?=?0.003). However, counselees shared very little information; nearly 20 % of family members reported their sister or mother shared nothing with them about her genetic counseling. Family members were generally not aware of the existence of a genetic counseling summary letter. Our findings underscore the need for effective strategies that facilitate counselees to share information about their genetic counseling sessions. Such communication may help their relatives better understand their cancer risks and enhance risk appropriate cancer prevention.     

Buy three but get only two: The smallest effect in a 2 × 2 ANOVA is always uninterpretable

Loftus (Memory & Cognition 6:312–319, 1978) distinguished between interpretable and uninterpretable interactions. Uninterpretable interactions are ambiguous, because they may be due to two additive main effects (no interaction) and a nonlinear relationship between the (latent) outcome variable and its indicator. Interpretable interactions can only be due to the presence of a true interactive effect in the outcome variable, regardless of the relationship that it establishes with its indicator. In the present article, we first show that same problem can arise when an unmeasured mediator has a nonlinear effect on the measured outcome variable. Then we integrate Loftus’s arguments with a seemingly contradictory approach to interactions suggested by Rosnow and Rosenthal (Psychological Bulletin 105:143–146, 1989). We show that entire data patterns, not just interaction effects alone, produce interpretable or noninterpretable interactions. Next, we show that the same problem of interpretability can apply to main effects. Lastly, we give concrete advice on what researchers can do to generate data patterns that provide unambiguous evidence for hypothesized interactions.     

Testing Manifest Monotonicity Using Order-Constrained Statistical Inference

Most dichotomous item response models share the assumption of latent monotonicity, which states that the probability of a positive response to an item is a nondecreasing function of a latent variable intended to be measured. Latent monotonicity cannot be evaluated directly, but it implies manifest monotonicity across a variety of observed scores, such as the restscore, a single item score, and in some cases the total score. In this study, we show that manifest monotonicity can be tested by means of the order-constrained statistical inference framework. We propose a procedure that uses this framework to determine whether manifest monotonicity should be rejected for specific items. This approach provides a likelihood ratio test for which the p-value can be approximated through simulation. A simulation study is presented that evaluates the Type I error rate and power of the test, and the procedure is applied to empirical data.