An R package for state-trace analysis

State-trace analysis (Bamber, Journal of Mathematical Psychology, 19, 137-181, 1979) is a graphical analysis that can determine whether one or more than one latent variable mediates an apparent dissociation between the effects of two experimental manipulations. State-trace analysis makes only ordinal assumptions and so, is not confounded by range effects that plague alternative methods, especially when performance is measured on a bounded scale (such as accuracy). We describe and illustrate the application of a freely available GUI driven package, StateTrace, for the R language. StateTrace automates many aspects of a state-trace analysis of accuracy and other binary response data, including customizable graphics and the efficient management of computationally intensive Bayesian methods for quantifying evidence about the outcomes of a state-trace experiment, developed by Prince, Brown, and Heathcote (Psychological Methods, 17, 78-99, 2012).     

State-trace analysis can be an appropriate tool for assessing the number of cognitive systems: A reply to Ashby (2014)

Ashby (2014) has argued that state-trace analysis (STA) is not an appropriate tool for assessing the number of cognitive systems, because it fails in its primary goal of distinguishing single-parameter and multiple-parameter models. We show that this is based on a misunderstanding of the logic of STA, which depends solely on nearly universal assumptions about psychological measurement and clearly supersedes inferences based on functional dissociation and the analysis of interactions in analyses of variance. We demonstrate that STA can be used to draw inferences concerning the number of latent variables mediating the effects of a set of independent variables on a set of dependent variables. We suggest that STA is an appropriate tool to use when making arguments about the number of cognitive systems that must be posited to explain behavior. However, no statistical or inferential procedure is able to provide definitive answers to questions about the number of cognitive systems, simply because the concept of a “system” is not defined in an appropriate way.     

反馈延迟对信息整合类别学习的影响：视觉项目精准性的调节作用

研究旨在探究更精确的视觉项目表征能否消除延迟反馈对信息整合学习的损害。实验采用2（视觉项目表征精确性：精确/非精确）×2（反馈延迟：即时反馈/延迟反馈）组间设计,结果发现无论有无延迟反馈,精确的项目表征条件下类别学习成绩都显著高于非精确条件,精确性与反馈延迟存在交互作用。采用状态痕迹分析进一步证明精确性显著影响信息整合类别学习的稳定性。在反馈延迟条件下,精确的视觉项目表征能提高信息整合类别学习的成绩。     

Is state-trace analysis an appropriate tool for assessing the number of cognitive systems?

There is now much evidence that humans have multiple memory systems, and evidence is also building that other cognitive processes are mediated by multiple systems. Even so, several recent articles have questioned the existence of multiple cognitive systems, and a number of these have based their arguments on results from state-trace analysis. State-trace analysis was not developed for this purpose but, rather, to identify data sets that are consistent with variation in a single parameter. All previous applications have assumed that state-trace plots in which the data fall on separate curves rule out any model in which only a single parameter varies across the two tasks under study. Unfortunately, this assumption is incorrect. Models in which only one parameter varies can generate any type of state-trace plot, as can models in which two or more parameters vary. In addition, it is straightforward to show that both single-system and multiple-systems models can generate state-trace plots that are considered in the literature to be consistent with either one or multiple cognitive systems. Thus, without additional information, there is no empirical state-trace plot that supports any inferences about the number of underlying parameters or systems.     

The effect of feedback delay and feedback type on perceptual category learning: the limits of multiple systems

Evidence that learning rule-based (RB) and information-integration (II) category structures can be dissociated across different experimental variables has been used to support the view that such learning is supported by multiple learning systems. Across 4 experiments, we examined the effects of 2 variables, the delay between response and feedback and the informativeness of feedback, which had previously been shown to dissociate learning of the 2 types of category structure. Our aim was twofold: first, to determine whether these dissociations meet the more stringent inferential criteria of state-trace analysis and, second, to determine the conditions under which they can be observed. Experiment 1 confirmed that a mask-filled feedback delay dissociated the learning of RB and II category structures with minimally informative (yes/no) feedback and also met the state-trace criteria for the involvement of multiple latent variables. Experiment 2 showed that this effect is eliminated when a less similar, fixed pattern mask is presented in the interval between response and feedback. Experiment 3 showed that the selective effect of feedback delay on II learning is reduced with fully informative feedback (in which the correct category is specified after an incorrect response) and that feedback type did not dissociate RB and II learning. Experiment 4 extended the results of Experiment 2, showing that the differential effect of feedback delay is eliminated when a fixed pattern mask is used. These results pose important challenges to models of category learning, and we discuss their implications for multiple learning system models and their alternatives.     

Taxometric Investigation of PTSD: data from two nationally representative samples

Current psychiatric nosology depicts posttraumatic stress disorder (PTSD) as a discrete diagnostic category. However, only one study has examined the latent structure of PTSD, and this study suggested that PTSD may be more accurately conceptualized as an extreme reaction to traumatic life events rather than a discrete clinical syndrome. To build on the existing literature base, the present research examined the latent structure of posttraumatic stress reactions by applying three taxometric procedures (MAXEIG, MAMBAC, and L-Mode) to data collected from large nationally representative samples of women (ns = 2684 and 3033) and adolescents (n = 3775). Results consistently provided evidence for a dimensional PTSD solution across samples and statistical procedures. These findings have important implications for the theory, assessment, and investigation of posttraumatic stress reactions.     

潜变量建模的贝叶斯方法

贝叶斯统计是统计学的两大流派之一,近年来贝叶斯统计在社会及行为科学领域日益流行。鉴于国内心理学界对贝叶斯统计应用仍不广泛,本文尝试从非技术性的角度对贝叶斯统计用于潜变量建模的过程进行简要介绍。主要涉及贝叶斯与频率论在统计学基本概念上的对比;贝叶斯统计的基本原理和分析过程。最后以一个验证性因子分析为例,简要介绍贝叶斯统计用于潜变量建模的分析过程。希望本文能为国内心理学者进行潜变量建模提供新的视角。     

The dimensionality of perceptual category learning: A state-trace analysis

State-trace analysis was used to investigate the effect of concurrent working memory load on perceptual category learning. Initial reanalysis of Zeithamova and Maddox (2006, Experiment 1) revealed an apparently two-dimensional state-trace plot consistent with a dual-system interpretation of category learning. However, three modified replications of the original experiment found evidence of a single resource underlying the learning of both rule-based and information integration category structures. Follow-up analyses of the Zeithamova and Maddox data, restricted to only those participants who had learned the category task and performed the concurrent working memory task adequately, revealed a one-dimensional plot consistent with a single-resource interpretation and the results of the three new experiments. The results highlight the potential of state-trace analysis in furthering our understanding of the mechanisms underlying category learning.     

The construct of psychophysiological reactivity: Statistical and psychometric issues

The purpose of this paper is to review major statistical and psychometric issues impacting the study of psychophysiological reactivity and discuss their implications for applied developmental researchers. We first cover traditional approaches such as the observed difference score (DS) and the observed residual score (RS), including a review of classic and recent research on their reliability and validity from two related bodies of work: the measurement of change and the Law of Initial Values. Second, we review several types of latent variable modeling in this context: latent difference score (LDS) models, latent residual score (LRS) models, latent state-trait (LST) models, and latent growth curve (LGC) models. Finally, we provide broad guidelines for applied researchers broken down by key stages of a psychophysiological project: study planning, data analysis, and reporting of results. Our recommendations highlight the need for (1) increased attention to the ubiquitous nature of measurement error in observed variables and the importance of employing latent variable models when possible, and (2) increased specification of theories relating to the construct of reactivity, especially in regards to the distinction between baseline arousal and change over time in broader systems of variables.     

State-trace analysis of the effects of a visual illusion on saccade amplitudes and perceptual judgments

Visual illusions often appear to have a larger influence on subjective judgments than on visuomotor behavior. Although this effect has been taken as evidence for multiple estimates of stimulus size in the visual brain, dissociations between subjective judgments and visuomotor measures can frequently be reconciled with a singleestimate model. To circumvent this difficulty, we used state-trace analysis in a pair of experiments to examine the effects of the Müller-Lyer illusion on subjective length estimates, voluntary saccade amplitudes, and reflexive saccade amplitudes. All dependent measures were affected by the illusion. However, state-trace analyses revealed nonmonotonic relationships among all three variables, a pattern inconsistent with the possibility of a single underlying estimate of stimulus size.     

Mechanisms and the Evidence Hierarchy

Evidence-based medicine (EBM) makes use of explicit procedures for grading evidence for causal claims. Normally, these procedures categorise evidence of correlation produced by statistical trials as better evidence for a causal claim than evidence of mechanisms produced by other methods. We argue, in contrast, that evidence of mechanisms needs to be viewed as complementary to, rather than inferior to, evidence of correlation. In this paper we first set out the case for treating evidence of mechanisms alongside evidence of correlation in explicit protocols for evaluating evidence. Next we provide case studies which exemplify the ways in which evidence of mechanisms complements evidence of correlation in practice. Finally, we put forward some general considerations as to how the two sorts of evidence can be more closely integrated by EBM.     

Information-theoretic latent distribution modeling: distinguishing discrete and continuous latent variable models

Distinguishing between discrete and continuous latent variable distributions has become increasingly important in numerous domains of behavioral science. Here, the authors explore an information-theoretic approach to latent distribution modeling, in which the ability of latent distribution models to represent statistical information in observed data is emphasized. The authors conclude that loss of statistical information with a decrease in the number of latent values provides an attractive basis for comparing discrete and continuous latent variable models. Theoretical considerations as well as the results of 2 Monte Carlo simulations indicate that information theory provides a sound basis for modeling latent distributions and distinguishing between discrete and continuous latent variable models in particular.     

Working memory span tasks: A methodological review and user’s guide

Working memory (WM) span tasks—and in particular, counting span, operation span, and reading span tasks—are widely used measures of WM capacity. Despite their popularity, however, there has never been a comprehensive analysis of the merits of WM span tasks as measurement tools. Here, we review the genesis of these tasks and discuss how and why they came to be so influential. In so doing, we address the reliability and validity of the tasks, and we consider more technical aspects of the tasks, such as optimal administration and scoring procedures. Finally, we discuss statistical and methodological techniques that have commonly been used in conjunction with WM span tasks, such as latent variable analysis and extreme-groups designs.     

Elucidating the Functional Relationship Between Working Memory Capacity and Psychometric Intelligence: A Fixed-Links Modeling Approach for Experimental Repeated-Measures Designs

Numerous studies reported a strong link between working memory capacity (WMC) and fluid intelligence (Gf), although views differ in respect to how close these two constructs are related to each other. In the present study, we used a WMC task with five levels of task demands to assess the relationship between WMC and Gf by means of a new methodological approach referred to as fixed-links modeling. Fixed-links models belong to the family of confirmatory factor analysis (CFA) and are of particular interest for experimental, repeated-measures designs. With this technique, processes systematically varying across task conditions can be disentangled from processes unaffected by the experimental manipulation. Proceeding from the assumption that experimental manipulation in a WMC task leads to increasing demands on WMC, the processes systematically varying across task conditions can be assumed to be WMC-specific. Processes not varying across task conditions, on the other hand, are probably independent of WMC. Fixed-links models allow for representing these two kinds of processes by two independent latent variables. In contrast to traditional CFA where a common latent variable is derived from the different task conditions, fixed-links models facilitate a more precise or purified representation of the WMC-related processes of interest. By using fixed-links modeling to analyze data of 200 participants, we identified a non-experimental latent variable, representing processes that remained constant irrespective of the WMC task conditions, and an experimental latent variable which reflected processes that varied as a function of experimental manipulation. This latter variable represents the increasing demands on WMC and, hence, was considered a purified measure of WMC controlled for the constant processes. Fixed-links modeling showed that both the purified measure of WMC (β = .48) as well as the constant processes involved in the task (β = .45) were related to Gf. Taken together, these two latent variables explained the same portion of variance of Gf as a single latent variable obtained by traditional CFA (β = .65) indicating that traditional CFA causes an overestimation of the effective relationship between WMC and Gf. Thus, fixed-links modeling provides a feasible method for a more valid investigation of the functional relationship between specific constructs.     

Four Research Designs and a Comprehensive Analysis Strategy for Investigating Common Method Variance with Self-Report Measures Using Latent Variables

Common method variance (CMV) is an ongoing topic of debate and concern in the organizational literature. We present four latent variable confirmatory factor analysis model designs for assessing and controlling for CMV—those for unmeasured latent method constructs, Marker Variables, Measured Cause Variables, as well as a new hybrid design wherein these three types of method latent variables are used concurrently. We then describe a comprehensive analysis strategy that can be used with these four designs and provide a demonstration using the new design, the Hybrid Method Variables Model. In our discussion, we comment on different issues related to implementing these designs and analyses, provide supporting practical guidance, and, finally, advocate for the use of the Hybrid Method Variables Model. Through these means, we hope to promote a more comprehensive and consistent approach to the assessment of CMV in the organizational literature and more extensive use of hybrid models that include multiple types of latent method variables to assess CMV.     

Statistical mediation analysis with a multicategorical independent variable

Virtually all discussions and applications of statistical mediation analysis have been based on the condition that the independent variable is dichotomous or continuous, even though investigators frequently are interested in testing mediation hypotheses involving a multicategorical independent variable (such as two or more experimental conditions relative to a control group). We provide a tutorial illustrating an approach to estimation of and inference about direct, indirect, and total effects in statistical mediation analysis with a multicategorical independent variable. The approach is mathematically equivalent to analysis of (co)variance and reproduces the observed and adjusted group means while also generating effects having simple interpretations. Supplementary material available online includes extensions to this approach and Mplus, SPSS, and SAS code that implements it.     

Testing Differential Item Functioning in Small Samples

Abstract

Differential item functioning (DIF) is a pernicious statistical issue that can mask true group differences on a target latent construct. A considerable amount of research has focused on evaluating methods for testing DIF, such as using likelihood ratio tests in item response theory (IRT). Most of this research has focused on the asymptotic properties of DIF testing, in part because many latent variable methods require large samples to obtain stable parameter estimates. Much less research has evaluated these methods in small sample sizes despite the fact that many social and behavioral scientists frequently encounter small samples in practice. In this article, we examine the extent to which model complexity—the number of model parameters estimated simultaneously—affects the recovery of DIF in small samples. We compare three models that vary in complexity: logistic regression with sum scores, the 1-parameter logistic IRT model, and the 2-parameter logistic IRT model. We expected that logistic regression with sum scores and the 1-parameter logistic IRT model would more accurately estimate DIF because these models yielded more stable estimates despite being misspecified. Indeed, a simulation study and empirical example of adolescent substance use show that, even when data are generated from / assumed to be a 2-parameter logistic IRT, using parsimonious models in small samples leads to more powerful tests of DIF while adequately controlling for Type I error. We also provide evidence for minimum sample sizes needed to detect DIF, and we evaluate whether applying corrections for multiple testing is advisable. Finally, we provide recommendations for applied researchers who conduct DIF analyses in small samples.     

Combining Item Response Theory and Diagnostic Classification Models: A Psychometric Model for Scaling Ability and Diagnosing Misconceptions

Traditional testing procedures typically utilize unidimensional item response theory (IRT) models to provide a single, continuous estimate of a student’s overall ability. Advances in psychometrics have focused on measuring multiple dimensions of ability to provide more detailed feedback for students, teachers, and other stakeholders. Diagnostic classification models (DCMs) provide multidimensional feedback by using categorical latent variables that represent distinct skills underlying a test that students may or may not have mastered. The Scaling Individuals and Classifying Misconceptions (SICM) model is presented as a combination of a unidimensional IRT model and a DCM where the categorical latent variables represent misconceptions instead of skills. In addition to an estimate of ability along a latent continuum, the SICM model provides multidimensional, diagnostic feedback in the form of statistical estimates of probabilities that students have certain misconceptions. Through an empirical data analysis, we show how this additional feedback can be used by stakeholders to tailor instruction for students’ needs. We also provide results from a simulation study that demonstrate that the SICM MCMC estimation algorithm yields reasonably accurate estimates under large-scale testing conditions.     

Detecting latent taxa: Monte Carlo comparison of taxometric, mixture model, and clustering procedures

A Monte Carlo evaluation of four procedures for detecting taxonicity was conducted using artificial data sets that were either taxonic or nontaxonic. The data sets were analyzed using two of Meehl's taxometric procedures, MAXCOV and MAMBAC, Ward's method for cluster analysis in concert with the cubic clustering criterion and a latent variable mixture modeling technique. Performance of the taxometric procedures and latent variable mixture modeling were clearly superior to that of cluster analysis in detecting taxonicity. Applied researchers are urged to select from the better procedures and to perform consistency tests.     

State-trace analysis: A method of testing simple theories of causation

An independent variable measures some aspect of a treatment applied to a person; a dependent variable measures some aspect of the treatment's effect upon the person. Two dependent variables will often covary with each other because they are affected by a common independent variable. A state trace is a graph which plots the value of one dependent variable as a function of another. (Thus, a state trace is a generalization of the yes-no receiver-operating-characteristic curve.) By appropriately analyzing state traces, it is possible to test theories concerning (a) the mechanisms by which independent variables affect dependent variables and (b) how these mechanisms differ from one population of persons to another. As an illustration, a study of long-term memory in normal persons, schizophrenics, and persons with organic brain syndrome is presented. The effects of number of word presentations and length of retention interval upon probability of word recall and recognition were investigated. The results were analyzed by state-trace analysis.