Bayesian Estimation for Item Factor Analysis Models with Sparse Categorical Indicators

Psychometric models for item-level data are broadly useful in psychology. A recurring issue for estimating item factor analysis (IFA) models is low-item endorsement (item sparseness), due to limited sample sizes or extreme items such as rare symptoms or behaviors. In this paper, I demonstrate that under conditions characterized by sparseness, currently available estimation methods, including maximum likelihood (ML), are likely to fail to converge or lead to extreme estimates and low empirical power. Bayesian estimation incorporating prior information is a promising alternative to ML estimation for IFA models with item sparseness. In this article, I use a simulation study to demonstrate that Bayesian estimation incorporating general prior information improves parameter estimate stability, overall variability in estimates, and power for IFA models with sparse, categorical indicators. Importantly, the priors proposed here can be generally applied to many research contexts in psychology, and they do not impact results compared to ML when indicators are not sparse. I then apply this method to examine the relationship between suicide ideation and insomnia in a sample of first-year college students. This provides an important alternative for researchers who may need to model items with sparse endorsement.     

Confirmatory Factor Analysis of the Childhood Anxiety Sensitivity Index: A Gender Comparison

The Childhood Anxiety Sensitivity Index (CASI) is an 18-item self-report tool designed to measure the construct of anxiety sensitivity (i.e. the belief that anxiety may have harmful consequences such as sickness, embarrassment, or loss of control) in children and adolescents. Previous factor analytic examinations of the CASI have produced varied results. Gender may play a role in this observed variability. In an effort to confirm the factor structure of the measure across gender, CASI items for 671 children and adolescents were subjected to confirmatory factor analysis. Results indicated that for boys two-, three-, and four-factor structures provided a relatively good fit to the data, with the three-factor structure emerging as having the best fit overall. In contrast, for girls only the three-factor structure fitted the data well. Direct comparison of fit of the three-factor model across gender provided evidence to support the notion that childhood anxiety sensitivity is similar in structure across gender.     

A generalized,likelihood-free method for posterior estimation

Recent advancements in Bayesian modeling have allowed for likelihood-free posterior estimation. Such estimation techniques are crucial to the understanding of simulation-based models, whose likelihood functions may be difficult or even impossible to derive. However, current approaches are limited by their dependence on sufficient statistics and/or tolerance thresholds. In this article, we provide a new approach that requires no summary statistics, error terms, or thresholds and is generalizable to all models in psychology that can be simulated. We use our algorithm to fit a variety of cognitive models with known likelihood functions to ensure the accuracy of our approach. We then apply our method to two real-world examples to illustrate the types of complex problems our method solves. In the first example, we fit an error-correcting criterion model of signal detection, whose criterion dynamically adjusts after every trial. We then fit two models of choice response time to experimental data: the linear ballistic accumulator model, which has a known likelihood, and the leaky competing accumulator model, whose likelihood is intractable. The estimated posterior distributions of the two models allow for direct parameter interpretation and model comparison by means of conventional Bayesian statistics—a feat that was not previously possible.     

Aggression and Fire: Antisocial Behavior in Firesetting and Nonfiresetting Juvenile Offenders

This study examined the association between firesetting and antisocial behavior in 219 juvenile offenders. The study showed, through a series of a priori model comparisons using confirmatory factor analysis, that reliable data on both firesetting (n = 85) and nonfiresetting (n = 134) juvenile offenders best fit a 3-factor model composed of aggressive, nonaggressive, and oppositional antisocial behavior. Although the same general structure of antisocial behavior best fit the data for both groups, the firesetting group exhibited a significantly higher variety and frequency of aggressive and total antisocial acts and an earlier age of index arrest. It is argued that results from this study support a conceptualization of firesetting as accompanying serious and versatile antisocial behavior. Consistency with an early starter pattern of antisocial behavior among firesetting offenders is noted, and evidence suggesting that firesetting is indicative of developmentally advanced, serious, and varied antisocial behavior among troubled youth is discussed.     

The Santa Clara Strength of Religious Faith Questionnaire: Confirmatory Factor Analysis

Within contemporary psychology there is increasing interest in the role of religion on behaviour and psychological functioning. Such interest can be attested to by the growth in the number of pertinent books and articles that have been published and also the development of new self-report measures of religiosity. The Santa Clara Strength of Religious Faith Questionnaire is one such recently developed measure. It is a brief self-report measure comprising 10 items that were designed to measure strength of religious faith regardless of religious denomination or affiliation. Although Plante and Boccaccini (1997a, 1997b) have provided some preliminary evidence for the reliability and validity of the scale, the factor structure of the scale has not yet been examined. The aim of the present paper was to confirm the factor structure of the scale. The hypothesised one-factor model was tested using confirmatory factor analytic methods. Data from a sample of 106 Northern Irish undergraduate university students were examined using confirmatory factor analytic methods. A one-factor model was tested and accepted on the basis of fit statistics, therefore supporting the hypothesised unidimensional structure of the scale. The present results provide further evidence that the Santa Clara Strength of Religious Faith Questionnaire is psychometrically sound and therefore it can be recommended for further use by researchers interested in the construct of strength of religious faith.     

The Counseling Center Assessment of Psychological Symptoms Distress Index: A pragmatic exploration of general factors to enhance a multidimensional scale

Objective: The authors attempted to develop and validate a general distress index for a multidimensional psychological symptom/outcome measure used in over 300 college counseling centers with more than 100,000 cases annually: the Counseling Center Assessment of Psychological Symptoms (CCAPS). Method: Four models were compared for fit indices (n = 19,247): the existing first-order factor model (without a general factor), a second-order factor model, a bifactor model, and a single factor or “total score” model. In separate clinical and non-clinical samples, concurrent and divergent validity were examined using several well-established measures of psychological symptoms, as well as two-week test–retest and treatment utilization data. Results: Second-order and bifactor models which captured a single “distress” factor both exhibited good fit to the data relative to the baseline and “total score” model. Validity data indicated that factors adequately measured meaningful clinical onstructs. Conclusion: Both the bifactor and second-order models indicated the presence of a “distress index” comprised items across many of the CCAPS subscales. This distress scale has strong applicability for benchmarking the overall severity and complexity of patients at different centers, and can be used to help identify colleges and universities with areas of clinical strength, which can be studied to improve the field. Clinically, the distress index offers a parsimonious and efficient method for clinicians to monitor patients’ progress through treatment.     

Efficient estimation in image factor analysis

The image factor analytic model (IFA), as related to Guttman's image theory, is considered as an alternative to the traditional factor analytic model (TFA). One advantage with IFA, as compared with TFA, is that more factors can be extracted without yielding a perfect fit to the observed data. Several theorems concerning the structural properties of IFA are proved and an iterative procedure for finding the maximum likelihood estimates of the parameters of the IFA-model is given. Substantial experience with this method verifies that Heywood cases never occur. Results of an artificial experiment suggest that IFA may be more factorially invariant than TFA under selection of tests from a large battery. The first part of this work was done at the University of Uppsala, Sweden and supported by the Swedish Council for Social Science Research. The second part was done at Educational Testing Service and supported by a grant (NSF-GB-1985) from the National Science Foundation to Educational Testing Service. The author is indebted to Mr. G. Gruvaeus, who wrote many of the computer programs, checked the mathematical derivations and gave other invaluable assistance throughout the work.     

Multilevel Factor Analysis: Reporting Guidelines and a Review of Reporting Practices

We provide reporting guidelines for multilevel factor analysis (MFA) and use these guidelines to systematically review 72 MFA applications in journals across a range of disciplines (e.g., education, health/nursing, management, and psychology) published between 1994 and 2014. Results are organized in terms of the (a) characteristics of the MFA application (e.g., construct measured), (b) purpose (e.g., measurement validation), (c) data source (e.g., number of cases at Level 1 and Level 2), (d) statistical approach (e.g., maximum likelihood), and (e) results reported (e.g., intraclass correlations for indicators and latent variables, standardized factor loadings, fit indices). Results from this review have implications for applied researchers interested in expanding their approaches to psychometric analyses and construct validation within a multilevel framework and for methodologists using Monte Carlo methods to explore technical and methodological issues grounded in realistic research design conditions.     

测验相对拟合检验方法CVLL法在认知诊断中的拓展及应用

本文将IRT中表现较好的CVLL法引入到认知诊断领域,同时比较并分析CVLL及认知诊断领域已有的测验相对拟合检验统计量的表现,为实际工作者在认知诊断模型选用上提供方法学支持和借鉴。结果表明:CVLL的表现比其它传统测验相对拟合统计量要好;且当对Q矩阵进行误设时,该统计量也能选择较优的Q矩阵,说明CVLL在Q矩阵侦查上有较好的应用前景。     

Modeling the Structure of the Attitudes and Belief Scale 2 using CFA and Bifactor Approaches: Toward the Development of an Abbreviated Version

The Attitudes and Belief Scale-2 (ABS-2: DiGiuseppe, Leaf, Exner, & Robin, 1988. The development of a measure of rational/irrational thinking. Paper presented at the World Congress of Behavior Therapy, Edinburg, Scotland.) is a 72-item self-report measure of evaluative rational and irrational beliefs widely used in Rational Emotive Behavior Therapy research contexts. However, little psychometric evidence exists regarding the measure's underlying factor structure. Furthermore, given the length of the ABS-2 there is a need for an abbreviated version that can be administered when there are time demands on the researcher, such as in clinical settings. This study sought to examine a series of theoretical models hypothesized to represent the latent structure of the ABS-2 within an alternative models framework using traditional confirmatory factor analysis as well as utilizing a bifactor modeling approach. Furthermore, this study also sought to develop a psychometrically sound abbreviated version of the ABS-2. Three hundred and thirteen (N = 313) active emergency service personnel completed the ABS-2. Results indicated that for each model, the application of bifactor modeling procedures improved model fit statistics, and a novel eight-factor intercorrelated solution was identified as the best fitting model of the ABS-2. However, the observed fit indices failed to satisfy commonly accepted standards. A 24-item abbreviated version was thus constructed and an intercorrelated eight-factor solution yielded satisfactory model fit statistics. Current results support the use of a bifactor modeling approach to determining the factor structure of the ABS-2. Furthermore, results provide empirical support for the psychometric properties of the newly developed abbreviated version.     

Masking misfit in confirmatory factor analysis by increasing unique variances: a cautionary note on the usefulness of cutoff values of fit indices

Fit indices are widely used in order to test the model fit for structural equation models. In a highly influential study, Hu and Bentler (1999) showed that certain cutoff values for these indices could be derived, which, over time, has led to the reification of these suggested thresholds as "golden rules" for establishing the fit or other aspects of structural equation models. The current study shows how differences in unique variances influence the value of the global chi-square model test and the most commonly used fit indices: Root-mean-square error of approximation, standardized root-mean-square residual, and the comparative fit index. Using data simulation, the authors illustrate how the value of the chi-square test, the root-mean-square error of approximation, and the standardized root-mean-square residual are decreased when unique variances are increased although model misspecification is present. For a broader understanding of the phenomenon, the authors used different sample sizes, number of observed variables per factor, and types of misspecification. A theoretical explanation is provided, and implications for the application of structural equation modeling are discussed.     

The big none: No evidence for a general factor of personality in chimpanzees, orangutans, or rhesus macaques

We examined whether a general factor of personality (GFP) was present in chimpanzees, orangutans, or rhesus macaques. We used confirmatory factor analysis (CFA) to model correlations among first-order factors as arising from a GFP. We then conducted principal axis factor analyses (PFA) of the first-order factors to extract a single higher-order factor and then to extract two oblique higher-order factors. The CFA model fit was poor for chimpanzees and orangutans, but not rhesus macaques. The single higher-order factors extracted via PFA did not resemble the GFP in all three species. The oblique higher-order factors extracted via PFA were only weakly correlated in all three species. These results do not support the existence of a GFP in nonhuman primates.     

The Factor Structure of the Frost Multidimensional Perfectionism Scale in University Students

The Frost Multidimensional Perfectionism Scale (FMPS) is a widely used measure of perfectionism. Previous studies of the nature of the factors of the FMPS have resulted in mixed findings and have failed to identify a stable factor structure. This study investigated the factor structure of the FMPS using both exploratory and confirmatory factor analysis (CFA), addressing previous methodological limitations. Participants were 6,449 students from two Australian universities. Exploratory factor analysis suggested a five‐factor model and a reduction in variables from 35 to 29 (FMPS‐29). CFA demonstrated goodness of fit of the revised measure. We also compared our model with a number of existing measurement models of perfection and found the proposed model provided a superior fit for this university student population. Support was found for the two second‐order constructs commonly referred to as adaptive and maladaptive perfectionism, although there appears to be little benefit in using this model over the first‐order model. Implications of the findings are discussed.     

Genetic and environmental covariance structures among WISC-R subtests: A twin study

Genetic and environmental correlations among 11 Wechsler Intelligence Scale for Children-Revised (WISC-R) subtests were estimated from a sample of 143 twin pairs using the methodology of multivariate behavioral genetics. The genetic correlations among subtests varied in magnitude and exhibited a pattern similar to corresponding phenotypic correlations. Correlations due to within-pair environmental influences were generally small, whereas those due to shared environmental influences tended to be intermediate. Several hypotheses regarding the structures of the additive-genetic, common environmental, and within-pair environmental covariance matrices were tested and evidence for differential factor structures was obtained. A model hypothesizing a three-factor structure for the genetic covariance matrix, a single factor for the common environmental matrix, and a single factor plus specifics for the within-pair environmental matrix fit the data well. Thus, the three-factor phenotypic structure typically observed in Wechsler Intelligence Scale for Children (WISC) data may be due largely to genetic influences.     

Gender role conflict scale: validation for a sample of gay men and lesbian women

This study tested the oblique four-factor model of the Gender Role Conflict Scale for a sample of gay men and lesbian women residing in the United States. 400 gay men and 292 lesbian women recruited from university and college gay, lesbian, bisexual, and transgender LISTSERVs participated. The internal consistency reliability of the Gender Role Conflict Scale scores was high, but low means on the expressive dimension of gender role conflict were noted. The results of two separate sets of confirmatory factor analyses indicated that the four-factor model's fit to the data could be enhanced for both groups by applying item parceling to lower the number of indicators per factor, suggesting that the actual structural validity of the Gender Role Conflict Scale may be better than suggested by the reported fit indices.     

题目因素分析：基于SEM和基于IRT的两类方法

心理和教育测量一般只能达到顺序量表的水平,其测量数据与被测因子间并非简单线性关系。题目因素分析是用来描述测量题目与因子间非线性关系的统计模型。题目因素分析主要有基于结构方程模型和基于项目反应理论两类方法,两类方法之间存在紧密的联系,甚至可以看作是同一模型的两种表现形式。本文详细阐述了该关系,同时对两类方法在参数估计、模型拟合指标、测量一致性检验和支撑软件等方面的特点进行了分析和比较,以便研究者选择最为适合其研究的方法。     

The multidimensionality of test anxiety: A test of different models

Abstract

Confirmatory factor analysis was employed to test the multidimensional structure of test anxiety. Starting with the traditional two-factor model, it was of interest whether additional dimensions could still be represented by a single higher-order factor of test anxiety or whether the additional dimensions represented correlates of test anxiety. The Revised Test Anxiety (RTA) scale and the German Test Anxiety Inventory (TAI-G) were combined collecting data from a binational sample of 218 American and 218 German university students. Considering the statistical and theoretical aspects of model fit, they indicated that a model consisting of three primary factors (worry, emotionality, and lack of confidence) fit the data best. While distraction and self-efficacy may be regarded as correlates of test anxiety, lack of confidence was substantiated as a component of test anxiety. Implications for the conceptualization of the content domain and hence area of test anxiety were discussed.     

探索性因素分析——最近10年的评述

目的：(1)介绍国外心理统计学界对探索性因素分析中几个重大问题的基本观点;(2)系统地评述过去10年里(1991～2000年)我国心理学研究对这一技术的使用情况;(3)强调运用这一技术时值得注意的一些事项,以期这一技术对我国心理学研究发挥更大的作用。方法：作检索了《心理学报》和《心理科学》在1991～2000年间与探索性因素分析有关的章,对其中以探索性因素分析为主要研究方法的重点章(feature article)进行了编码,统计了频数及百分比分布。结果：我国心理学研究在积极使用探索性因素分析这一先进统计技术时还存在一些问题,主要表现在：(1)在确定因素个数时,倾向于机械地依靠桌个单一方法来作决定,(2)大量使用正交旋转,(3)过于依赖SPSS,(4)对因素分析过程中的重要信息／结果报告不够。结论：探索性因素分析在过去十年得到了广泛的应用,如能吸收国外同行的一些观点,探索性因素分析这一技术在我国心理学研究中必将获得更广泛有效的应用。     

Selecting the number of classes under latent class regression: a factor analytic analogue

Recently, the regression extension of latent class analysis (RLCA) model has received much attention in the field of medical research. The basic RLCA model summarizes shared features of measured multiple indicators as an underlying categorical variable and incorporates the covariate information in modeling both latent class membership and multiple indicators themselves. To reduce complexity and enhance interpretability, one usually fixes the number of classes in a given RLCA. Often, goodness of fit methods comparing various estimated models are used as a criterion to select the number of classes. In this paper, we propose a new method that is based on an analogous method used in factor analysis and does not require repeated fitting. Two ideas with application to many settings other than ours are synthesized in deriving the method: a connection between latent class models and factor analysis, and techniques of covariate marginalization and elimination. A Monte Carlo simulation study is presented to evaluate the behavior of the selection procedure and compare to alternative approaches. Data from a study of how measured visual impairments affect older persons’ functioning are used for illustration.This work was supported by National Institute on Aging (NIA) Program Project P01-AG-10184-03. The author wishes to thank Dr. Karen Bandeen-Roche for her stimulating comments and helpful discussions, and Drs. Gary Rubin and Sheila West for kindly making the Salisbury Eye Evaluation data available.     

Evaluating Structural Equation Models for Categorical Outcomes: A New Test Statistic and a Practical Challenge of Interpretation

This research is concerned with two topics in assessing model fit for categorical data analysis. The first topic involves the application of a limited-information overall test, introduced in the item response theory literature, to structural equation modeling (SEM) of categorical outcome variables. Most popular SEM test statistics assess how well the model reproduces estimated polychoric correlations. In contrast, limited-information test statistics assess how well the underlying categorical data are reproduced. Here, the recently introduced C₂ statistic of Cai and Monroe (2014) is applied. The second topic concerns how the root mean square error of approximation (RMSEA) fit index can be affected by the number of categories in the outcome variable. This relationship creates challenges for interpreting RMSEA. While the two topics initially appear unrelated, they may conveniently be studied in tandem since RMSEA is based on an overall test statistic, such as C₂. The results are illustrated with an empirical application to data from a large-scale educational survey.