Use and interpretation of test scores from limited cognitive test batteries: How g + Gc can equal g

Single scores from limited and unbalanced test batteries of cognitive ability can be ambiguous to interpret theoretically. In this study, a limited verbally and knowledge‐loaded cognitive test battery, from applicants to the Swedish police academies (N = 1,344), was examined to provide foundations for the use and interpretation of test scores. Three measurement models were compared: one single factor model and two bifactor models, which decomposed the variance of the battery into orthogonal components. The models were evaluated by fit indices and omega coefficients, and then applied to the prediction of academic performance. The overall prediction of all models was similar, although specific abilities also were found to provide substantial predictive validity over and above general intelligence (g). The findings provide support for the use of single scores in applied settings (selection), but suggest that it may be more appropriate to interpret such scores as composites of substantive components, and not just as measures of g.     

Bifactor models and rotations: exploring the extent to which multidimensional data yield univocal scale scores

The application of psychological measures often results in item response data that arguably are consistent with both unidimensional (a single common factor) and multidimensional latent structures (typically caused by parcels of items that tap similar content domains). As such, structural ambiguity leads to seemingly endless "confirmatory" factor analytic studies in which the research question is whether scale scores can be interpreted as reflecting variation on a single trait. An alternative to the more commonly observed unidimensional, correlated traits, or second-order representations of a measure's latent structure is a bifactor model. Bifactor structures, however, are not well understood in the personality assessment community and thus rarely are applied. To address this, herein we (a) describe issues that arise in conceptualizing and modeling multidimensionality, (b) describe exploratory (including Schmid-Leiman [Schmid & Leiman, 1957] and target bifactor rotations) and confirmatory bifactor modeling, (c) differentiate between bifactor and second-order models, and (d) suggest contexts where bifactor analysis is particularly valuable (e.g., for evaluating the plausibility of subscales, determining the extent to which scores reflect a single variable even when the data are multidimensional, and evaluating the feasibility of applying a unidimensional item response theory (IRT) measurement model). We emphasize that the determination of dimensionality is a related but distinct question from either determining the extent to which scores reflect a single individual difference variable or determining the effect of multidimensionality on IRT item parameter estimates. Indeed, we suggest that in many contexts, multidimensional data can yield interpretable scale scores and be appropriately fitted to unidimensional IRT models.     

Assessing the factor structure of the Behaviour of Young Novice Drivers Scale (BYNDS)

The Behaviour of Young Novice Drivers Scale (BYNDS) is a self-report instrument to assess risky driving behaviours among youth population. Although previous studies have offered evidences of reliability and validity for the BYNDS scores, results are not conclusive as regards its factor structure. The aim of this research was to assess the factor structure of the BYNDS via Confirmatory Factor Analyses (CFA). The BYNDS was administered to a sample of 832 drivers aged from 18 to 25 years (Argentina = 270, Colombia = 350, and Mexico = 212). Six models were examined: three derived from previous literature, one emerged from an exploratory factor analysis, a unidimensional model, and a bifactor model. Results showed that none of the models analysed offered a completely acceptable fit to the data. However, the original model hypothesizing five orthogonal factors showed the best fit indices, with all of the items loading onto the associated factor. Results of the bifactor analysis suggested that three of the subscales (transient violations, fixed violations, and misjudgements) share a strong common basis and their scores would not offer much information above and beyond the total score. On the contrary, the items in the subscale of risk-exposure are clearly influenced by a specific domain factor, contributing to the multidimensionality of the BYNDS. In line with previous studies, males scored higher than females in driving violations, fixed violations, and risk-exposure. This study contributes to the psychometric refinement of a novel measure of risky driving in youth population. This self-report could be a valuable tool in the evaluation of road safety performance.     

Factor Structure and Incremental Utility of the Multidimensional Cognitive Attentional Syndrome Scale (MCASS): A Bifactor Analysis

The Multidimensional Cognitive Attentional Syndrome Scale (MCASS) was developed to assess the seven maladaptive forms of self-regulation that make up the cognitive attentional syndrome (CAS). Both theory and empirical evidence highlight important distinctions among the seven forms of self-regulation underlying the CAS. The primary purpose of the present study was to determine whether the MCASS item scores are sufficiently multidimensional to warrant the use of subscale scores. A secondary aim was to examine the incremental utility of the MCASS domain-specific factors. A battery of self-report measures was administered to adults recruited through a crowd-sourcing website (N?=?359). Bifactor analysis was used to examine the multidimensionality of MCASS item scores. This analytic approach allowed for the quantification of variance captured by each domain-specific item score independent of the general factor. Results from the bifactor analysis suggest that the MCASS is a multidimensional measure, consisting of a strong general factor and domain-specific factors that are sufficiently distinct. Additionally, the majority of domain-specific factors provided incremental utility in predicting two criterion variables (i.e., general distress, happiness emotion goals) after accounting for the general factor. Taken together, results support continued use of the MCASS total scale and subscale scores and suggest that researchers may want to consider using a bifactor model when examining structural models that include the MCASS.

     

Modeling the Balanced Inventory of Desirable Responding: Evidence in Favor of a Revised Model of Socially Desirable Responding

Paulhus (1984) proposed a 2-factor model of socially desirable responding (SDR) and created the Balanced Inventory of Desirable Responding (BIDR) to capture the 2 dimensions: self-deceptive enhancement and impression management. However, the 2-factor model has yet to be supported via confirmatory factor analysis (CFA) of the BIDR. Paulhus and Reid (1991) proposed a revised model of SDR, which bifurcated the self-deceptive enhancement dimension into 2 factors: self-deceptive enhancement (SDE; positively keyed items) and self-deceptive denial (SDD; negatively keyed items). Thus, it was considered useful to test the revised SDR model on the BIDR, as this might have implications for the manner in which the BIDR should be scored. Additionally, as the BIDR subscales might be positively correlated, it was considered useful from a both a theoretical and practical perspective to test the possibility that the BIDR might measure a general SDR process. Based on a sample of 466 adults, Paulhus and Reid's revised model of SDR was largely supported when tested via CFA on a bifactor model, which included a first-order general SDR factor and 2 nested factors. Thus, applied researchers might consider using total BIDR composite scores, a self-deceptive enhancement composite score (positively keyed items), and impression management composite scores. However, ideally, researchers would use a bifactor model to test substantive hypotheses, as the bifactor model partitions true score variance into unique sources, which facilitates less ambiguous interpretations of effects.     

The challenges of fitting an item response theory model to the Social Anhedonia Scale

This study explored the application of latent variable measurement models to the Social Anhedonia Scale (SAS; Eckblad, Chapman, Chapman, & Mishlove, 1982), a widely used and influential measure in schizophrenia-related research. Specifically, we applied unidimensional and bifactor item response theory (IRT) models to data from a community sample of young adults (n = 2,227). Ordinal factor analyses revealed that identifying a coherent latent structure in the 40-item SAS data was challenging due to (a) the presence of multiple small content clusters (e.g., doublets); (b) modest relations between those clusters, which, in turn, implies a general factor of only modest strength; (c) items that shared little variance with the majority of items; and (d) cross-loadings in bifactor solutions. Consequently, we conclude that SAS responses cannot be modeled accurately by either unidimensional or bifactor IRT models. Although the application of a bifactor model to a reduced 17-item set met with better success, significant psychometric and substantive problems remained. Results highlight the challenges of applying latent variable models to scales that were not originally designed to fit these models.     

Disentangling Gratitude: A Theoretical and Psychometric Examination of the Gratitude Resentment and Appreciation Test–Revised Short (GRAT–RS)

This study extended our theoretical and applied understanding of gratitude through a psychometric examination of the most popular multidimensional measure of gratitude, the Gratitude, Resentment, and Appreciation Test–Revised Short form (GRAT–RS). Namely, the dimensionality of the GRAT–RS, the model-based reliability of the GRAT–RS total score and 3 subscale scores, and the incremental evidence of validity for its latent factors were assessed. Dimensionality measures (e.g., explained common variance) and confirmatory factor analysis results with 426 community adults indicated that the GRAT–RS conformed to a multidimensional (bifactor) structure. Model-based reliability measures (e.g., omega hierarchical) provided support for the future use of the Lack of a Sense of Deprivation raw subscale score, but not for the raw GRAT–RS total score, Simple Appreciation subscale score, or Appreciation of Others subscale score. Structural equation modeling results indicated that only the general gratitude factor and the lack of a sense of deprivation specific factor accounted for significant variance in life satisfaction, positive affect, and distress. These findings support the 3 pillars of gratitude conceptualization of gratitude over competing conceptualizations, the position that the specific forms of gratitude are theoretically distinct, and the argument that appreciation is distinct from the superordinate construct of gratitude.     

Orthogonal higher order structure and confirmatory factor analysis of the French Wechsler Adult Intelligence Scale (WAIS-III)

According to the most widely accepted Cattell-Horn-Carroll (CHC) model of intelligence measurement, each subtest score of the Wechsler Intelligence Scale for Adults (3rd ed.; WAIS-III) should reflect both 1st- and 2nd-order factors (i.e., 4 or 5 broad abilities and 1 general factor). To disentangle the contribution of each factor, we applied a Schmid-Leiman orthogonalization transformation (SLT) to the standardization data published in the French technical manual for the WAIS-III. Results showed that the general factor accounted for 63% of the common variance and that the specific contributions of the 1st-order factors were weak (4.7%-15.9%). We also addressed this issue by using confirmatory factor analysis. Results indicated that the bifactor model (with 1st-order group and general factors) better fit the data than did the traditional higher order structure. Models based on the CHC framework were also tested. Results indicated that a higher order CHC model showed a better fit than did the classical 4-factor model; however, the WAIS bifactor structure was the most adequate. We recommend that users do not discount the Full Scale IQ when interpreting the index scores of the WAIS-III because the general factor accounts for the bulk of the common variance in the French WAIS-III. The 4 index scores cannot be considered to reflect only broad ability because they include a strong contribution of the general factor.     

项目表述效应对人格测验信度和效标效度的影响——以核心自我评价量表(CSES)为例

项目表述效应是指由项目表述方式的差异引起的与测量内容无关的系统变异,项目表述效应模型的统计本质是一种双因子模型。本研究以核心自我评价量表(CSES)为例,探讨项目表述效应对人格测验信效度的影响。采用核心自我评价量表、生活满意度量表和积极情感消极情感量表对340名“蚁族”进行测查。结果表明,CSES在核心自我评价特质以外,还存在一个反向题项目表述效应因子;忽视项目表述效应对CSES的同质性信度和效标关联效度有重要影响：高估CSES的同质性信度,低估核心自我评价与生活满意度、积极情感的正相关,高估核心自我评价与消极情感的负相关。     

Using bifactor models to identify faking on Big Five questionnaires

To identify faking, bifactor models were applied to Big Five personality data in three studies of laboratory and applicant samples using within‐subjects designs. The models were applied to homogenous data sets from separate honest, instructed faking, applicant conditions, and to simulated applicant data sets containing random individual responses from honest and faking conditions. Factor scores from the general factor in a bifactor model were found to be most highly related to response condition in both types of data sets. Domain factor scores from the faking conditions were found less affected by faking in measurement of Big Five domains than summated scale scores across studies. We conclude that bifactor models are efficacious in assessing the Big Five domains while controlling for faking.     

Structural validity of the Mental Health Continuum-Short Form: The bifactor model of emotional,social and psychological well-being

The Mental Health Continuum-Short Form (MHC-SF) is a growingly popular questionnaire designed to assess three components of well-being: emotional, social, and psychological. The main goal of the present study was to evaluate the structural validity of the MHC-SF and test the bifactor model of the MHC-SF, which includes one general factor and three specific factors of well-being. Sample 1 consisted of 1095 Serbian students (aged 18–26 years), while Sample 2 included 325 Serbian adults (aged 27–63 years). The bifactor model of the MHC-SF yielded the best fit to the data across the two samples. The results showed that the general factor of well-being accounted for substantially greater amount of variance of the MHC-SF than three specific factors of well-being. After controlling for the general factor, three specific factors explained a small portion of variance in well-being. In addition, the three subscales of the MHC-SF showed low reliability as estimated by omega-subscale coefficients, indicating that these subscales comprise too small amount of reliable variance to interpret. The present findings suggest that researchers should not calculate separate scores for three types of well-being when using the MHC-SF and that alternative measures of specific components of well-being should be considered.     

The MMPI-2 Clinical Scales and Restructured Clinical (RC) Scales: comparative psychometric properties and relative diagnostic efficiency in young adults

We examined the psychometric properties of the Restructured Clinical (RC) scales (Tellegen et al., 2003) of the MMPI-2 (Butcher, Dahlstrom, Graham, Tellegen, & Kaemmer, 1989) in a large sample (N = 744) of 18-year-old college freshman. We found that the RC scales demonstrated good convergence with their Clinical scale counterparts and were more distinctive than the Clinical scales. The patterns of discriminant correlations for the RC scales were slightly clearer than those of the Clinical scales and a set of other existing MMPI-2 scales. Diagnostic efficiency statistics based on Clinical and RC scale elevation status did not differ appreciably. However, the diagnostic efficiency statistics of cutoff scores derived from mean RC and Clinical scale T scores improved on the traditional scale elevation measures. We consider the clinical implications of these findings.     

Bifactor Modeling and the Estimation of Model-Based Reliability in the WAIS-IV

Previous confirmatory factor analytic research that has examined the factor structure of the Wechsler Adult Intelligence Scale–Fourth Edition (WAIS-IV) has endorsed either higher order models or oblique factor models that tend to amalgamate both general factor and index factor sources of systematic variance. An alternative model that has not yet been examined for the WAIS-IV is the bifactor model. Bifactor models allow all subtests to load onto both the general factor and their respective index factor directly. Bifactor models are also particularly amenable to the estimation of model-based reliabilities for both global composite scores (ω _h ) and subscale/index scores (ω _s ). Based on the WAIS-IV normative sample correlation matrices, a bifactor model that did not include any index factor cross loadings or correlated residuals was found to be better fitting than the conventional higher order and oblique factor models. Although the ω _h estimate associated with the full scale intelligence quotient (FSIQ) scores was respectably high (.86), the ω _s estimates associated with the WAIS-IV index scores were very low (.13 to .47). The results are interpreted in the context of the benefits of a bifactor modeling approach. Additionally, in light of the very low levels of unique internal consistency reliabilities associated with the index scores, it is contended that clinical index score interpretations are probably not justifiable.     

Two Concepts or Two Approaches? A Bifactor Analysis of Psychological and Subjective Well-Being

Researchers often debate about whether there is a meaningful differentiation between psychological well-being and subjective well-being. One view argues that psychological and subjective well-being are distinct dimensions, whereas another view proposes that they are different perspectives on the same general construct and thus are more similar than different. The purpose of this investigation was to examine these two competing views by using a statistical approach, the bifactor model, that allows for an examination of the common variance shared by the two types of well-being and the unique variance specific to each. In one college sample and one nationally representative sample, the bifactor model revealed a strong general factor, which captures the common ground shared by the measures of psychological well-being and subjective well-being. The bifactor model also revealed four specific factors of psychological well-being and three specific factors of subjective well-being, after partialling out the general well-being factor. We further examined the relations of the specific factors of psychological and subjective well-being to external measures. The specific factors demonstrated incremental predictive power, independent of the general well-being factor. These results suggest that psychological well-being and subjective well-being are strongly related at the general construct level, but their individual components are distinct once their overlap with the general construct of well-being is partialled out. The findings thus indicate that both perspectives have merit, depending on the level of analysis.     

The stability of structural measures derived from repertory grids

Although the use of repertory grids in psychological research has proliferated, studies of their psychometric properties are relatively rare. For this reason, we studied the reliability and convergence of several measures of cognitive structure derived from grids, including intensity, percentage of variance accounted for by the first component, cognitive complexity, ordination, extremity of ratings, self-ideal discrepancy, and self-other discrepancy, as well two measures of rating stability, construct consistency and factor loading consistency. Eighty-two Spanish (Catalan) and American participants completed small and large grids on each of four occasions, which enabled exploratory analyses of the impact of grid size and subject characteristics on the structural scores obtained. Results indicated that the majority of scores showed impressive test-relest reliability (modal r = .85 for periods up to 1 month), but also suggested a gradual tightening effect across subsequent administrations. In general, scores intercorrelated in theoretically anticipated ways, converging on the assessment of cognitive differentiation and discrimination of elements within construct dimensions. The results have implications for the reactivity of structural scores to variations in grid format, as well as for use of grids in the study of sex differences and cross-cultural comparisons.     

Dimensionality and psychometric invariance of the Youth Self-Report Form of the Child Behavior Checklist in cross-national settings

The Youth Self-Report Form's (YSR's) factor model was derived from traditional exploratory factor analytical procedures. Assuming appropriate model specification, psychometrically invariant items, and that its items provide useful psychometric information across nations omitted from its normative samples, the YSR is widely used in cross-national studies of nonreferred children. Item response theory analytical procedures reveal (a) 2 dimensions partly overlapping with the YSR's Internalizing and Externalizing second-order factors; (b) variance (i.e., differential item functioning) in how well a few items discriminate for nonreferred children across two nations; and (c) variance in estimating severity levels in children with identical psychopathological severity cross-nationally. Addressing psychometric variance, limiting redundancy, and matching children's psychopathological severity levels with items measuring this severity might promote more accurate and economical assessment.     

Modeling the Structure of the Attitudes and Belief Scale 2 using CFA and Bifactor Approaches: Toward the Development of an Abbreviated Version

The Attitudes and Belief Scale-2 (ABS-2: DiGiuseppe, Leaf, Exner, & Robin, 1988. The development of a measure of rational/irrational thinking. Paper presented at the World Congress of Behavior Therapy, Edinburg, Scotland.) is a 72-item self-report measure of evaluative rational and irrational beliefs widely used in Rational Emotive Behavior Therapy research contexts. However, little psychometric evidence exists regarding the measure's underlying factor structure. Furthermore, given the length of the ABS-2 there is a need for an abbreviated version that can be administered when there are time demands on the researcher, such as in clinical settings. This study sought to examine a series of theoretical models hypothesized to represent the latent structure of the ABS-2 within an alternative models framework using traditional confirmatory factor analysis as well as utilizing a bifactor modeling approach. Furthermore, this study also sought to develop a psychometrically sound abbreviated version of the ABS-2. Three hundred and thirteen (N = 313) active emergency service personnel completed the ABS-2. Results indicated that for each model, the application of bifactor modeling procedures improved model fit statistics, and a novel eight-factor intercorrelated solution was identified as the best fitting model of the ABS-2. However, the observed fit indices failed to satisfy commonly accepted standards. A 24-item abbreviated version was thus constructed and an intercorrelated eight-factor solution yielded satisfactory model fit statistics. Current results support the use of a bifactor modeling approach to determining the factor structure of the ABS-2. Furthermore, results provide empirical support for the psychometric properties of the newly developed abbreviated version.     

State-trait characteristics of various measured variables of psychophysiologic activation, cognitive performance and mood in common everyday situations]

The distinction between state and trait sources of variation in psychological variables, familiar in differential psychology ever since the work of Cattell, is translated into a linear model of state-trait analysis. On the basis of this model indices of relative state- and trait variance of a measure (independent of errors of measurement) and the psychometric reliability of the same measure (independent of true state variance) can be determined. The model is applied to a larger set of data from in-field research employing 149 male high-school students (average age 17 years) from the city of Hamburg. Following a time-sampling plan of data acquisition, subjects were requested over a period of three successive weeks and on the average twelve times a day to indicate (on a portable computerized behavioral data recorder) their current mood and feeling and had to answer short psychometric tests (in-field psychometrics); in addition, two peripheral psychophysiological measures of activation (heart rate, finger temperature) were recorded. The psychophysiological measures depended on state variations to about two thirds of their true score variance, the different cognitive tests to about 50 to 80 percent of their true score variance on trait variations. Different scales for recording current mood and feeling states gave different state-trait variance percentages. The method also allows for determining intraindividual psychometric state reliability coefficients. Fields of application of the method in different areas of psychology are indicated.     

Stress assessment during adolescence: Psychometric properties and measurement invariance of the Student Stress Inventory-Stress Manifestations across gender and age

The main purpose of this study was to analyze the psychometric properties and measurement invariance across gender and age of the Student Stress Inventory-Stress Manifestations (SSI-SM) scores in a large sample of adolescents. The sample was comprised by a total of 1108 students (482 were male), with a mean age of 14.61 years (SD = 1.71). The results indicated that the SSI-SM scores presented adequate psychometric properties from both classical test theory and Item Response Theory (IRT). Confirmatory factorial analysis (CFA), showed that both the bifactor model and a three-factor model (emotional, physiological, and behavioural) were adequate. Multi-group CFA showed that the three-factor model had strong measurement invariance across gender and age. Statistically significant differences in gender were found between latent means as well as raw scores of SSI-SM. Ordinal alpha was .78 for Physiological, .90 for the Emotional, and .79 for the Behavioural subscales. Using IRT, the SSI-SM provides more accuracy information at the medium level of the latent trait. SSI-SM subscales were associated with emotional and behavioural problems. These results provided new sources of validity evidence of the SSI-SM scores in adolescents from general population. The SSI-SM appears to be a useful, brief, and easy to administrate self-report instrument for the screening of stress manifestations at school and educational settings.