Examination of the structural, convergent, and incremental validity of the Reynolds Intellectual Assessment Scales (RIAS) with a clinical sample

Empirical examination of the Reynolds Intellectual Assessment Scales (RIAS; C. R. Reynolds & R. W. Kamphaus, 2003a) has produced mixed results regarding its internal structure and convergent validity. Various aspects of validity of RIAS scores with a sample (N = 521) of adolescents and adults seeking psychological evaluations at a university-based clinic were examined. Results from exploratory factor analysis indicated only 1 factor, and confirmatory factor analysis (CFA) indicated that the 1-factor model was a good fit and a better fit than the 2-factor model. Hierarchical factor analysis indicated the higher order, general intelligence factor accounted for the largest amount of variance. Correlations with other measures of verbal/crystallized and nonverbal/fluid intelligence were supportive of the convergent validity of the Verbal Intelligence Index but not the Nonverbal Intelligence Index. Joint CFA with these additional measures resulted in a superior fit of the 2-factor model compared with the 1-factor model, although the Odd-Item-Out subtest was found to be a poor measure of nonverbal/fluid intelligence. Incremental validity analyses indicated that the Composite Intelligence Index explained a medium to large portion of academic achievement variance; the NIX and VIX explained a small amount of remaining variance. Implications regarding interpretation of the RIAS when assessing similar individuals are discussed.     

Common and specific dimensions of self-reported anxiety and depression: the BDI-II versus the BDI-IA

The Beck Depression Inventory-II (BDI-II) [Beck, A. T., Steer, R. A. & Brown, G. K. (1996). Manual for Beck Depression Inventory-II. San Antonio, TX: Psychological Corporation.] and Anxiety Inventory (BAI) [Beck, A. T. & Steer, R. A. (1993a). Manual for the Beck Anxiety Inventory. San Antonio, TX: Psychological Corporation.] were administered to 840 outpatients who were diagnosed with various types of psychiatric disorders to determine whether the general symptom compositions and relative amounts of variance of the common and specific dimensions of self-reported anxiety and depression for these instruments would be comparable to those that had been found by Steer et al. [Steer, R. A., Clark, D. A., Beck, A. T. & Ranieri, W. F. (1995). Common and specific dimensions of self-reported anxiety and depression: A replication. Journal of Abnormal Psychology, 104, 542–545.] with the BAI and amended Beck Depression Inventory (BDI-IA) [Beck, A. T. & Steer, R. A. (1993b). Manual for the Beck Depression Inventory. San Antonio, TX: Psychological Corporation.]. A Schmid–Leiman transformation was used with the iterated-principal-factor pattern matrix of the BAI and the BDI-II loadings and indicated that the overall symptom compositions and relative amounts of variance that were explained by the one common and two specific anxiety and depression dimensions were comparable to those previously found with the BDI-IA.     

Direct Schmid–Leiman Transformations and Rank-Deficient Loadings Matrices

The Schmid–Leiman (S–L; Psychometrika 22: 53–61, 1957) transformation is a popular method for conducting exploratory bifactor analysis that has been used in hundreds of studies of individual differences variables. To perform a two-level S–L transformation, it is generally believed that two separate factor analyses are required: a first-level analysis in which k obliquely rotated factors are extracted from an observed-variable correlation matrix, and a second-level analysis in which a general factor is extracted from the correlations of the first-level factors. In this article, I demonstrate that the S–L loadings matrix is necessarily rank deficient. I then show how this feature of the S–L transformation can be used to obtain a direct S–L solution from an unrotated first-level factor structure. Next, I reanalyze two examples from Mansolf and Reise (Multivar Behav Res 51: 698–717, 2016) to illustrate the utility of ‘best-fitting’ S–L rotations when gauging the ability of hierarchical factor models to recover known bifactor structures. Finally, I show how to compute direct bifactor solutions for non-hierarchical bifactor structures. An online supplement includes R code to reproduce all of the analyses that are reported in the article.     

Notes on selective influence, probabilistic causality, and probabilistic dimensionality

The paper provides conceptual clarifications for the issues related to the dependence of jointly distributed systems of random entities on external factors. This includes the theory of selective influence as proposed in Dzhafarov [(2003a). Selective influence through conditional independence. Psychometrika, 68, 7-26] and generalized versions of the notions of probabilistic causality [Suppes, P., & Zanotti, M. (1981). When are probabilistic explanations possible? Synthese, 48, 191-199] and dimensionality in the latent variable models [Levine, M. V. (2003). Dimension in latent variable models. Journal of Mathematical Psychology, 47, 450-466]. One of the basic observations is that any system of random entities whose joint distribution depends on a factor set can be represented by functions of two arguments: a single factor-independent source of randomness and the factor set itself. In the case of random variables (i.e., real-valued random entities endowed with Borel sigma-algebras) the single source of randomness can be chosen to be any random variable with a continuous distribution (e.g., uniformly distributed between 0 and 1).     

Short-term predictive validity of cluster analytic and dimensional classification of child behavioral adjustment in school

A constructive debate over the classification of child psychopathology can be stimulated by investigating the validity of different classification approaches. We examined and compared the short-term predictive validity of cluster analytic and dimensional classifications of child behavioral adjustment in school using the Behavior Assessment System for Children (BASC; [Reynolds, C. R., and Kamphaus, R. W. (1992). Behavior Assessment System for Children (BASC). Circle Pines, MN: AGS.]) Teacher Rating Scales for Children (TRS-C). We cluster analyzed 14 dimensional scores of children's behavior in first grade and identified seven clusters. Then we examined the predictive power of the dimensions and the clusters using a variety of school outcomes in second grade, including academic achievement scores in reading and math, absenteeism, discipline reports, and participation rates in pre-referral intervention. Both methods significantly predicted all school outcomes with similar magnitudes of effect sizes, but the dimensional approach was more powerful in predicting the outcomes except pre-referral intervention. Moreover, coherent group differences in the clusters were identified with respect to the school outcomes. This latter finding may suggest differentiated prognoses of cluster membership. The implications for future validation studies of alternative classification systems of child behavioral adjustment are discussed.     

Bifactor models and rotations: exploring the extent to which multidimensional data yield univocal scale scores

The application of psychological measures often results in item response data that arguably are consistent with both unidimensional (a single common factor) and multidimensional latent structures (typically caused by parcels of items that tap similar content domains). As such, structural ambiguity leads to seemingly endless "confirmatory" factor analytic studies in which the research question is whether scale scores can be interpreted as reflecting variation on a single trait. An alternative to the more commonly observed unidimensional, correlated traits, or second-order representations of a measure's latent structure is a bifactor model. Bifactor structures, however, are not well understood in the personality assessment community and thus rarely are applied. To address this, herein we (a) describe issues that arise in conceptualizing and modeling multidimensionality, (b) describe exploratory (including Schmid-Leiman [Schmid & Leiman, 1957] and target bifactor rotations) and confirmatory bifactor modeling, (c) differentiate between bifactor and second-order models, and (d) suggest contexts where bifactor analysis is particularly valuable (e.g., for evaluating the plausibility of subscales, determining the extent to which scores reflect a single variable even when the data are multidimensional, and evaluating the feasibility of applying a unidimensional item response theory (IRT) measurement model). We emphasize that the determination of dimensionality is a related but distinct question from either determining the extent to which scores reflect a single individual difference variable or determining the effect of multidimensionality on IRT item parameter estimates. Indeed, we suggest that in many contexts, multidimensional data can yield interpretable scale scores and be appropriately fitted to unidimensional IRT models.     

An exploration of genetic g: Hierarchical factor analysis of cognitive data from the western reserve twin project

Hierarchical factor analyses involving Schmid-Leiman transformations (Schmid & Leiman, 1957) were conducted on specific cognitive abilities data collected in a sample of 148 identical (MZ) and 135 same-sex fraternal (DZ) twin pairs. Two main questions were addressed: First, are genetic influences on specific cognitive abilities simply a reflection of their g loading, or are different sets of genes affecting separate abilities? Second, to the extent that specific cognitive abilities are affected by common genetic variance, how similar is the common genetic factor to a phenotypic factor reflecting g? Model fitting results suggest that genetic influences on specific abilities are a reflection of both general intelligence and genetic influences specific to separate abilities and that loadings on the common genetic factor are more highly correlated with phenotypic g loadings than are common environmental factor loadings.     

Exploring item and higher order factor structure with the Schmid-Leiman solution: Syntax codes for SPSS and SAS

To ease the interpretation of higher order factor analysis, the direct relationships between variables and higher order factors may be calculated by the Schmid-Leiman solution (SLS; Schmid & Leiman, 1957). This simple transformation of higher order factor analysis orthogonalizes first-order and higher order factors and thereby allows the interpretation of the relative impact of factor levels on variables. The Schmid-Leiman solution may also be used to facilitate theorizing and scale development. The rationale for the procedure is presented, supplemented by syntax codes for SPSS and SAS, since the transformation is not part of most statistical programs. Syntax codes may also be downloaded from www.psychonomic.org/archive/.     

Assessing the accuracy of the WISC-IV seven-subtest short form and the child and adolescent intellectual disability screening questionnaire in identifying intellectual disability in children

Little research has been conducted into the accuracy of abbreviated assessments in identifying children and young people with an intellectual disability (ID). The present study compared two such methods in a clinical population of individuals with (n = 106) and without (n = 170) ID: a 7-subtest short form of the Wechsler Adult Intelligence Scales for Children—fourth edition (WISC-IV) proposed by Crawford and colleagues and the Child and Adolescent Intellectual Disability Screening Questionnaire (CAIDS-Q). Both the CAIDS-Q and the WISC-IV short form had high and comparable levels of predicting group classification (88% and 91% correct classification, respectively). Both methods would appear to offer clinicians and researchers an efficient and accurate means of identifying those who are likely to have ID. The WISC-IV short form was slightly more accurate, but the CAIDS-Q may offer the advantages of being shorter to administer and having no requirement for the user to have a particular qualification or training.     

Test theory and psychometrika: The past twenty-five years

On the occasion ofPsychometrika's fiftieth anniversary, the past twenty-five years' developments in mental test theory are reviewed, with special emphasis on the topics receiving attention in the pages of this journal. (Analogous reviews forPsychometrika's first quarter century were given by Gulliksen and Guilford in 1961.) Much of the recent progress in test theory (and in other branches of psychometrics as well) has been made by treating the problems in this field as being essentially ones of statistical inference. It is concluded that (a) research in test theory is in a healthy state and (b)Psychometrika is an important source of information about that research.     

Dimensionality of posttraumatic stress symptoms: a confirmatory factor analysis of DSM-IV symptom clusters and other symptom models

Recent exploratory [Taylor, S., Kuch, K., Koch, W. J., Crockett, D. J., & Passey, G. (1998). The structure of posttraumatic stress symptoms. Journal of Abnormal Psychology, 107, 154-160.] and confirmatory [Buckley, T. C., Blanchard, E. B., & Hickling, E. J. (1998). A confirmatory factor analysis of posttraumatic stress symptoms. Behaviour Research and Therapy, 36, 1091-1099; King, D. W., Leskin, G. A., King, L. A., & Weathers, F. W. (1998). Confirmatory factor analysis of the clinician-administered PTSD scale: evidence for the dimensionality of posttraumatic stress disorder. Psychological Assessment, 10, 90-96.] factor analytic investigations suggest that the three symptom clusters of posttraumatic stress disorder (PTSD) as defined in the Diagnostic and Statistical Manual [4th ed.; DSM-IV; American Psychiatric Association (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: Author.] may not provide the best conceptualization of symptom dimensionality. However, the alternative models have not been in agreement, nor have they been compared against each other or models based on the DSM-IV. The purpose of the present investigation was to test a series of dimensional models suggested by these recent factor analytic investigations and the DSM-IV. Using data collected with the PTSD Checklist--Civilian Version [Weathers, F. W., Litz, B. T., Huska, J. A., & Keane, T. M. (1994). PCL-C for DSM-IV. Boston: National Center for PTSD--Behavioral Science Division.] from 349 referrals to a primary care medical clinic, we used confirmatory factor analysis to evaluate a: (1) hierarchical four-factor model, (2) four-factor intercorrelated model, (3) hierarchical three-factor model, (4) three-factor intercorrelated model, and (5) hierarchical two-factor model. The hierarchical four-factor model (comprising four first-order factors corresponding to reexperiencing, avoidance, numbing, and hyperarousal all subsumed by a higher-order general factor) provided the best overall fit to the data; although, all models met some standards specified for good model fit. More research is needed to establish the dimensional nature of PTSD symptoms and to assess whether identified dimensions differ as a function of the trauma experience. Implications for assessment, diagnosis, and treatment are also discussed.     

Illness Attitudes Scale dimensions and their associations with anxiety-related constructs in a nonclinical sample

The Illness Attitudes Scale (IAS) is a self-rated measure that consists of nine subscales designed to assess fears, attitudes and beliefs associated with hypochondriacal concerns and abnormal illness behavior [Kellner, R. (1986). Somatization and hypochondriasis. New York: Praeger; Kellner, R. (1987). Abridged manual of the Illness Attitudes Scale. Department of Psychiatry, School of Medicine, University of New Mexico]. The purposes of the present study were to explore the hierarchical factor structure of the IAS in a nonclinical sample of young adult volunteers and to examine the relations of each illness attitudes dimension to a set of anxiety-related measures. One-hundred and ninety-seven undergraduate university students (156 F, 41 M; mean age = 21.9 years) completed the IAS as well as measures of anxiety sensitivity, trait anxiety and panic attack history. The results of principal components analyses with oblique (Oblimin) rotation suggested that the IAS is best conceptualized as a four-factor measure at the lower order level (with lower-order dimensions tapping illness-related Fears, Behavior, Beliefs and Effects, respectively), and a unifactorial measure at the higher-order level (i.e. higher-order dimension tapping General Hypochondriacal Concerns). The factor structure overlapped to some degree with the scoring of the IAS proposed by Kellner (1986, 1987), as well as with the factor structures identified in previously-tested clinical and nonclinical samples [Ferguson, E. & Daniel, E. (1995). The Illness Attitudes Scale (IAS): a psychometric evaluation on a nonclinical population. Personality and Individual Differences, 18, 463-469; Hadjistavropoulos, H. D. & Asmundson, G. J. G. (1998). Factor analytic investigation of the Illness Attitudes Scale in a chronic pain sample. Behaviour Research and Therapy, 36, 1185-1195; Hadjistavropoulos, H. D., Frombach, I. & Asmundson, G. J. G. (in press). Exploratory and confirmatory factor analytic investigations of the Illness Attitudes Scale in a nonclinical sample. Behaviour Research and Therapy; Speckens, A. E., Spinhoven, P., Sloekers, P. P. A., Bolk, J. H. & van Hemert, A. M. (1996). A validation study of the Whitley Index, the Illness Attitude Scales and the Somatosensory Amplification Scale in general medical and general practice patients. Journal of Psychosomatic Research, 40, 95-104]. The Fears, Beliefs and Effects lower-order factors and the General Hypochondriacal Concerns higher-order factor, were shown to be strongly associated with anxiety sensitivity, even after accounting for trait anxiety and panic history. Implications for understanding the high degree of comorbidity between the diagnoses of panic disorder and hypochondriasis, as well as future research directions for exploring the utility of various IAS dimensions in predicting responses to lab-based bodily symptom-induction procedures, are discussed.     

The role of memory for compounds in cue competition

Revisions of common associative learning models incorporate a within-compound association mechanism in order to explain retrospective cue competition effects (e.g., [Dickinson, A., & Burke, J. (1996). Within-compound associations mediate the retrospective revaluation of causality judgements. Quarterly Journal of Experimental Psychology, 49B, pp. 60-80.]). These models predict a correlation between memory for compounds (as a measure for the strength of within-compound associations) and retrospective cue competition but not forward cue competition. This was indeed found in a study of [Melchers, K. G., Lachnit, H., & Shanks, D. (2004). Within-compound associations in retrospective revaluation and in direct learning: A challenge for comparator theory. Quarterly Journal of Experimental Psychology, 57B, pp. 25-54.]. We argue that a higher-order reasoning account of causal learning can also explain the evidence for the role of within-compound associations in cue competition. Moreover, this account predicts that if making inferences during the learning stage is impeded, a correlation between memory for compounds and forward cue competition should be found as well. The results of a new study confirmed this prediction.     

An Examination of the Latent Structure of the Difficulties in Emotion Regulation Scale

The Difficulties in Emotion Regulation Scale (DERS; Gratz and Roemer, Journal of Psychopathology and Behavioral Assessment 26:41?C54, 2004) is a popular multidimensional self-report measure of emotion regulation. The present study sought to examine the latent factor structure of the DERS. An examination of latent factor intercorrelations and a higher-order confirmatory factor analysis (CFA) suggested that the DERS-AWARENESS dimension may not represent the same higher-order emotion regulation construct as the other five DERS dimensions. Furthermore, findings supported the adequacy of a revised five-factor model of the DERS in which the AWARENESS dimension was removed. This revised DERS total scale did not diminish concurrent relations between the DERS and outcomes relevant to the emotion regulation domain (i.e., depression, anxiety, posttraumatic stress symptoms). Implications for the conceptualization and assessment of emotion regulation are discussed.     

Unconscious semantic categorization and mask interactions: An elaborate response to Kunde et al. (2005)

In their original report [Kunde, W., Kiesel, A., & Hoffmann, J. (2003). Conscious control over the content of unconscious cognition. Cognition, 88, 223-242] maintain that “unconscious stimuli [do not] owe their impact […] to automatic semantic categorization” (p.223), and instead propose the action-trigger theory of unconscious priming. In a reply to our paper [Kunde, W., Kiesel, A., & Hoffmann, J. (2005). On the masking and disclosure of unconscious semantic processing. A reply to Van Opstal, Reynvoet, & Verguts (2005). Cognition], the authors adopt a reconcilist position, and propose that both theories may be valid depending on the experimental situation. We discuss the evidence in favor of this position. [Kunde, W., Kiesel, A., & Hoffmann, J. (2005). On the masking and disclosure of unconscious semantic processing. A reply to Van Opstal, Reynvoet, & Verguts (2005). Cognition] also propose an alternative account of our mask-type blocking hypothesis. We report an experiment that distinguishes between our original and their alternative hypothesis.     

Generalized multivariate lens model analysis for complex human inference tasks

A generalized multivariate lens model is presented which will permit the analysis of complex human inference tasks. Such tasks, occurring in their natural ecology, may involve judgments or decisions on multiple criteria and/ or where the influence of theoretically interesting partitions or augmentations of cue profiles needs to be systematically delineated. The generalized model incorporates the standard, hierarchical, and fully partialed lens models, the initial elaborations of which were made by N. J. Castellan (1972, Organizational Behavior and Human Performance, 8, 242–261) and T. R. Stewart (1976, Psychometrika, 41, 101–120). Both multivariate and univariate lens model equations are presented within a common notational system. An example which demonstrates some aspects of the generalized model is discussed in detail. Other potential applications for the model are outlined.