A bayesian approach to confirmatory factor analysis

Confirmatory factor analysis is considered from a Bayesian viewpoint, in which prior information on parameter is incorporated in the analysis. An iterative algorithm is developed to obtain the Bayes estimates. A numerical example based on longitudinal data is presented. A simulation study is designed to compare the Bayesian approach with the maximum likelihood method.Computer facilities were provided by the Computer Services Center, The Chinese University of Hong Kong.     

Sixteen-item Anxiety Sensitivity Index: confirmatory factor analytic evidence, internal consistency, and construct validity in a young adult sample from the Netherlands

The present investigation examined the factor structure, internal consistency, and construct validity of the 16-item Anxiety Sensitivity Index (ASI; Reiss Peterson, Gursky, & McNally 1986) in a young adult sample (n = 420) from the Netherlands. Confirmatory factor analysis was used to comparatively evaluate two-factor, three-factor, and four-factor models of the anxiety sensitivity construct. Support was found for a hierarchical structure of anxiety sensitivity, with one global higher-order factor and four lower-order factors. Internal consistency for the global and lower-order factors of the 16-item ASI was adequate. Convergent and discriminant associations between the 16-item ASI and general mood and panic-specific variables were consistent with anxiety sensitivity theory. In addition, incremental validity of the anxiety sensitivity construct was established, relative to negative affectivity, for unexpected panic attacks and agoraphobic avoidance.     

Review of clinical validation of ADHD behavior rating scales

The purpose of this review is to assess the range of overall accuracies for Attention Deficit/Hyperactivity Disorder (ADHD) behavior rating scales evaluated in clinical validation studies. Studies were characterized according to the evidence standards of the American Academy of Neurology (AAN). Studies were excluded due to major design problems such as overfitting by discriminant analysis. The 13 included evaluations of rating scales revealed overall accuracy in the range of 59%-79% with a pooled mean of 69% (+/-7%, standard deviation) and a pooled sample size of 2228 subjects from nine studies. While some of the excluded studies demonstrated higher overall accuracies (>79%), these studies were observed to have factors in experimen tal design and statistics that are known to unduly inflate accuracy. We recommend further research following the full AAN standards, namely well-designed, blinded, pro spective studies of rating scales applied to clinically representative samples evaluated with a clinical standard.     

A general approach to confirmatory maximum likelihood factor analysis

We describe a general procedure by which any number of parameters of the factor analytic model can be held fixed at any values and the remaining free parameters estimated by the maximum likelihood method. The generality of the approach makes it possible to deal with all kinds of solutions: orthogonal, oblique and various mixtures of these. By choosing the fixed parameters appropriately, factors can be defined to have desired properties and make subsequent rotation unnecessary. The goodness of fit of the maximum likelihood solution under the hypothesis represented by the fixed parameters is tested by a large samplex ² test based on the likelihood ratio technique. A by-product of the procedure is an estimate of the variance-covariance matrix of the estimated parameters. From this, approximate confidence intervals for the parameters can be obtained. Several examples illustrating the usefulness of the procedure are given.This work was supported by a grant (NSF-GB 1985) from the National Science Foundation to Educational Testing Service.     

A confirmatory factor analytic approach on perceptions of knowledge and skills in teaching (PKST)

This paper reports the cross-validation of the factor pattern of the Perceptions of Knowledge and Skills in Teaching (PKST) survey, which was used to assess the self-perceived pedagogical knowledge and skills of pre-service and beginning teachers. The sample comprised 323 pre-service teachers enrolled in a 1-yr. post-graduate teacher education program in Singapore. The survey had 37 items distributed across six scales: student learning, lesson planning, instructional support, accommodating diversity, classroom management, and care and concern. A confirmatory factor analysis (CFA) was used to cross-validate the survey's factor pattern. The results showed that the model was an acceptable fit to the data. The PKST survey can thus be adapted by different teacher education programs to assess pre-service and beginning teachers' progress in developing their pedagogical knowledge and skills.     

Judgmental relativism as a validity threat to standardized psychiatric rating scales

Judgemental relativism is a threat to the replicability and validity of measures of client behavior from direct rating scales whenever raters are exposed to different levels of client functioning since the internal standards, or anchor points, used to judge dimensional continua may vary on the basis of prior experience. Traditional interrater reliability indexes fail to identify such effects. The influence of judgmental relativism on summated ratings from the Nurses Observational Scale for Inpatient Evaluation (NOSIE-30) for 1040 adult mentally ill clients was examined with clinical staff raters from 24 treatment units in which the Time-Sample Behavioral Checklist (TSBC) provided full-week objective measures of actual client functioning via hourly direct observational coding (DOC). Regression analyses found that the same level of objective performance received higher or lower ratings across treatment units dependent on the raters'exposure to client groups that differed in level of functioning. Analyses of rating errors found that clients with better levels of functioning relative to others within treatment units were rated even higher than performance warranted. The operation of halo and contrast effects is explored and guidelines are provided for determining when judgmental relativism may produce or nullify significant differences. DOC assessments should be used instead of retrospective ratings to support most decisions in residential settings. Specific recommendations for the application of rating scales and improving data quality are provided.This study was the basis of a master's thesis at the University of Houston by Betty E. Rich under the direction of Gordon L. Paul and Marco J. Mariotto. Richard M. Rozelle, to whom appreciation is expressed for helpful comments, served on the examination committee. This study was partially supported by grants to Gordon L. Paul from the National Institute of Mental Health, Public Health Service (MH-15353; MH-25464); the Illinois Department of Mental Health and Developmental Disabilities; the Joyce Foundation; the MacArthur Foundation; the Owsley Foundation; the Cullen Foundation; and the Center for Public Policy, University of Houston.     

Understanding source effects in ADHD rating scales: reply to DuPaul (2003)

G. J. DuPaul (2003) offered two suggestions for additional research to understand the strong source effects reported by R. Gomez, G. L. Burns, J. A. Walsh, and M. A. de Moura (2003) in attention-deficit/hyperactivity disorder (ADHD) rating scales. The first suggestion was to determine whether the source effects represent mostly bias or accuracy. The second suggestion was to minimize source effects through the development of better ADHD rating scales. Because source effects can represent bias or accuracy, it is important to minimize the bias aspect through content validation procedures prior to attempts to determine whether source effects better reflect bias or accuracy. This comment offers various suggestions to reduce the bias in ADHD rating scales.     

An extension of construct validity for personality scales using twin-based criteria

A twin-based analysis of personality inventory scales yields information relevant to construct validity. If scales validly measure dimensions influenced developmentally either by environmental factors common to cotwins or by genetic similarities, these influences should be apparent in patterns of intraclass correlations and within-pair variance ratios for monozygotic and dizygotic pairs. Criteria for construct validity derived from adolescent twin statistics are illustrated with Wiggins' MMPI Content Scales. A type of discriminant validity indicates that some scales are not distinct in terms of the influences of familial similarity. Sex differences are examined.     

Two promising shame and guilt scales: a construct validity comparison

This study compared the validity of two promising measures of shame and guilt proneness: revisions of the Harder Personal Feelings Questionnaire (PFQ2; Harder & Lewis, 1987) and the Hoblitzelle Adapted Shame and Guilt Scale (ASGS; Hoblitzelle, 1982). Internal consistency, test-retest stability, factor structure, and construct validity with convergent and discriminant personality dimensions were examined for both scales. In addition to the shame and guilt measures, 63 (37 male, 26 female) mostly freshman college students completed a randomly ordered battery of personality scales theoretically relevant to shame and guilt proneness. Results support the reliability and shame/guilt factor structure of each scale. ASGS Shame correlations appeared marginally more valid with 11 external construct variables than PFQ2 Shame, whereas PFQ2 Guilt was clearly more valid than its corresponding ASGS subscale. New, potentially improved scales were constructed from the factor analyses and from item analyses. However, the resulting scales did not show improved validity.     

The clinical utility of behavior rating scales: Comments on the diagnostic assessment of ADHD

Responding to Reid and Maag's critique of behavior rating scales in assessing ADHD, published in a previous issue of the Journal of School Psychology, we provided specific recommendations for improving the accuracy of diagnostic decisions using rating scales and affirmed the importance of these procedures in the assessment of ADHD. Strategies for managing the error intrinsic to the use of diagnostic measures, specifically behavior rating scales, were outlined. A multiple stage procedure, involving one or more levels of screening prior to conducting a diagnostic assessment, was recommended as a strategy to increase diagnostic accuracy in a time- and cost-efficient manner.     

Cross-sectional and longitudinal confirmatory factor models for the german test anxiety inventory: A construct validation

Construct validity of the German Test Anxiety Inventory (TAI-G) was tested in two respects. Firstly, the purported four-dimensional structure of the TAI-G (comprising subscales Emotionality, Worry, Interference, and Lack of Confidence) as well as relations of the test anxiety dimensions to self-efficacy were tested. Secondly, the trait conception of the TAI-G was tested within the framework of Latent State-Trait theory. The TAI-G was given to a student sample (N=302) on three occasions with a time interval of 2 weeks along with a study-specific self-efficacy scale on occasion 1. Dimensionality assumptions as well as relations with self-efficacy were tested using cross-sectional second-order confirmatory factor analysis. The trait conception was tested separately for TAI-G subscales by specifying longitudinal confirmatory factor models (Latent State-Trait models) and by calculating variance proportions of manifest variables (Latent State-Trait coefficients) referring to different sources of systematic variance (person, situation, and method) based on parameter estimates of the models. Results were supportive of both the purported four-dimensional structure and hypothesized relationships to self-efficacy (i.e., acceptable model fit) as well as of the trait conception of test anxiety (i.e., acceptable model fit and high proportion of variance due to person component). Implications for further validation studies were discussed.     

Evidence of construct validity of the interest scales on the Campbell Interest and Skill Survey

This study explored evidence of the construct validity of the interest scales on the Campbell Interest and Skill Survey (CISS; Campbell, Hyne, & Nilsen, 1992) by testing evidence for convergent validity with the Strong Interest Inventory (SII; Hansen & Campbell, 1985). Two hypotheses were formulated. First, matching CISS and SII scales were expected to be more positively correlated compared to non-matching scales. Second, Holland's hexagonal calculus assumption (Holland, 1973) was expected to emerge in the pattern of intercorrelations among CISS and SII scales. These hypotheses were tested using correlational and factor analyses. Results supported the hypotheses and demonstrated evidence of good convergent and construct validity for scores on the CISS interest scales. Implications for the use of the CISS in counseling and for research on Holland's theory are discussed, and directions for future research are presented.     

Investigating different factor structures of the psychopathy checklist: youth version: confirmatory factor analytic findings

There has been a recent push to extend the construct of psychopathy into adolescence, primarily as a result of the impressive reliability, validity, and utility of this construct in samples of adults. The value of this work rests, however, on creating an equally reliable and valid assessment tool for adolescents. One promising measure is the Psychopathy Checklist: Youth Version (A. E. Forth, D. S. Kosson, & R. D. Hare, 2003). The current study uses a large, diverse sample of serious adolescent offenders to assess the overall fit of various underlying factor structures of this measure and to test the equivalence of these models across sex and race/ethnicity. The results suggest that either a 3- or 4-factor model provides the best overall fit and that these models are invariant across sex and race/ethnicity. The decision to use the 3- or 4-factor model will likely hinge on researchers' underlying conceptualization of psychopathy, specifically whether antisocial behavior is viewed as a core feature of this construct.     

Exploring the psychometric properties and construct validity of the MCMI-III anxiety and avoidant personality scales

The Millon Clinical Multiaxial Inventory (MCMI; Millon, 1983) is a commonly used self-report instrument designed to aid in the assessment of Axis I and Axis II disorders. Concerns have been expressed regarding the procedures used in the normative research for the current version of the MCMI (MCMI-III; Millon, 1994) leading to a call for additional validity research on the MCMI-III (Retzlaff, 1996). In this study, we investigated the psychometric properties of the MCMI-III's Anxiety and Avoidant personality scales in a sample of patients diagnosed with Diagnostic and Statistical Manual of Mental Disorders (4th ed.; American Psychiatric Association, 1994) anxiety disorders. Our results suggest that the MCMI-III Avoidant scale is reliable (r =.89) and it was found to demonstrate appropriate convergent and divergent validity with other self-report measures. The MCMI-III Anxiety scale also showed adequate reliability (r =.78); however, our findings raise some concerns about the discriminant validity of this scale. A scale composed of the MCMI-III core anxiety items was found to have better discriminant validity. These findings are consistent with those reported by other researchers regarding the relationship between self-report measures of anxiety, avoidance, and depression. We conclude that the MCMI-III measures of anxiety and avoidance are consistent with other measures of these constructs and may provide valuable clinical information in this regard.     

Assessment of executive function in ADHD adolescents: contribution of performance tests and rating scales

This study aimed to analyze performance on measures of neuropsychological and behavioral executive functions (EF) in adolescents with attention deficit hyperactivity disorder (ADHD), and to evaluate the utility of performance-based tests for predicting scores on behavioral EF ratings. One hundred eighteen adolescents (75 ADHD and 43 controls) aged 12–16 years performed neuropsychological tests and completed a behavior rating scale of EF. The ADHD group presented significantly lower scores than controls on Full Scale IQ (FSIQ) and all indexes of the WISC-IV, except the verbal comprehension index (VCI). The ADHD group had significantly lower scores on performance-based tests of working memory, planning and inhibition, and on EF rating scales. Scores on the cognitive EF working memory, planning and flexibility modestly predicted performance on behavioral EF. The results suggest that the combined use of performance-based tests and rating scales provides valuable complementary information that can improve the assessment of executive domains in ADHD.     

On the history of rating scales

Rating scales constitute one of the most widely employed techniques in research on personality and individual differences. The historical background of rating scales is therefore a matter of considerable interest. Though Galton has generally been given credit for originating rating scale methodology, several applications of rating scales prior to Galton can be identified, and the seminal idea of rating scales can be traced back to Galen.     

How many fidgets in a pretty much: A critique of behavior rating scales for identifying students with ADHD

The school psychologist's role and responsibilities for meeting the needs of students with Attention-Deficit Hyperactivity Disorder (ADHD) have been expanding in recent years. Students with this disorder present refractory problems that may involve the school psychologist in the assessment and diagnosis of ADHD. Rating scales commonly are used, sometimes in conjunction with other techniques, for assessing and diagnosing ADHD. They often are presented as an objective way to quantify the severity of a child's behavior in comparison with a normative standard. Because rating scales have become such an integral component in the identification of children with ADHD, school psychologists should understand the limitations associated with this methodology. In this article we first describe behavior rating scales and difficulties in the use of cutoff scores to identify students as ADHD. Second, we describe how problems with interobserver agreement hamper the validity of rating scales and the subsequent conclusions that can be drawn about students' behavior. Finally, we present recommendations for obtaining more reliable and valid information from rating scales.     

Mapping the MMPI-2 Restructured Clinical scales onto normal personality traits: evidence of construct validity

The MMPI-2 Restructured Clinical (RC; Tellegen et al., 2003) scales were developed to remove common factor variance that saturates the clinical scales and create a more distinct set of measures, yielding a new set of scales with improved convergent and discriminant validity. In this study, we examined the relation between RC scale scores and scores on the Multidimensional Personality Questionnaire (MPQ; Tellegen, in press), a measure of normal personality, using a sample of 811 college students. The results indicate strong convergence between the RC scales and expected MPQ higher order factors and primary scales. The RC scales also demonstrated expected improved convergent and discriminant validity over the clinical scales.