Measurement invariance,factor analysis and factorial invariance

Several concepts are introduced and defined: measurement invariance, structural bias, weak measurement invariance, strong factorial invariance, and strict factorial invariance. It is shown that factorial invariance has implications for (weak) measurement invariance. Definitions of fairness in employment/admissions testing and salary equity are provided and it is argued that strict factorial invariance is required for fairness/equity to exist. Implications for item and test bias are developed and it is argued that item or test bias probably depends on the existence of latent variables that are irrelevant to the primary goal of test constructers.Presidential address delivered at the Annual Meeting of the Psychometric Society in Berkeley, California, June 18–20, 1993.     

Satisfaction with life scale: analysis of factorial invariance for adolescents and elderly persons

The purpose of this work is to analyze the factorial invariance of the Satisfaction With Life Scale across samples of adolescents and elderly persons. Data from 266 subjects were analyzed. Half were Spanish junior high-school students (65 girls and 68 boys) and the other half were Spanish elderly people (68 women and 65 men). Single-group analyses showed an acceptable one-factor model for both adolescent and elderly groups. Sequential multigroup analyses to test the equivalence of factor structures for adolescent and elderly groups showed that factor loadings and variances are not invariant. The scale is sensitive to age in these groups.     

A new measure for an important construct: the attachment and object relations inventory

This article describes the development of the Attachment and Object Relations Inventory (AORI). The AORI, a dimensional measure based on factor structure, was developed in response to the psychometric problems typical to attachment style measures that are categorical in nature. Test-retest reliability and internal consistency were high. A factor analysis of the 75 AORI items confirmed the existence of at least 6 factors and explained approximately 50% of the item variance. Convergent and discriminant validity of the AORI was supported through appropriate relationships with measures of depression, anxiety, and personality, and other measures of attachment and object relations.     

Construct validity of the relationship profile test: three-year retest reliability and links with core personality traits, object relations, and interpersonal problems

The Relationship Profile Test (RPT; Bornstein & Languirand, 2003) is a 30-item self-report measure of destructive overdependence, dysfunctional detachment, and healthy dependency. Previous research has shown that the 3 RPT subscales have adequate internal consistency, good retest reliability over 23 and 85 weeks, and good convergent and discriminant validity with respect to measures of attachment style, relatedness, gender role, self-concept, alexithymia, locus of control, need for approval, and life satisfaction. In this investigation, we assessed the 3-year retest reliability of RPT scores in a northeastern liberal arts college student sample (Study 1) and evaluated the convergent and discriminant validity of RPT scores in a southern university sample (Study 2). Results generally supported the long-term stability and convergent and discriminant validity of RPT scores but also pointed to some limitations of the measure. We discuss implications of these results and describe future research directions.     

Further evidence of reliability and validity for the driving anger expression inventory

This study provided evidence of reliability and validity for the four scales of the Driving Anger Expression Inventory. Alpha reliabilities for scales ranged from .84 to .89. Measures of aggressive anger expression while driving (Verbal Aggressive Expression, Personal Physical Aggressive Expression, and Use of the Vehicle to Express Anger scales) correlated positively with each other and negatively with the Adaptive/Constructive Expression scale. Scores on the three aggressive forms of anger expression correlated positively with trait anger and measures of driving-related anger, aggression, and risky behavior, whereas scores on the Adaptive/Constructive Expression scale correlated negatively with these variables. Reports of aggressive and risky behavior correlated most strongly with the Use of the Vehicle to Express Anger scale. Forms of anger expression were minimally or uncorrelated with rated trait anxiety and reports of moving violations, close calls, and accidents. Findings replicated earlier findings and provided further evidence for the reliability and validity of the Driving Anger Expression Inventory.     

Obsessions and compulsions: Dimensional structure, reliability, convergent and divergent validity of the padua inventory

This study presents data on dimensional structure, reliability, convergent and divergent validity of the Padua Inventory (PI). In a sample (n = 430) of normal Dutch subjects. The dimensional structure and the strength of the factors were comparable to those found in the Italian sample. The reliability of the PI was found to be satisfactory. As hypothesized substantial correlations were found between the PI and related scales of the Maudsley Obsessive-Compulsive Inventory (MOCI) and between the PI and the subscales Sensitivity, Hostility and Depression of the revised version of the Symptom Checklist. Low correlations were found between the PI on the one side and the Eysenck Personality Questionnaire revised Extraversion, Psychoticism and Social Desirability scale on the other side. Finally, the mean score of the PI differs across the Italian, American and Dutch samples. This is of theoretical interest and deserves further study.     

A factorial, reliability, and validity study of the Devereux Elementary School Behavior Rating Scale

This investigation explored the factor structure, reliability, and validity of the Devereux Elementary School Behavior Rating Scale, employing a large (n = 648) sample of children. Factor analysis suggested that the Devereux scale can be described by four factors, which were named Disruptive-Oppositional, Poor Comprehension-Disattention, Cooperative-Initiating, and Performance Anxiety. All four factors showed high internal consistency, and three of the four were stable over a 17-mo. period. Correlations of the four factors with academic achievement, IQ, socioeconomic status, and peer ratings of social competence are presented. All four factors showed significant relations with these variables, with Poor Comprehension-Disattention the strongest of all. Multiple regression analysis indicated that the Poor Comprehension-Disattention factor accounted for significant variance in academic achievement even after IQ was taken into account. Large differences between classroom means on the factor scores suggested that Devereux ratings for individual students may need to be interpreted cautiously.     

Examination of the reliability and validity of the personality assessment inventory

The reliability, discriminant validity, and construct validity of the Personality Assessment Inventory (PAI) — a multidimensional self-report measure of abnormal personality traits — were examined within the Australian context. Subjects included 151 normals, 30 alcoholics, and 30 schizophrenic patients. A subsample of 70 nonpsychiatric adults responded to the PAI items twice over a test-retest interval of 28 days. The resulting median retest coefficient was 0.7, indicating less than optimal stability. The median alpha (KR₂₁) coefficient was 0.8, suggesting somewhat narrow measurement scales. A significant multivariate main effect was obtained across groups after the effects of age and gender were removed. Multiple comparisons for each of the PAI scales revealed significant differences between the respective groups, as discussed. A higher-order scale factoring did not strongly support the purported PAI structure. In reanalyses of the correlation matrices included in the Professional Manual, the purported PAI factor structure was unable to be replicated for the standardization clinical sample (N=1246), and a confirmatory factor analysis using the normative (validation) correlational data (N=1000) revealed poor fit indices, raising further concerns about construct validity.     

Real and ideal self-images in second- and fifth-grade children: Reliability,validity and factorial structure

The reliability, construct validity, and factorial structure of three self-image instruments were assesses in a total of 80 second- and fifth-grade children. Both real self-images (children's current views of themselves) and ideal self-images (the self views to which children aspire) were assessed. the split-half and test—retest reliabilities of the instruments were adequate even for children's as young as second graders. Stronger evidence of construct validity was found for the older children, although the intercorrelation patterns for both age groups were characterized by larger correlations between assessments of real and ideal self-images using the same instrument than between assessments of the same component of the self-image across the three instruments. Contrary to predictions gener ated by developmental theory, the factor analyses provided non evidence of increasing differentiation of the self-image with age. They did, however, reveal that second and fifth graders distinguish distinctly different functional domains within their self-images.     

Convergent validity of Rorschach and TAT scales of object relations.

The Rorschach Mutuality of Autonomy Scale (MOA) and the Social Cognition and Object Relations Scale (SCORS) have been shown to be reliable and valid measures of interpersonal functioning. Utilizing a sample of 57 outpatients with a Diagnostic and Statistical Manual of Mental Disorders Axis II diagnosis, this study extends the findings of previous research demonstrating the reliability and convergent validity of each measure. Analyses focused on the convergent validity between the Rorschach MOA Scale and 8 SCORS variables (complexity, affect, emotional investment in relationships, emotional investment in values and morals, understanding of social causality, management of impulses/aggression, self-esteem, identity/coherence of self) ratings of Thematic Apperception Test narratives. The conceptual nature and clinical utility of these findings are discussed in relation to psychological assessment.     

The new MAACL scales with adolescents: preliminary reliability and validity determinations

This study reports on the reliability and concurrent validity of both the trait and state forms of the Multiple Affect Adjective Check List (MAACL) with a high school population, age range of 14-16 years and educational range of 9th and 10th grades (N = 403). Alphas of the four scales (anxiety, depression, hostility, and positive affect) ranged from .67 to .91. Correlations with a check-list of stress-related symptoms and an inventory of the number of extracurricular activities in which the students participated were significant and in the expected direction. Means and standard deviations on the MAACL scales were similar to those of college students. Sex differences were found on some of the scales. The findings indicate that the MAACL is sufficiently reliable and valid to warrant additional use with an adolescent population.     

A cross-sectional evaluation of the factorial invariance of anxiety sensitivity in adolescents and young adults

This study examined the cross-sectional factorial invariance of anxiety sensitivity in an ethnically diverse sample of adolescents (n = 173; mean age 15.5 years) and young adults (n = 291; mean age 20.1 years). Research in adult and youth samples suggests that anxiety sensitivity is best understood as a hierarchical construct with several lower-order factors. Factor models based on previous research using both adult and youth samples were compared and a hierarchical model with three lower-order factors provided the best fit to the data. Results supported the hypothesis that the factor structure of the Anxiety Sensitivity Index was invariant across age and gender. The factor scores also demonstrated differential correlations with symptoms of anxiety and depression. Results are discussed with regard to construct validation and understanding the structure of anxiety sensitivity in youth.     

Health and safety at work in the transport industry (TRANS-18): factorial structure, reliability and validity

In this article, we study the psychometric properties of a short scale (TRANS-18) which was designed to detect safe behaviors (personal and vehicle-related) and psychophysiological disorders. 244 drivers participated in the study, including drivers of freight transport vehicles (regular, dangerous and special), cranes, and passenger transport (regular transport and chartered coaches), ambulances and taxis. After carrying out an exploratory factor analysis of the scale, the findings show a structure comprised of three factors related to psychophysiological disorders, and to both personal and vehicle-related safety behaviors. Furthermore, these three factors had adequate reliability and all three also showed validity with regard to burnout, fatigue and job tension. In short, this scale may be ideally suited for adequately identifying the safety behaviors and safety problems of transport drivers. Future research could use the TRANS-18 as a screening tool in combination with other instruments.     

Achievement Emotions Questionnaire-Mathematics (AEQ-M) in adolescents: Factorial structure,measurement invariance and convergent validity with personality

ABSTRACT

The Achievement Emotions Questionnaire-Mathematics (AEQ-M) is a self-report measure of emotions experienced in class, when self-studying, and during tests for the domain of mathematics. Our aim was to present a Portuguese version of this instrument for use with adolescents and to test its reliability, factorial structure, measurement invariance, and construct validity with personality dimensions. Our sample comprised 1,387 Portuguese students from the 7th, 8th and 9th grades (mean age = 13.2 years). Student responses to the AEQ-M were found to be reliable. Confirmatory factor analysis validated a seven-emotion × three-setting factorial structure. This model demonstrated measurement invariance across gender and grade. As a demonstration of construct validity, the emotions measured by the AEQ-M showed a pattern of associations with psychobiological personality dimensions that were in line with theoretical predictions. These results validate the AEQ-M as a suitable instrument for assessing adolescents’ mathematics-related achievement emotions and their associations with personality.     

The Flow State Scale-2 and Dispositional Flow Scale-2: Examination of factorial validity and reliability for Japanese adults

ObjectiveThe primary purpose of this study was to examine the factorial validity and reliability of the Japanese versions of the Flow State Scale-2 and Dispositional Flow Scale-2 (JFSS-2 and JDFS-2) for use with Japanese adults.MethodTo accomplish the aim, a multi-staged approach was employed. Following the guidelines for test adaptation [Tanzer, N. K., & Sim, C. Q. E. (1999). Adapting instruments for use in multiple languages and cultures: A review of the ITC guidelines for test adaptations. European Journal of Psychological Assessment, 15, 258–269], the two flow scales were translated from English to Japanese and the best 36 items for each instrument was identified through two pilot studies. Employing a 9-factor 1st-order hypothesized model, the factorial validity of the JFSS-2 and JDFS-2 was tested and cross-validated with confirmatory factor analysis (CFA). In addition to internal consistency reliability for the both scales, stability was assessed over a 4-week time period for the JDFS-2. Furthermore, measurement equivalence was examined across Japanese independent samples as well as two cultural samples.ResultsThe results of a series of CFAs revealed that the data for the JFSS-2 and JDFS-2 were represented appropriately by the hypothesized 1st-order model. For the both scales, internal consistency estimates for all factors were satisfactory, whereas the stability of single factors over time were medium to high. Measurement invariance was established across the Japanese samples as well as the cultural samples.ConclusionsThe findings from this study provided strong support for the validity and reliability of the JFSS-2 and JDFS-2 in assessing flow experiences in physical activity for Japanese adults. In addition, this study indicated that the Japanese versions of the flow scales are useful instruments for cross-cultural research.     

The Adolescent Activities Checklist: Reliability,standardization data,and factorial validity

This study was conducted to provide standardization data and information on the reliability and factorial validity of the recently developed Adolescent Activities Checklist (AAC). A total of 563 adolescents in grades 7 through 12 served as subjects. Significant main effects for gender, race, and grade were obtained in a multivariate analysis of variance. On the basis of this information, standardization data were established for these three variables. Further investigation indicated that the internal consistency of the AAC was high. In addition, results of a principal components analysis conducted on the frequencies of the Unpleasant and Pleasant Activities subscales revealed four and three factors, respectively. For unpleasant activities, the major dimensions were found to occur in three situations-namely, social interactions, family situations, and school situations. Stressful events also occurred as one of the four unpleasant activities dimensions. For pleasant activities, three dimensions appeared: heterosocial behavior, reinforcing interpersonal situations, and social reinforcement.