Group Differences in Regression Intercepts: Implications for Factorial Invariance

Studies of differential prediction typically examine group differences in linear regression slopes or intercepts for predicting criterion scores from one or more test scores. When there are no group differences in slopes, what are the implications of differences in regression intercepts for the measurement equivalence of the tests or criterion across groups? Measurement equivalence is here defined as factorial invariance under a single-factor model for the tests and criterion. Two theorems are given that describe conditions under which intercept differences can exist under factorial invariance. In such cases, intercept differences do not result from measurement bias in either the tests or criterion. The conditions of the theorems are testable using multiple-group confirmatory factor analysis. These test procedures are illustrated in real data. The implications of the theorems and the test procedures for studies of differential prediction are discussed.     

TESTING FOR FAIRNESS WITH A MODERATED MULTIPLE REGRESSION STRATEGY: AN ALTERNATIVE TO DIFFERENTIAL ANALYSIS1

It is argued that analyses of subgroup differences utilizing a bivariate correlation strategy do not provide an adequate examination of test fairness. An analysis of differential prediction, which involves slopes and intercepts of regression lines results in more complete coverage of the test fairness issue, since the overall regression line determines the way in which a test is used for prediction. While subgroup correlation coefficients yield information concerning the slopes and intercepts, means and standard deviations must also be examined. A moderated multiple regression strategy is recommended as an alternative to separate analyses by subgroups. An ordered step-up regression procedure is presented which is more encompassing than the bivariate strategies, while avoiding inherent problems associated with subgroup coding in multiple regression.     

Differential impairment as an indicator of sex bias in DSM-IV criteria for four personality disorders

The aim of the present study was to investigate the possibility of sex bias in the diagnostic criteria for borderline, schizotypal, avoidant, and obsessive-compulsive personality disorders. A clinical sample of 668 individuals was evaluated for personality disorder criteria using a semistructured interview, and areas of functional impairment were assessed with both self-report and semistructured interview. The authors used a regression model of bias to identify bias as differences in slopes or intercepts between men and women in the relationship between each diagnostic criterion and level of impairment. The results suggest that most of the diagnostic criteria examined do not seem to display sex bias. However, those criteria that displayed evidence of bias came largely from the borderline diagnosis.     

The Traditional Regression Model for Measuring Test Bias is Incorrect and Biased Against Minorities

For 25 years psychologists have measured systematic measurement bias in terms of regression lines. According to this traditional approach a test is an unbiased predictor of a criterion for all subgroups if all subgroups have identical Y regression lines (i.e., identical slopes and identical Y intercepts). This paper shows that the traditional model is fundamentally incorrect and identical Y regression lines are not expected to occur with an unbiased test in a testing situation in which one group score lower than another group on both the test and criterion. This is the case even if the test is perfectly reliable. The traditional model for measuring bias actually results in a consistent error or bias against groups which score lower than average on both the test and criterion. In practice this bias operates against minority groups. Tests now thought to be unbiased or even biased in favor of minority groups may in fact be biased against minority groups. A new model of test bias, which is based solely on measurement principles, is briefly introduced. In this model unbiased tests produce groups with identical test-criterion common-factor axes having a slope of S _YC/S _XC and with each axis intersecting the group centroids.     

MacPotthoff: automated calculation of the Potthoff regression bias procedure

Test bias, in contrast to test fairness, is best conceptualized in validity terms amenable to statistical analysis. Evidence of predictive validity may be most salient in many situations. Evaluation of predictive bias is generally operationalized via linear regression. Potthoff (1978) provided an efficient and parsimonious regression bias procedure that allows both simultaneous and separate tests of regression slopes and intercepts across groups. A Macintosh computer program, MacPotthoff, is presented for automated calculation of Potthoff regression bias statistics.     

The difference engine: A model of diversity in speeded cognition

A theory of diversity in speeded cognition, the difference engine, is proposed, in which information processing is represented as a series of generic computational steps. Some individuals tend to perform all of these computations relatively quickly and other individuals tend to perform them all relatively slowly, reflecting the existence of a general cognitive speed factor, but the time required for response selection and execution is assumed to be independent of cognitive speed. The difference engine correctly predicts the positively accelerated form of the relation between diversity of performance, as measured by the standard deviation for the group, and task difficulty, as indexed by the mean response time (RT) for the group. In addition, the difference engine correctly predicts approximately linear relations between the RTs of any individual and average performance for the group, with the regression lines for fast individuals having slopes less than 1.0 (and positive intercepts) and the regression lines for slow individuals having slopes greater than 1.0 (and negative intercepts). Similar predictions are made for comparisons of slow, average, and fast subgroups, regardless of whether those subgroups are formed on the basis of differences in ability, age, or health status. These predictions are consistent with evidence from studies of healthy young and older adults as well as from studies of depressed and age-matched control groups.     

An evaluation of differential prediction of WIAT achievement scores from WISC-III FSIQ across ethnic and gender groups

This study examines differential prediction of WIAT achievement scores based on WISC-III FSIQ in white as compared with African American and Hispanic children, and in females as compared with males. A procedure which allows simultaneous comparisons of slopes and intercepts across groups is employed. The results are consistent with previous research findings in supporting the general absence of bias in predicting achievement from IQ.     

The homogeneity assumption in differential prediction analysis: does it really matter?

In simulation studies, the F test for differences in regression slopes has tended to distort nominal Type I and II error rates when the 2 subgroup error variances exceeded a 1.50:1 ratio. This study examines the frequency and extent that this ratio is violated within data sets relevant to applied psychology. The General Aptitude Test Battery (GATB) validity study database contained ability data and overall job performance ratings. The Project A military database contained both ability and personality data, along with job performance factor scores and an overall job performance rating. Results suggest that subgroup (White-Black, male-female) error variances are often homogeneous enough to support F test results from past empirical work. Enough heterogeneity was found, however, to urge applied psychologists investigating differential prediction to explore their data and consider the possibility of alternative statistical tests.     

Higher-order structure in the trajectories of depression and anxiety following sudden involuntary unemployment

Recent work on comorbidity finds evidence for hierarchical structure of mood and anxiety disorders and symptoms. This study tests whether a higher-order internalizing factor accounts for variation in depression and anxiety symptom severity and change over time in a sample experiencing a period of major life stress. Data on symptoms of depression, chronic worry, and social anxiety were collected five times across seven months from 426 individuals who had recently lost jobs. Growth models for each type of symptom found significant variation in individual trajectories. Slopes were highly correlated across symptom type, as were intercepts. Multilevel confirmatory factor analyses found evidence for a higher-order internalizing factor for both slopes and intercepts, reflective of comorbidity of depression and anxiety, with the internalizing factor accounting for 54% to 91% of the variance in slopes and intercepts of specific symptom sets, providing evidence for both a general common factor and domain-specific factors characterizing level and change in symptoms. Loadings on the higher order factors differed modestly for men and women, and when comparing African American and White participants, but did not differ by age, education, or history of depression. More distal factors including gender and history of depression were strongly associated with internalizing in the early weeks after job loss, but rates of change in internalizing were associated most strongly with reemployment. Findings suggest that stressors may contribute in different ways to the common internalizing factor as compared to variance in anxiety and depression that is independent of that factor.     

大学生人际宽恕发展趋势及其反刍思维的作用

目的探讨大学生人际宽恕(回避、报复和仁慈)的增长模型,以及反刍思维在人际宽恕及其发展过程中的影响作用。方法采用人际侵犯动机量表对在"一周内"遭受过冒犯行为的116名大学生的人际宽恕过程进行"每周一测"的5次跟踪测评,并使用反刍思维反应量表评估被试的反刍思维。结果 1回避和报复动机符合线性增长模型,而仁慈动机更符合曲线增长模型;2在控制了冒犯严重性、关系亲密性因素后,反刍思维对大学生人际宽恕各维度及其变化趋势具有显著预测作用。结论研究结果揭示了将人际宽恕各维度区分对待的重要性,同时说明反刍思维对大学生人际宽恕的适应性心理过程具有阻碍作用。     

A meta-analytic approach to growth curve analysis

A meta-analytic approach to growth curve analysis is described and illustrated by applying it to the evaluation of the Arizona Pilot Project, an experimental project for financing the treatment of the severely mentally ill. In this approach to longitudinal data analysis, each individual subject for which repeated measures are obtained is initially treated as a separate case study for analysis. This approach has at least two distinct advantages. First, it does not assume a balanced design (equal numbers of repeated observations) across all subjects; to accommodate a variable number of observations for each subject, individual growth curve parameters are differentially weighted by the number of repeated measures on which they are based. Second, it does not assume homogeneity of treatment effects (equal slopes) across all subjects. Individual differences in growth curve parameters representing potentially unequal developmental rates through time are explicitly modeled. A meta-analytic approach to growth curve analysis may be the optimal analytical strategy for longitudinal studies where either (1) a balanced design is not feasible or (2) an assumption of homogeneity of treatment effects across all individuals is theoretically indefensible. In our evaluation of the Arizona Pilot Project, individual growth curve parameters were obtained for each of the 13 rationally derived subscales of the New York Functional Assessment Survey, over time, by linear regression analysis. The slopes, intercepts, and residuals obtained for each individual were then subjected to meta-analytic causal modeling. Using factor analytic models and then general linear models for the latent constructs, the growth curve parameters of all individuals were systematically related to each other via common factors and predicted based on hypothesized exogenous causal factors. The same two highly correlated common factors were found for all three growth curve parameters analyzed, a general psychological factor and a general functional factor. The factor patterns were found to be nearly identical across the separate analyses of individual intercepts, slopes, and residuals. Direct effects on the unique factors of each subscale of the New York Functional Assessment Survey were tested for each growth curve parameter by including the common factors as hierarchically prior predictors in the structural model for each of the indicator variables, thus statistically controlling for any indirect effect produced on the indicator through the common factors. The exogenous predictors modeled were theoretically specified orthogonal contrasts for Method of Payment (comparing Arizona Pilot Project treatment or "capitation" to traditional or "fee-for-service" care as a control), Treatment Administration Site (comparing various locations within treatment or control groups), Pretreatment Assessment (comparing general functional level at intake as assigned by an Outside Assessment Team), and various interactions among these main effects. The intercepts, representing the initial status of individual subjects on both the two common factors and the 13 unique factors of the subscales of the New York Functional Assessment Survey, were found to vary significantly across many of the various different treatment conditions, treatment administration sites, and pretreatment functional levels. This indicated a severe threat to the validity of the originally intended design of the Arizona Pilot Project as a randomized experiment. When the systematic variations were statistically controlled by including intercepts as hierarchically prior predictors in the structural models for slopes, recasting the experiment as a nonequivalent groups design, the effects of the intercepts on the slopes were found to be both statistically significant and substantial in magnitude. (ABSTRACT TRUNCATED)     

Group Differences on US Air Force Pilot Selection Tests

Sex and ethnic group differences were examined on the operational composites and tests used to select applicants for US Air Force officer commissioning programmes and for pilot training. Results showed that large mean score differences in applicant samples were substantially reduced among the pilot trainees. Despite differences in test performance, there was no evidence of differential validity for groups. When group differences in predicted pilot training completion rate were observed, performance was overestimated for the minority group relative to the majority group. When regression equations were adjusted for unreliability of the predictors, the observed differences in intercepts were reduced or eliminated. No prediction bias was observed against the minority groups.     

Information processing abilities and performance on two perceptual-motor tasks

The purpose of this study was to investigate whether the hypothesized information-processing mechanisms, thought to be reflected by the slope and intercept of the regression of choice reaction time on stimulus information or information transmitted could account for performance on two novel laboratory tasks. In addition, analyses were performed to determine whether these theorized mechanisms represent stable within individual abilities or task specific capabilities. Analysis indicated that the slopes obtained from two different choice reaction time paradigms were unrelated while intercepts were moderately correlated. Examination of the pattern of zero-order correlations further showed that regression components were not meaningfully related to performance on either novel task, suggesting that knowledge of information-processing mechanisms, as assessed in choice RT paradigms, did not enhance the understanding of performance in more complex tasks.     

Moderated multiple regression for interactions involving categorical variables: a statistical control for heterogeneous variance across two groups

Moderated multiple regression (MMR) arguably is the most popular statistical technique for investigating regression slope differences (interactions) across groups (e.g., aptitude-treatment interactions in training and differential test score-job performance prediction in selection testing). However, heterogeneous error variances can greatly bias the typical MMR analysis, and the conditions that cause heterogeneity are not uncommon. Statistical corrections that have been developed require special calculations and are not conducive to follow-up analyses that describe an interaction effect in depth. A weighted least squares (WLS) approach is recommended for 2-group studies. For 2-group studies, WLS is statistically accurate, is readily executed through popular software packages (e.g., SAS Institute, 1999; SPSS, 1999), and allows follow-up tests.     

The ideologically objectionable premise model: Predicting biased political judgments on the left and right

This paper introduces the ideologically objectionable premise model (IOPM), which predicts that biased political judgments will emerge on both the political left and right, but only when the premise of a judgment is not ideologically objectionable to the perceiver. The IOPM generates three hypothesized patterns of bias: biases among both those on the left and right, bias only among those on the right, and bias only among those on the left. These hypotheses were tested within the context of the dual process motivational model of ideological attitudes (DPM; Duckitt, 2001), which posits that right-wing authoritarianism (RWA) and social dominance orientation (SDO) are related but distinct ideological attitudes. Across two studies, all three IOPM hypotheses were tested and supported on the RWA ideological attitude dimension, and two of the three IOPM hypotheses were tested and supported on the SDO dimension. These findings indicate that the context of the judgment is an important determinant of whether biases emerge in political judgment.     

The Moderating Effect of Sex on the Prediction of Job Satisfaction in the Public Sector

The author reports the results of his study examining whether sex moderates the relationship between general job satisfaction or dissatisfaction and the 25 job satisfaction factors of the Triple Audit Opinion Survey (TAOS). The author conducted Hierarchical Multiple Regression Covariance Analysis to test the homogeneity of regression (intercepts and slopes) between males and females using the five orthogonal factors derived from the 25 job satisfaction components of the TAOS as the independent variables and the general job satisfaction scores of the TAOS as the dependent variable.     

A checklist for testing measurement invariance

The analysis of measurement invariance of latent constructs is important in research across groups, or across time. By establishing whether factor loadings, intercepts and residual variances are equivalent in a factor model that measures a latent concept, we can assure that comparisons that are made on the latent variable are valid across groups or time. Establishing measurement invariance involves running a set of increasingly constrained structural equation models, and testing whether differences between these models are significant. This paper provides a step-by-step guide to analysing measurement invariance.     

Evaluation of evidence for sufficiency, for necessity, and for necessity-and-sufficiency

We report two experiments with 120 undergraduate subjects. The tasks presented clearly articulated hypotheses concerning necessity, sufficiency, and necessity-and-sufficiency, together with possible combinations of treatment conditions, and required subjects to judge whether, according to the hypotheses, the relevant outcomes will occur, will not occur, or might occur. The patterns of responses reveal how subjects understand the extensions of the concepts, i.e. what each hypothesis permits, requires, and excludes. Only necessity-and-sufficiency hypotheses generally led to logically adequate responses, and the most typical error for both necessity hypotheses and sufficiency hypotheses was to treat them as extensionally equivalent to necessity-and-sufficiency. This error tendency was more evident for necessity than for sufficiency hypotheses. Further, although responses to both necessity hypotheses and sufficiency hypotheses were affected by the complexity of the hypothesized conditions, responses to necessity-and-sufficiency hypotheses were not. We argue that the results are not a function of a response bias but reflect differences in the way the hypotheses are understood. Implications for the causal attribution literature are discussed.     

A comparison of methods for estimating quadratic effects in nonlinear structural equation models

Two Monte Carlo simulations were performed to compare methods for estimating and testing hypotheses of quadratic effects in latent variable regression models. The methods considered in the current study were (a) a 2-stage moderated regression approach using latent variable scores, (b) an unconstrained product indicator approach, (c) a latent moderated structural equation method, (d) a fully Bayesian approach, and (e) marginal maximum likelihood estimation. Of the 5 estimation methods, it was found that overall the methods based on maximum likelihood estimation and the Bayesian approach performed best in terms of bias, root-mean-square error, standard error ratios, power, and Type I error control, although key differences were observed. Similarities as well as disparities among methods are highlight and general recommendations articulated. As a point of comparison, all 5 approaches were fit to a reparameterized version of the latent quadratic model to educational reading data.     

Posttraumatic stress disorder's frequency and intensity ratings are associated with factor structure differences in military veterans

We examined possible differences in the factor structure of posttraumatic stress disorder (PTSD) on the basis of whether frequency or intensity symptom response formats were used to assess PTSD. Participants included 669 veterans recruited from an epidemiological study of four Veterans Affairs Medical Centers' primary care clinics in the southeastern United States. Confirmatory factor analysis using measurement invariance testing found that the frequency and intensity symptom formats were significantly different from each other on PTSD's factor structure parameters, including factor loadings, observed variable intercepts, and measurement errors. The only exception was for PTSD's effortful avoidance symptoms, which were associated with equivalent parameter estimates for both the frequency and intensity formats. Implications for the clinical assessment of PTSD and interpretation of the extant literature base on PTSD's factor structure are considered.