Detecting differential item functioning with confirmatory factor analysis and item response theory: toward a unified strategy

In this article, the authors developed a common strategy for identifying differential item functioning (DIF) items that can be implemented in both the mean and covariance structures method (MACS) and item response theory (IRT). They proposed examining the loadings (discrimination) and the intercept (location) parameters simultaneously using the likelihood ratio test with a free-baseline model and Bonferroni corrected critical p values. They compared the relative efficacy of this approach with alternative implementations for various types and amounts of DIF, sample sizes, numbers of response categories, and amounts of impact (latent mean differences). Results indicated that the proposed strategy was considerably more effective than an alternative approach involving a constrained-baseline model. Both MACS and IRT performed similarly well in the majority of experimental conditions. As expected, MACS performed slightly worse in dichotomous conditions but better than IRT in polytomous cases where sample sizes were small. Also, contrary to popular belief, MACS performed well in conditions where DIF was simulated on item thresholds (item means), and its accuracy was not affected by impact.     

Differential item functioning by sex and race in the Hogan Personality Inventory

The authors examined measurement bias in the Hogan Personality Inventory by investigating differential item functioning (DIF) across sex and two racial groups (Caucasian and Black). The sample consisted of 1,579 Caucasians (1,023 men, 556 women) and 523 Blacks (321 men, 202 women) who were applying for entry-level, unskilled jobs in factories. Although the group mean differences were trivial, more than a third of the items showed DIF by sex (38.4%) and by race (37.3%). A content analysis of potentially biased items indicated that the themes of items displaying DIF were slightly more cohesive for sex than for race. The authors discuss possible explanations for differing clustering tendencies of items displaying DIF and some practical and theoretical implications of DIF in the development and interpretation of personality inventories.     

Item analysis and differential item functioning of a brief conduct problem screen

Research has shown that boys display higher levels of childhood conduct problems than girls, and Black children display higher levels than White children, but few studies have tested for scalar equivalence of conduct problems across gender and race. The authors conducted a 2-parameter item response theory (IRT) model to examine item characteristics of the Authority Acceptance scale from the Teacher Observation of Classroom Adaptation-Revised (AA-TOCA-R; L. Larsson-Werthamer, S. G. Kellam, & L. Wheeler, 1991) in 8,820 kindergarten children and estimated the degree of differential item functioning (DIF) by gender and race/urban status. The mean level of latent conduct problems was best represented by behaviors such as being stubborn, breaking rules, and being disobedient, whereas breaking things and taking others' property best represented the construct at one standard deviation above the mean. DIF by gender was detected, such that at equivalent levels of latent conduct problems, males received more endorsements of overt behaviors from teachers, whereas females received more endorsements of nonphysical behaviors. Moreover, overt behaviors were better discriminators of latent conduct problems for males, and nonphysical behaviors were better discriminators of latent conduct problems for females. Differences across race/urban status were not found to be conceptually meaningful. The authors' analyses also suggest that the item scaling of the AA-TOCA-R may be best represented by 5e categories instead of 6. These findings provide support for the use of IRT modeling to examine item characteristics of conduct problem scales and DIF to test for scalar equivalence across diverse subpopulations.     

Differential functioning of the Beck depression inventory in late-life patients: use of item response theory

The present analyses examined age-related measurement bias in responses to items on the revised Beck Depression Inventory (BDI) in depressed late-life patients versus midlife patients. Item response theory (IRT) models were used to equate the scale and to differentiate true-group differences from bias in measurement in the 2 samples. Baseline BDI data (218 late life and 613 midlife) were used for the present analysis. IRT results indicated that late-life patients tended to report fewer cognitive symptoms, especially at low to average levels of depression. Conversely, they tended to report more somatic symptoms, especially at higher levels of depression. Adjusted cutoff scores in the late-life group are provided, and possible reasons for age-related differences in the performance of the BDI are discussed.     

The mapping of Spanish social psychology through its conferences: a bibliometric perspective

This study of papers gathered from the proceedings presented at Spanish social psychology conferences explores the use of bibliometrics for studying scientific disciplines. A reference database of all the papers included in the conference proceedings of events held from 1983 to 2000 was generated and classified by thematic area, paper type and author institutional affiliation. The references were laid out on contingency tables and mapped with correspondence analysis. The results show that there is a growing number of co-authored papers and a predominance of empirical over theoretical paper types. Some institutions have a higher concentration of theoretical papers while others work mostly in the areas of organizational and health psychology. In terms of empirical papers, there is a tendency towards generating more qualitative-based studies over the span of time captured by this work. There are also a number of papers written about such areas as cultural psychology that points to the emergence of an interest in critical social psychology. Concluding remarks underline the role of conferences and scientific meetings as an important indicator of the dynamic development of a scientific discipline.     

Gender bias and construct validity in vocational interest measurement: Differential item functioning in the Strong Interest Inventory

Item response theory was used to address gender bias in interest measurement. Differential item functioning (DIF) technique, SIBTEST and DIMTEST for dimensionality, were applied to the items of the six General Occupational Theme (GOT) and 25 Basic Interest (BI) scales in the Strong Interest Inventory. A sample of 1860 women and 1105 men was used. The scales were not unidimensional and contain both primary and minor dimensions. Gender-related DIF was detected in two-thirds of the items. Item type (i.e., occupations, activities, school subjects, types of people) did not differ in DIF. A sex-type dimension was found to influence the responses of men and women differently. When the biased items were removed from the GOT scales, gender differences favoring men were reduced in the R and I scales but gender differences favoring women remained in the A and S scales. Implications for the development, validation and use of interest measures are discussed.     

Differential item functioning on the Five Facet Mindfulness Questionnaire is minimal in demographically matched meditators and nonmeditators

A recent study of the Five Facet Mindfulness Questionnaire reported high levels of differential item functioning (DIF) for 18 of its 39 items in meditating and nonmeditating samples that were not demographically matched. In particular, meditators were more likely to endorse positively worded items whereas nonmeditators were more likely to deny negatively worded (reverse-scored) items. The present study replicated these analyses in demographically matched samples of meditators and nonmeditators (n = 115 each) and found that evidence for DIF was minimal. There was little or no evidence for differential relationships between positively and negatively worded items for meditators and nonmeditators. Findings suggest that DIF based on items' scoring direction is not problematic when the Five Facet Mindfulness Questionnaire is used to compare demographically similar meditators and nonmeditators.     

Differential item functioning in an international 360-degree assessment: Evidence of gender stereotype,environmental complexity,and organizational contingency

This research used logistic regression to model item responses from a popular 360-degree-for-development survey used in a leadership development programme given to middle and upper level European managers in Brussels. The survey contained 106 items on 16 scales. The model used gender of ratee and rater group to identify items that exhibited differential item functioning (DIF). The rater groups were self, boss, peer, and direct report. The sample consisted of 356 survey families where a survey family consisted of a matched set of four surveys: one self, one boss, one peer, and one direct report. The sample contained 88% male and 12% female raters. The sample contained 1424 total surveys. The procedure for flagging items exhibiting differential functioning used effect size computed from Wald chi-square statistics rather than statistical significance, resulting in fewer flagged items. One item exhibited rating anomalies due to the gender of the ratee; 55 items exhibited DIF attributable to rater group. The apparent effect of the DIF was small with each item. An examination of the maximum likelihood parameter estimates suggested the rater group DIF was the result of either hierarchical complexity or organizational contingency. The DIF due to gender conformed to prior expectations of gender-related stereotypical interpretations. This research further suggested that DIF due to environmental complexity or organizational contingency could be a naturally occurring phenomenon in some 360-degree assessment, and that the interpretation of some 360-degree feedback could need to include the potential for such DIF to exist.     

Cognitive functioning in delusions: a longitudinal analysis

BACKGROUND: This study explored the longitudinal course of the relationship between delusions and different aspects of cognitive functioning. METHODS: Deluded patients were compared to psychiatric and non-clinical controls on three tasks: negative priming, a probabilistic judgement task (the 'beads' task), and the pragmatic inference task (PIT). All groups were tested at two time points, once when actively symptomatic, and once when in remission. RESULTS: Deluded individuals exhibited a 'jump-to-conclusions' (JTC) reasoning bias: i.e., they made decisions on the basis of limited evidence and were more likely to revise their estimates when faced with disconfirmatory evidence. This JTC bias remained stable over time, although probability judgments seemed to normalise in remission. No deficits in cognitive inhibition were found on negative priming. The deluded group displayed an excessive self-focus on the PIT at both time points, but did not show a depressive attributional style. Only a small sub-sample, characterised by the "bad-me" type of paranoia [Trower & Chadwick, 1995 Clinical Psychology: Science and Practice, 2, 263-278.], demonstrated depressive schemas when symptomatic, but no longer did so when remitted. Few relationships were found between tasks, suggesting that different areas of functioning are relatively independent. The only measures associated with delusion symptom scores were from the 'beads' task. CONCLUSIONS: Overall these findings suggest that the JTC bias is a stable factor associated with delusional thinking, while the depressive attributional style characteristic of a small sub-sample of paranoid patients fluctuates with delusional course.     

Gender-based differential item functioning in common measures of body dissatisfaction

Many widely used measures of body image were developed using all-female samples and thus may not adequately capture the male experience of body dissatisfaction. The current study examined differential item functioning (DIF) in three commonly-used measures of body image: The Body Shape Questionnaire (N = 590, 39.7% male), the Body Dissatisfaction subscale of the Eating Disorders Inventory (N = 529, 44.6% male), and the Shape and Weight Concern subscales of the Eating Disorders Examination Questionnaire (N = 1116, 43.5% male). Participants completed a series of measures evaluating body image and eating pathology. Results evidenced statistically significant DIF in several of the items; one item met criteria for clinically significant DIF. While most items did not evidence clinically elevated levels of DIF, additional evaluation is necessary in order to determine overall quality of the measures in terms of capturing the experience of male body image concerns.     

Invariance on a reading comprehension test in European Portuguese: A differential item functioning analysis between students from rural and urban areas

The aim of this study was to determine whether the items from a reading comprehension test in European Portuguese function differently across students from rural and urban areas, which biases the test validity and the equity in assessment. The sample was composed of 653 students from second, third and fourth grades. The presence of differential item functioning (DIF) was analysed using logistic regression and the Mantel–Haenszel procedure. Although 17 items were flagged with DIF, only five items showed non-negligible DIF in all effect-size measures. The evidence of invariance across students with rural or urban backgrounds for most of the items supports the validity of the test though the five identified items should be further investigated.     

Exploratory factor analysis: its role in item analysis

The special characteristics of items-low reliability, confounds by minor, unwanted covariance, and the likelihood of a general factor-and better understanding of factor analysis means that the default procedure of many statistical packages (Little Jiffy) is no longer adequate for exploratory item factor analysis. It produces too many factors and precludes a general factor even when that means the factors extracted are nonreplicable. More appropriate procedures that reduce these problems are presented, along with how to select the sample, sample size required, and how to select items for scales. Proposed scales can be evaluated by their correlations with the factors; a new procedure for doing so eliminates the biased values produced by correlating them with either total or factor scores. The role of exploratory factor analysis relative to cluster analysis and confirmatory factor analysis is noted.     

Differential effects of age on item and associative measures of memory: a meta-analysis

In this meta-analysis, the authors evaluated recent suggestions that older adults' episodic memory impairments are partially due to a reduced ability to encode and retrieve associated/bound units of information. Results of 90 studies of episodic memory for both item and associative information in 3,197 young and 3,192 older adults provided support for the age-related associative/binding deficit suggestion, indicating a larger effect of age on memory for associative information than for item information. Moderators assessed included the type of associations, encoding instructions, materials, and test format. Results indicated an age-related associative deficit in memory for source, context, temporal order, spatial location, and item pairings, in both verbal and nonverbal material. An age-related associative deficit was quite pronounced under intentional learning instructions but was not clearly evident under incidental learning instructions. Finally, test format was also found to moderate the associative deficit, with older adults showing an associative/binding deficit when item memory was evaluated via recognition tests but not when item memory was evaluated via recall tests, in which case the age-related deficits were similar for item and associative information.     

Index of norms and ratings published in the Psychonomic Society journals

The journals of the Psychonomic Society have served as outlets for numerous stimulus norms and ratings. Such norms are useful to researchers in a variety of areas for manipulating and controlling stimulus attributes. This article presents an index of 142 norms published in the Society’s journals, categorized according to the types of materials and ratings that are included in each.     

Statistical power of articles published in three health psychology-related journals.

Power was calculated for 8,266 statistical tests in 187 journal articles published in the 1997 volumes of Health Psychology (HP), Addictive Behaviors (AB), and the Journal of Studies on Alcohol (JSA). Power to detect small, medium, and large effects was .34. .74. and .92 for HP; .34, .75, and .90 for AB; and .41, .81. and .92 for JSA. Mean power estimates are .36, .77, and .91, giving a good estimation for the field of health psychology. J. Cohen (1988) recommended that power to detect effects should be approximately .80. Using this criterion, the articles in these journals have adequate power to detect medium and large effects. Intervention studies have much less power to detect effects than nonintervention studies do. Results are encouraging for this field, although studies examining small effects are still very much underpowered. This issue is important, because most intervention effects in health psychology are small.     

Predicting black and white differential item functioning in verbal analogy performance

Differential item functioning (DIF) is one technique for comparing ethnic populations that test makers employ to help ensure the fairness of their tests. The purpose of this ethnic comparison study is to investigate factors that may have a significant influence on DIF values associated with 217 SAT and 234 GRE analogy items obtained by comparing large samples of Black and White examinees matched for total verbal score. In one study, five significant regression predictors of ethnic differences were found to account for approximately 30% of the DIF variance. A second study replicated these findings. These significant ethnic comparisons are interpreted as consistent with a cultural/contextualist framework although competing explanations involving social-economic status and biological contributions could not be ruled out. Practical implications are discussed.     

Multiple victimization of Spanish adolescents: a multilevel analysis

Multiple victimization in adolescence is an issue that has received little research attention. Furthermore, adolescents are particularly vulnerable to victimization in different contexts. The aim of this study is to analyze correlates of multiple victimization in three contexts (home, school, and street). The following forms of victimization were considered: stealing, hitting, insulting, threatening, blackmailing, and weapon intimidation. Multiple victimization correlates explored were: sex, age, public/private school, socioeconomic status, quality of family relationships, and antisocial behavior. A probabilistic sample of 1,908 adolescents (ages 13 to 18) was used. Multilevel analyses were conducted to separate correlates at the individual level from those operating at the contextual level. Results show that gender, quality of family relationships, and deviant behavior were related to multiple victimization in adolescence.