Ethnic group differences in measures of job performance: a new meta-analysis

The authors conducted a new meta-analysis of ethnic group differences in job performance. Given a substantially increased set of data as compared with earlier analyses, the authors were able to conduct analyses of Black-White differences within more homogeneous categories of job performance and to reexamine findings on objective versus subjective measurement. Contrary to one perspective sometimes adopted in the field, objective measures are associated with very similar, if not somewhat larger, standardized ethnic group differences (ds) than subjective measures across a variety of indicators. This trend was consistent across quality, quantity, and absenteeism measures. Further, work samples and job knowledge tests are associated with larger ds than performance ratings or measures of absenteeism. Analysis of Hispanic-White standardized differences shows that they are generally lower than Black-White differences in several categories.     

Assessment and Intervention with Black-White Multiracial Couples

SUMMARY

Effective therapy with Black-White multiracial couples begins with therapist work on self attitudes and with sensitive assessment of the ways racism and race may or may not be connected to a couple's problems. We suggest ways to begin to explore self attitudes and suggest basic assessment questions for work with Black-White couples. In addition, we point out important clinical issues that may come up for Black-White couples in the areas of social support/social network, parenting, and grief.     

College grade point average as a personnel selection device: ethnic group differences and potential adverse impact

College grade point average (GPA) is often used in a variety of ways in personnel selection. Unfortunately, there is little empirical research literature in human resource management that informs researchers or practitioners about the magnitude of ethnic group differences and any potential adverse impact implications when using cumulative GPA for selection. Data from a medium-sized university in the Southeast (N = 7,498) indicate that the standardized average Black-White difference for cumulative GPA in the senior year is d = 0.78. The authors also conducted analyses at 3 GPA screens (3.00, 3.25, and 3.50) to demonstrate that employers (or educators) might face adverse impact at all 3 levels if GPA continues to be implemented as part of a selection system. Implications and future research are discussed.     

Work Sample Selection Tests and Expected Reduction in Adverse Impact: A Cautionary Note

Almost all book chapters, review articles, and textbooks in the field of personnel selection suggest that work sample tests are associated with lower levels of ethnic group adverse impact than paper‐and‐pencil tests of cognitive ability. However, the empirical literature is heavily dependent on adverse impact estimates obtained from incumbent samples rather than applicant samples. As such, parameter estimates are subject to range restriction from prior selection and on‐the‐job experiences. Further, an emerging consensus in the selection literature indicates that any method of assessment can be associated with high or low levels of adverse impact – depending on the nature of the construct(s) being measured. To begin to examine these issues, we present two recent sets of applicant data from public sector jobs (for a management and entry‐level job, each with technical and interpersonal skill requirements) and show that adverse impact of work sample exams might be more extensive than realized. We discuss the mismatch between what the field of employee selection “knows” and what is said in articles/summaries about work samples. Employers and other practitioners who depend on advice in academic overview articles may be overly optimistic and eventually disappointed by minimal reduction in adverse impact. Implications for workforce diversity and future research needs are also discussed.     

EXAMINING SELECTION UTILITY WHERE COMPETING PREDICTORS DIFFER IN ADVERSE IMPACT

This paper discusses the roles of validity, cut score choice, and adverse impact on selection system utility using data from two concurrent validation studies. We contrast an assessment center and published aptitude test on several metrics, including validity, testing costs, adverse impact, and utility. The assessment center produced slightly lower validity than the aptitude test while costing roughly 10 times as much per candidate. In spite of these advantages for the aptitude test, the assessment center produced so much less adverse impact its operational utility would be higher given cut scores likely to be chosen in this organization. Potential concerns with applying net utility models to this type of situation are discussed in comparison to gross utility models.     

Evidence for hypodescent and racial hierarchy in the categorization and perception of biracial individuals

Individuals who qualify equally for membership in two racial groups provide a rare window into social categorization and perception. In 5 experiments, we tested the extent to which a rule of hypodescent, whereby biracial individuals are assigned the status of their socially subordinate parent group, would govern perceptions of Asian-White and Black-White targets. In Experiment 1, in spite of posing explicit questions concerning Asian-White and Black-White targets, hypodescent was observed in both cases and more strongly in Black-White social categorization. Experiments 2A and 2B used a speeded response task and again revealed evidence of hypodescent in both cases, as well as a stronger effect in the Black-White target condition. In Experiments 3A and 3B, social perception was studied with a face-morphing task. Participants required a face to be lower in proportion minority to be perceived as minority than in proportion White to be perceived as White. Again, the threshold for being perceived as White was higher for Black-White than for Asian-White targets. An independent categorization task in Experiment 3B further confirmed the rule of hypodescent and variation in it that reflected the current racial hierarchy in the United States. These results documenting biases in the social categorization and perception of biracials have implications for resistance to change in the American racial hierarchy.     

A META-ANALYSIS OF THE CRITERION-RELATED VALIDITY OF ASSESSMENT CENTER DIMENSIONS

We used meta-analytic procedures to investigate the criterion-related validity of assessment center dimension ratings. By focusing on dimension-level information, we were able to assess the extent to which specific constructs account for the criterion-related validity of assessment centers. From a total of 34 articles that reported dimension-level validities, we collapsed 168 assessment center dimension labels into an overriding set of 6 dimensions: (a) consideration/awareness of others, (b) communication, (c) drive, (d) influencing others, (e) organizing and planning, and (f) problem solving. Based on this set of 6 dimensions, we extracted 258 independent data points. Results showed a range of estimated true criterion-related validities from .25 to .39. A regression-based composite consisting of 4 out of the 6 dimensions accounted for the criterion-related validity of assessment center ratings and explained more variance in performance (20%) than Gaugler, Rosenthal, Thornton, and Bentson (1987) were able to explain using the overall assessment center rating (14%).     

Parent-child relationships of adopted adolescents in a psychiatric hospital

This study compared the parent-child relationships of 140 adopted and non-adopted adolescents treated in a psychiatric hospital through examination of information contained in the adolescents' medical records. Specifically, comparisons were made of the mention of parental contribution to the problems precipitating hospitalization, psychiatric restriction of parental visits to hospitalized offspring, and referral of parents to an adjunct parents' group. Data were gathered by uninformed research assistants, from the hospital charts of adolescents who had already been discharged from the hospital. Statistical analysis of the data yielded the following results. Adoptive parents were significantly more frequently restricted in their visits to their children. In addition, they were also more likely to have been involved in the precipitants to hospitalization and to have been referred subsequently for adjunctive treatment. It was concluded that parent-child relations may be more problematic among hospitalized adopted, as compared with non-adopted , adolescents. It was also suggested that psychiatric bias concerning "typical" adoptive family dynamics might have contributed to the observed differences.     

Personality characteristics as a function of frequency and type of substance use

Problematic and unhealthy personality characteristics have typically been the focus of researchers interested in the possible causes of substance use/abuse. Recent research has begun to identify differences between individuals based upon frequency of use, with indications that moderate users may have "healthier" characteristics than either abstainers or regular users. The present study was conducted in a rural setting and compared personality structure (based on the California Psychological Inventory) among three groups of college students: abstainers, experimenters, and regular users. The comparisons were conducted as a function of all illicit substance use (excluding alcohol) and for alcohol use only. Minimal support was obtained for the above hypothesis when comparing the three groups on illicit substance use. When comparing the three groups on alcohol use only, support for the theory was identified, with fewer differences found between the experimenters and the abstainers than between the experimenters and the regular users. It is suggested that rural use of psychoactive substances may be more limited to alcohol due to easier availability and less stigma. Data supporting the "healthier" personality structure of moderate users of alcohol would thus be more likely in rural settings even though such use would still be "risky" behavior given its illegality for individuals under age 21. The need for adaptations to current treatment approaches is suggested.     

Characteristics of useful evaluation data: An example from community mental health centers

The results of an empirical study of the use of evaluation data in community mental health centers are reported. A mailed survey on evaluation use was conducted among the directors of 164 community mental health centers in 19 states; 140 completed questionnaires were returned. Results indicate that certain types of data have important impacts in a majority of centers. Systems resources management data were most highly used, followed by need assessment data, and client utilization data. Least used were data on outcomes of intervention and community impact. Data use appears closely tied to the utility of the data in carrying out priority management tasks in a center. Findings have important implications for community psychologists who plan, administer, or evaluate mental health services. The broader role of evaluation in community psychology is also discussed.     

ETHNIC GROUP DIFFERENCES IN COGNITIVE ABILITY IN EMPLOYMENT AND EDUCATIONAL SETTINGS: A META-ANALYSIS

The cognitive ability levels of different ethnic groups have interested psychologists for over a century. Many narrative reviews of the empirical literature in the area focus on the Black-White differences, and the reviews conclude that the mean difference in cognitive ability ( g ) is approximately 1 standard deviation; that is, the generally accepted effect size is about 1.0. We conduct a meta-analytic review that suggests that the one standard deviation effect size accurately summarizes Black-White differences for college application tests (e.g., SAT) and overall analyses of tests of g for job applicants in corporate settings. However, the 1 standard deviation summary of group differences fails to capture many of the complexities in estimating ethnic group differences in employment settings. For example, our results indicate that job complexity, the use of within job versus across job study design, focus on applicant versus incumbent samples, and the exact construct of interest are important moderators of standardized group differences. In many instances, standardized group differences are less than 1 standard deviation. We conduct similar analyses for Hispanics, when possible, and note that Hispanic-White differences are somewhat less than Black-White differences.     

Oxford cognitive screen: A critical review and independent psychometric evaluation

The Oxford cognitive screen (OCS) is a stroke-specific cognitive screening assessment. Although the test developers have provided psychometric information for the assessment, the OCS has received minimal external scrutiny, with which to triangulate the underpinning psychometrics. The purpose of this study is to provide a critical review and independent validation of the OCS. This study analysed data from an anonymised clinical database, which consisted of 316 patients who were assessed using the OCS on an Acute Stroke Unit. The rates of impairment on tests of memory and receptive communication were lower than expectation, suggesting that these subtests may be relatively insensitive. Patients with aphasia were more likely to be unable to categorised as impaired on non-language tests, suggesting that they remain sensitive to language processing or non-dominant hand usage. Some of the subtests of the OCS achieve high retest reliability, which makes them good candidates for measuring cognitive change over time. Although the OCS has many advantages, it is also important to adequately consider its limitations, that is insensitivity to memory problems, the potential confounding impact of non-dominant hand usage, and the potential that some tests may sample overall cognitive ability instead of domain-specific functioning.     

THE DIVERSITY–VALIDITY DILEMMA: STRATEGIES FOR REDUCING RACIOETHNIC AND SEX SUBGROUP DIFFERENCES AND ADVERSE IMPACT IN SELECTION

Pyburn, Ployhart, and Kravitz (this issue, 2008) introduced the diversity–validity dilemma: that some of the most valid predictors of job performance are also associated with large racioethnic and sex subgroup predictor score differences. This article examines 16 selection strategies hypothesized to minimize racioethnic and sex subgroup differences and adverse impact and, hence, balance diversity and validity. Rather than presenting a highly technical review, our purpose is to provide practitioners with a concise summary, paying particular attention to comparing and contrasting the effectiveness of the strategies and reporting new developments. The paper is organized around 4 key questions: (a) Which strategies are most effective for reducing subgroup differences? (b) Which strategies do not involve a validity tradeoff? (c) What are the major new developments in strategies for reducing adverse impact? (d) What are the major new developments in alternative predictor measurement methods (e.g., interviews, situational judgment tests, assessment centers) for reducing adverse impact? We then conclude with recommendations and caveats for how to best balance diversity and validity. These ideas are developed further in Kravitz (this issue, 2008) , who considers even broader approaches for solving the diversity–validity dilemma.     

Juror decision-making in a mock sexually violent predator trial: gender differences in the impact of divergent types of expert testimony

Despite widespread use of mental health testimony in cases where violence risk is at issue, relatively little is known about the impact of such information on juror decision-making. This study addressed the effects of testimony based on three types of risk assessment instrument or method (clinical opinion, actuarial assessment, and ratings of psychopathy) to examine whether they would have differential impact on jurors' perceptions of the defendant. In a mock sexually violent predator civil commitment trial, 172 undergraduates were presented a case summary that included prosecution and defense expert testimony related to violence risk based on one of the three methods noted above. Consistent with earlier research, the hypothesis that a defendant described as a "high risk psychopath" by the prosecution would be judged more severely than a defendant judged as "high risk" based on other evaluation procedures was supported, but only among female jurors. Unlike prior studies, little support was found for the hypothesis that clinical opinion testimony would be more influential than actuarially based testimony for either gender. Mechanisms that may underlie the observed gender differences are discussed, as are the potential implications of these findings for civil commitment proceedings.     

Relationship stigma and Black-White interracial marital satisfaction: the mediating role of religious/spiritual well-being

Although the percentage of Black-White interracial marriage continues to increase in the United States, research is limited related to the impact of stigma on these couples and the processes that influence their responses to negative racial experiences. In the current study, the Vulnerability-Stress-Adaptation model was used as a theoretical foundation to investigate the relationship between stigma and couple satisfaction among Black-White interracially married couples, along with the potential mediating role of religious/spiritual well-being. A community sample of 180 interracially married individuals responded to surveys assessing their experiences of stigma due to being a member of an interracial couple, religious/spiritual well-being, and couple satisfaction. The analysis revealed that relationship stigma was negatively associated with couple satisfaction, and religious/spiritual well-being mediated the relationship between these two variables. To conclude, recommendations for intervention and prevention in the lives of Black-White interracial couples are offered.     

Racial/ethnic differences in the criterion-related validity of cognitive ability tests: a qualitative and quantitative review

The correlation between cognitive ability test scores and performance was separately meta-analyzed for Asian, Black, Hispanic, and White racial/ethnic subgroups. Compared to the average White observed correlation ( = .33, N = 903,779), average correlations were lower for Black samples ( = .24, N = 112,194) and Hispanic samples ( = .30, N = 51,205) and approximately equal for Asian samples ( = .33, N = 80,705). Despite some moderating effects (e.g., type of performance criterion, decade of data collection, job complexity), validity favored White over Black and Hispanic test takers in almost all conditions that included a sizable number of studies. Black-White validity comparisons were possible both across and within the 3 broad domains that use cognitive ability tests for high-stakes selection and placement: civilian employment, educational admissions, and the military. The trend of lower Black validity was repeated in each domain; however, average Black-White validity differences were largest in military studies and smallest in educational and employment studies. Further investigation of the reasons for these validity differences is warranted.     

A further examination of the constructs underlying assessment center ratings

Previous factor analytic work (Sackett and Dreher, 1982) has found exercise factors, rather than the expected dimension factors, underlying ratings made in managerial assessment centers. The present paper responds to criticisms that inappropriate analyses were conducted and that unrepresentative assessment centers were studied. Exercise factors were found to predominate in two new sets of assessment data which we obtained and in three studies published by other authors, thus indicating that the results were not idiosyncratic to the three centers studied by Sackett and Dreher. Analyses using confirmatory, rather than exploratory, factor analysis also support the dominance of exercise factors. Although dimension factors were found in three of four data sets studied, variables loading on a dimension factor consistently loaded more highly on an exercise factor as well.     

Personalized and generalized comparisons: causes and consequences of variations in the focus of social comparisons

When people compare with another person they can focus on how they compare either with just that target (a personalized comparison) or with others more generally (a generalized comparison). Four studies (two event-contingent diary studies, one study of comparisons during a triathlon, and one controlled experiment) showed that personalized comparisons were more likely when the target's attribute was distinctive or there was an interaction or a close or emotional relationship with the target. Perhaps because these conditions that increase interest in the target as a distinct individual were less common during the triathlon than in everyday life, personalized comparisons were relatively uncommon during the triathlon but relatively common in everyday life. Across studies, generalized comparisons magnified the impact of upward comparisons on overall feelings (but not on interpersonal feelings about the self-target relationship), presumably because generalizing broadens the implications of comparisons, whereas personalizing restricts their relevance to the self-target relationship.     

A randomised controlled trial of acceptance-based cognitive behavioural therapy for command hallucinations in psychotic disorders

Command hallucinations represent a special problem for the clinical management of psychosis. While compliance with both non-harmful and harmful commands can be problematic, sometimes in the extreme, active efforts to resist commands may also contribute to their malignancy. Previous research suggests Cognitive Behaviour Therapy (CBT) to be a useful treatment for reducing compliance with harmful command hallucinations. The purpose of this trial was to evaluate whether CBT augmented with acceptance-based strategies from Acceptance and Commitment Therapy could more broadly reduce the negative impact of command hallucinations. Forty-three participants with problematic command hallucinations were randomized to receive 15 sessions of the intervention "TORCH" (Treatment of Resistant Command Hallucinations) or the control, Befriending, then followed up for 6 months. A sub-sample of 17 participants was randomized to a waitlist control before being allocated to TORCH or Befriending. Participants engaged equally well with both treatments. Despite TORCH participants subjectively reporting greater improvement in command hallucinations compared to Befriending participants, the study found no significant group differences in primary and secondary outcome measures based on blinded assessment data. Within-group analyses and comparisons between the combined treatments and waitlist suggested, however, that both treatments were beneficial with a differential pattern of outcomes observed across the two conditions.     

An ecological momentary assessment of the effects of weight and shape social comparisons on women with eating pathology, high body dissatisfaction, and low body dissatisfaction

This research examined the effects of naturally occurring appearance comparisons on women's affect, body satisfaction, and compensatory cognitions and behaviors. Using ecological momentary assessment, women with high body dissatisfaction and eating pathology (EPHB), high body dissatisfaction (HB), or low body dissatisfaction (LB) recorded their reactions to appearance-focused social comparisons. EPHB and HB women made more upward appearance comparisons than LB women. All women experienced negative emotions and cognitions after upward comparisons, including increased guilt, body dissatisfaction, and thoughts of dieting. EPHB women were most negatively affected by comparisons; they experienced more intense negative emotions, more thoughts of dieting/exercising, and an increase in eating-disordered behavior after upward comparisons. HB women experienced more negative affective consequences and thoughts of dieting than LB women. Results are consistent with social comparison theory and provide important information that may be used to inform eating disorder treatment and prevention efforts.