Range Shrinkage of Cognitive Ability Test Scores in Applicant Pools for German Governmental Jobs: Implications for range restriction corrections

Range restriction corrections require the predictor standard deviation in the applicant pool of interest. Unfortunately, this information is frequently not available in applied contexts. The common strategy in this type of situations is to use national‐norm standard deviation estimates. This study used data from 8,276 applicants applying to nine jobs in German governmental organizations to compare applicant pool standard deviations for two cognitive ability tests with national‐norm standard deviation estimates, and standard deviations for the total group of governmental applicants. Results revealed that job‐ and organizational context‐specific applicant pool standard deviations were on average about 10–12% smaller than estimates from national norms, and about 4–6% smaller than standard deviations for the total group of governmental applicants.     

TELL ME SOME MORE: EXPLORING HOW VERBAL ABILITY AND ITEM VERIFIABILITY INFLUENCE RESPONSES TO BIODATA QUESTIONS IN A HIGH‐STAKES SELECTION CONTEXT

Although self‐rated or self‐scored selection measures are commonly used in selection contexts, they are potentially susceptible to applicant response distortion or faking. The response elaboration technique (RET), which requires job applicants to provide supporting information to justify their responses, has been identified as a potential way to minimize applicant response distortion. In a large‐scale, high‐stakes selection context (N= 16,304), we investigate the extent to which RET affects responding on a biodata test as well as the underlying reasons for any potential effect. We find that asking job applicants to elaborate their responses leads to overall lower scores on a biodata test. Item verifiability affects the extent to which RET decreases faking, which we suggest is due to increased accountability. In addition, verbal ability was more strongly related to biodata item scores when items require elaboration, although the effect of verbal ability was small. The implications of these findings for reducing faking in personnel selection are delineated.     

Offsetting Performance Losses Due to Cheating in Unproctored Internet‐based Testing by Increasing the Applicant Pool

A concern about unproctored Internet‐based testing (UIT) without follow‐up verification is that because applicants are able to cheat when completing tests, test validity decreases to an unknown extent. However, this view ignores a potential advantage to UIT: Should UIT increase applicant pool size, an organization with a fixed goal in terms of the number of candidates to advance to the next stage of the hiring process can increase the cutoff score and thus be more selective. In this study, a simulation was conducted to determine if this advantage outweighs the cheating disadvantage in the prediction of job performance. When the applicant pool is increased substantially through UIT, job performance outcomes are often higher even when many applicants increase their test scores through cheating.     

Applicant reactions and faking in real-life personnel selection

Faking may affect hiring decisions in personnel selection. All the antecedents of faking are still not known. The present study investigates the association between applicants' reactions about the selection procedure and their tendency to fake. The subjects (N = 180) were real-life applicants for a fire and rescue personnel school. After completing the selection process, the applicants filled out a questionnaire about their test reactions (Chan, Schmitt, Sacco & DeSohon, 1998b) and a faking scale, the Balanced Inventory of Desirable Responding (Paulhus, 1991). The results based on Structural Equation Modelling (SEM) indicated that the more positive reactions applicant had about the selection procedure the more impression management they had. The applicant reactions were not associated with self-deception.     

Job-specific applicant pools and national norms for personality scales: implications for range-restriction corrections in validation research

This article compares variability in national norms to variability in job-specific applicant pools for personality scales. Across 23,231 job applicants for 111 jobs, we found that job-specific applicant pools were slightly less variable than broad normative data (by about 4%) based on a national norm sample of 40,474 job applicants. For some personality variables, the amount of variability reduction correlated with work characteristics as captured by the Data, People and Things scales of the Dictionary of Occupational Titles (US Department of Labor, 1977). Most correlations were small in magnitude. In general, the small reductions in job-specific applicant pool variability provide evidence that researchers' decisions to use manual norm group standard deviations in range restriction corrections has not appreciably inflated the personality-criterion relationships studied. Implications for attraction and gravitation to jobs are discussed.     

Exploring Differential Subgroup Functioning on SAT Writing Items: What Happens When English Is Not a Test Taker's Best Language?

The purpose of this study is to explore patterns in model-data fit related to subgroups of test takers from a large-scale writing assessment. Using data from the SAT, a calibration group was randomly selected to represent test takers who reported that English was their best language from the total population of test takers (N = 322,011). A reference scale for the items was constructed based on EBL responses. Response behaviors of test takers who reported that English was not their best language (ENBL) were examined in relationship to this reference scale. This study illustrates the use of differential subgroup analyses to identify patterns related to person misfit within subgroups, as well as subsets of items, that may affect the validity of writing scores for ENBL test takers. The methodology described here offers an approach that can be used to explore, understand, and improve the validity of scores obtained from ENBL test takers in large-scale writing assessments.     

Comparing applicants and incumbents: Effects of response distortion on mean scores and validity of personality measures

This study examined the negative effect of likely applicant distortion on mean scores and validity of personality measures. The personality test scores and performance ratings of applicants were directly compared to those of incumbents with the same occupation in four different samples. The results showed that applicant mean scores were higher and validity coefficients were lower than for incumbents.     

A Meta‐Analytic Investigation of Job Applicant Faking on Personality Measures

This study investigates the extent to which job applicants fake their responses on personality tests. Thirty‐three studies that compared job applicant and non‐applicant personality scale scores were meta‐analyzed. Across all job types, applicants scored significantly higher than non‐applicants on extraversion (d=.11), emotional stability (d=.44), conscientiousness (d=.45), and openness (d=.13). For certain jobs (e.g., sales), however, the rank ordering of mean differences changed substantially suggesting that job applicants distort responses on personality dimensions that are viewed as particularly job relevant. Smaller mean differences were found in this study than those reported by Viswesvaran and Ones (Educational and Psychological Measurement, 59 (2), 197–210), who compared scores for induced “fake‐good” vs. honest response conditions. Also, direct Big Five measures produced substantially larger differences than did indirect Big Five measures.     

Gender Bias in Asylum Adjudications: Evidence for Leniency toward Token Women

Gender is one of the most frequently studied variables in the literature on judicial decision-making. We add to this literature by hypothesizing that the impact of applicant gender is conditional on the gender balance in a judge’s caseload. We expect that female applicants receive more favorable decisions from judges whose caseload skews strongly male. Analyzing over 40,000 rulings by the Austrian Asylum Court between 2008 and 2013, we find support for direct gender effects for applicants and judges (yet no significant interaction between the two). We also show that gender balance in the caseload is a strong moderator of applicant gender. Judges with predominantly male caseloads are strongly biased toward female applicants, whereas judges facing a gender-balanced set of applicants display hardly any gender bias at all. These findings tackle essential questions of democratic rule of law and human rights. They indicate that applicants’ fundamental rights to a fair and equal trial may have been compromised. We discuss institutional remedies to reduce the potential for gender bias in Austrian asylum adjudication.

     

APPLICANT WITHDRAWAL: THE ROLE OF TEST-TAKING ATTITUDES AND RACIAL DIFFERENCES

The role of test-taking attitudes in the decisions of applicants to withdraw from a selection process was examined. Measures of test-taking attitudes were administered to 3,290 police officer applicants. Interviews were conducted with 618 applicants who withdrew from the selection process. Comparative anxiety, motivation, and literacy scales were found to predict withdrawal, but the effects were quite small. African-Americans were more likely to withdraw. Small race differences were found on test attitude scales. The requirement of taking a test was not a major factor in applicant withdrawal; procedural fairness and several other factors appeared to play a greater role. A model of applicant withdrawal is proposed based on the qualitative data from applicants who withdrew.     

Contextualized personality measures in employee selection: Extending frame‐of‐reference research with job applicant samples

This study compared contextualized and non‐contextualized personality measures in operational hiring situations, unlike previous research which has largely relied on student or job incumbent samples. Comparisons were made with regard to (A) the frames‐of‐reference adopted by applicants when responding to the measures, (B) relations with subsequent employment interview scores, (C) applicant reactions, and (D) mean scores for the personality scales. The findings highlight potential concerns with using non‐contextualized personality measures for employee selection, as job applicants will likely adopt various frames‐of‐reference unrelated to the intended focus of measurement (i.e., work). Results also indicated that it may be premature to assume contextualized measures elicit improved applicant reactions despite their apparent job‐relatedness. The discussion concludes with suggested directions for future research.     

Nonparticipation in Huntington's Disease Predictive Testing: Reasons for Caution in Interpreting Findings

When given the opportunity to clarify their genetic status, most individuals at risk for Huntington's disease (HD) currently show a preference not to know. Our understanding of the characteristics of those who do not request HD predictive testing, and the factors influencing their decision, lags behind our knowledge of test applicants. In the light of our experience with interviewing a random sample of nonparticipants in an ongoing study, we critically analyze research concerning the differences between participants and nonparticipants in HD predictive testing programs and the interpretive characterization of these two groups. The findings concerning nonparticipants are limited to a small sample of the at-risk population, with the problems of biased samples and low response rates. We discuss this and other aspects of research in this area, notably the context and timing of data collection and the predominantly questionnaire-based methodology, that lead us to interpret the conclusions drawn about these two groups with caution.     

A Field Study of the Role of Big Five Personality in Applicant Perceptions of Selection Fairness,Self, and the Hiring Organization

Although there is a growing applicant reactions literature, relatively little work has addressed the role of personality in applicant perceptions. Using a sample of actual law enforcement applicants (N=120), we studied the relationship between Big Five personality measured before a written test and applicants' post‐test fairness perceptions, perceptions of themselves, and perceptions of the hiring organization. Personality was related to applicant perceptions after controlling for gender and test score. Personality also accounted for significant variance in self‐perceptions and perceptions of the hiring organization beyond that accounted for by fairness perceptions. Neuroticism and agreeableness were the most consistent predictors of applicant perceptions. Our discussion focuses on the consideration of individual differences in applicant reactions research.     

The occurrence of an increase in correlation with explicit selection

It is generally believed that correlations and standard deviations measured within an explicitly selected group must be smaller than those within an applicant population. A small data set is used to show that this is not always true, and that both validity and reliability estimates within a selected group can exceed those within the applicant population. The increase in correlation within the explicitly selected group is tied to an increase in standard deviation of the predictor in the selected group. This conclusion extends Levin's (1972) result for the case of incidental selection, selection on an unmeasured third variable. If possible, theoretical derivations should not be limited to the case where the predictor standard deviation in the applicant population exceeds that in the selected group. When such a situation occurs in observed data, it cannot be immediately dismissed as an artifact.The author would like to thank Mark L. Davison and other reviewers for their comments on earlier drafts of the paper. An anonymous reviewer provided a better example than the original one, resulting in a more general conclusion.This work was supported in part by a Dissertation Fellowship from the University of Minnesota.     

A comparison of MMPI profiles for state and private disability insurance applicants

The use of the MMPI in predicting successful rehabilitation outcomes has met with limited success. Because the motives of a disability insurance applicant may differ greatly from a disability insurance recipient, disability applicants were investigated. The MMPI scores of state disability applicants and private industrial insurance applicants were compared to a control group. Positive outcomes of test taking was possible for all three groups. Significant group differences were found for scales F, Hs, D, Hy, Pd, Pa, Pt, Sc, and Ma. Significant sex differences were found for scales Hs, D, Hy, Mf, Pt, and Sc.     

Interaction of Recruiter and Applicant Gender in Resume Evaluation: A Field Study

In the present study, we examined the effect of recruiter and applicant gender on recruiters' evaluations of applicants' qualifications as reported on actual applicant resumes. Forty recruiters evaluated applicant resumes that were randomly allocated to them. In total, 388 recruiter evaluations of applicant resumes comprised the sample. Results indicate that recruiter and applicant gender interacted to predict recruiters' perceptions of applicants' qualifications. Male recruiters' perceptions of applicants' work experiences did not differ depending on applicant gender. However, female recruiters perceived male applicants' resumes to report more work experiences than resumes of female applicants. Furthermore, male recruiters perceived female applicants as having more extracurricular interests than male applicants. Female recruiters rated both male and female applicants as having about the same amount of extracurricular activity information on their resumes. Gender role theory provides a possible explanation for the study's findings.     

Predicting Individual Score Elevation in an Applicant Setting: The Influence of Individual Differences and Situational Perceptions

Previous research has indicated that applicants can increase their scores on non‐cognitive measures by half of 1 standard deviation (e.g., Rosse, Stecher, Miller, & Levin, 1998 ; Viswesvaran & Ones, 1999 ). Two influential factors have been proposed to influence this elevation: individual differences and situational influences (e.g., Douglas, McDaniel, & Snell, 1996 ). The current study examined how individual differences and motivation (expectancy theory) predicted individual response elevation from a general to a job applicant context using a conscientiousness measure. Results indicated that elevation was primarily predicted by emotional stability, instrumentality, and the interaction between expectancy and instrumentality. Practical implications and future research directions are discussed.     

Development and psychometric examination of a German video‐based situational judgment test for social competencies in medical school applicants

The present study describes the development and validation of a video‐based situational judgment test (SJT) assessing social competencies in applicants to medical school. Study 1 explored the psychometric properties of the SJT based on two applicant cohorts (N₁ = 769, N₂ = 787). Study 2 compared SJT data from applicants and 90 medical students. Tests for mean differences, correlation and factor analyses were used. In Study 1, the supposed two‐factor model for the SJT must be rejected. According to the minimum average partial test a single factor solution explaining 19–21% of the variance was suggested. The SJT showed satisfactory psychometric properties and stronger correlations to personality than to cognition as well as high acceptance. In study 2, applicants reached higher SJT scores than students.