The Validity of Verifiable and Non‐verifiable Biodata Items: An Examination Across Applicants and Incumbents

This study examines the influence of item verifiability (non‐verifiable vs. verifiable), context (applicant vs. incumbent), and keying procedure on biodata mean test scores and validity. Concurrent and predictive validation studies were conducted using a sample of 425 call center incumbents and a sample of 410 call center applicants. Although applicants did not obtain significantly higher mean biodata scores, results provide support for the hypothesis that the non‐verifiable biodata composite would be less valid in the applicant context, while the verifiable biodata composite would be equally valid across both the applicant and incumbent contexts. The same pattern of results was obtained using both item‐ and option‐keying procedures. Implications for research and practice are discussed.     

TELL ME SOME MORE: EXPLORING HOW VERBAL ABILITY AND ITEM VERIFIABILITY INFLUENCE RESPONSES TO BIODATA QUESTIONS IN A HIGH‐STAKES SELECTION CONTEXT

Although self‐rated or self‐scored selection measures are commonly used in selection contexts, they are potentially susceptible to applicant response distortion or faking. The response elaboration technique (RET), which requires job applicants to provide supporting information to justify their responses, has been identified as a potential way to minimize applicant response distortion. In a large‐scale, high‐stakes selection context (N= 16,304), we investigate the extent to which RET affects responding on a biodata test as well as the underlying reasons for any potential effect. We find that asking job applicants to elaborate their responses leads to overall lower scores on a biodata test. Item verifiability affects the extent to which RET decreases faking, which we suggest is due to increased accountability. In addition, verbal ability was more strongly related to biodata item scores when items require elaboration, although the effect of verbal ability was small. The implications of these findings for reducing faking in personnel selection are delineated.     

JUDGMENTS OF FIT IN THE SELECTION PROCESS: THE ROLE OF WORK VALUE CONGRUENCE

Work values were examined as an antecedent of recruiters' judgments of applicant fit with the organization. Data were collected on the work values of recruiters, their organizations, and job applicants in actual job interviews conducted through the placement center of a large university. Following the interviews, recruiters evaluated applicants' general employability and organization-specific fit. Approximately 4 months following the interviews, data on whether the applicants were invited for a second interview were also obtained. Work value congruence between the applicant and the recruiter was found to be related to judgments of general employability and organization-specific fit. Congruence between the applicant and the organization (as perceived by the recruiter) was not related to judgments of employability and organization-specific fit. Recruiter ratings of employability were related to the decision to invite the applicant for a second interview. Work value congruence was not related to second interview decisions. It is concluded that if work values and judgments of applicant fit influence the personnel selection process, they are more likely to do so at later stages when job offer decisions are made. Work values and judgments of applicant fit seem to have minimal impact on decisions to retain the applicant for additional consideration in early stages of the selection process.     

Impression management and overclaiming on job applications: Related to future sales performance?

Despite decades of research on the role of socially desirable responding (SDR) in hiring contexts, it remains unclear whether job applicant SDR is related to future employee performance. Drawing on a large sample of 1,084 retail sales employees and their original job application data, this study examined the predictive validity of two operationalizations of SDR: (a) impression management captured using a personality scale and (b) overclaiming familiarity with a list of products that were directly relevant to the job. Neither measure was used to inform the hiring decisions. Results showed that job applicant impression management and overclaiming were unrelated to four indicators of future sales performance as employees. Implications for organizational selection policies involving applicant SDR are discussed.     

Specification of Scales in Biodata Form Development: Rational vs. Empirical and Global vs. Specific

Although empirical keying has been the most popular scoring strategy with biodata, researchers have increasingly argued that rational approaches are better for advancing theory. Higher validities and less faking with empirical keys, however, have made many reluctant to abandon them. Research in the personality field provided support for the notion that many rational biodata scales may be multi-faceted. Development of more specific scales was suggested as a technique for creating rational scales with validities that more closely approximate that of empirical keys. Three different strategies (rational, internal, and external/empirical) were used in the scoring of a biodata inventory for use with mechanical equipment franchise owners. The rational and internal approaches were investigated within two samples and at two levels - very specific constructs and global constructs. In addition, two types of criteria were used, including an objective measure of sales and supervisory performance ratings. The specific rational scales were as predictive as the empirical item key. Strategies for developing and scoring a biodata form using a rational approach are discussed.     

NONE OF YOUR BUSINESS: PARAMETERS OF BIODATA INVASIVENESS

In this research, biodata items were evaluated for their perceived invasiveness by over 200 professionals (psychologists and social scientists), as well as three samples of nonprofessionals. In addition to item subject matter, perceived invasiveness of items was related to various biodata attributes, in that items that were more verifiable, more transparent in purpose, and more impersonal were seen as less invasive. Subjects with more positive attitudes toward biodata and organizational selection measures viewed fewer items as invasive, as did those with more education. Dispositional variables and gender were related to invasiveness perceptions in some samples. Four general motives or topics were determined to generate the greatest concern: fear of stigmatization, concern about having applicants recall traumatic events, intimacy, and religion. The results may help develop consensus about defining "invasive" and contribute to ways of reducing perceptions of invasiveness.     

Construct-oriented Biodata Development for Selection to a Differentiated Performance Domain

This study describes the development of a multidimensional biodata form which used explicit constructs to guide item generation and rational scale development, construct validation, criterion measurement and empirical keying. These constructs were goal-orientation, teamwork, customer service, resourcefulness, learning ability and leadership. Exploratory and confirmatory factor analyses in both applicant and incumbent samples were used to identify and test the model which included the thirteen, more differentiated rational scales relating to these six, broader constructs. Empirical keying of the rationally developed scales was conducted against criterion construct scales conceptually related to each predictor construct. Empirical keying at the item level was found to result in higher validities and cross-validities than either empirical keying at the scale level, or rational keying. The item keyed instrument also demonstrated incremental validity over a test of cognitive ability for specific work performance domains as well as overall work performance.     

Applicant Reactions to Selection Procedures: An Updated Model and Meta-Analysis

An updated theoretical model of applicant reactions to selection procedures is proposed and tested using meta-analysis. Results from 86 independent samples  ( N = 48,750)  indicated that applicants who hold positive perceptions about selection are more likely to view the organization favorably and report stronger intentions to accept job offers and recommend the employer to others. Applicant perceptions were positively correlated with actual and perceived performance on selection tools and with self-perceptions. The average correlation between applicant perceptions and gender, age, and ethnic background was near zero. Face validity and perceived predictive validity were strong predictors of many applicant perceptions including procedural justice, distributive justice, attitudes towards tests, and attitudes towards selection. Interviews and work samples were perceived more favorably than cognitive ability tests, which were perceived more favorably than personality inventories, honesty tests, biodata, and graphology. The discussion identifies remaining theoretical and methodological issues as well as directions for future research.     

BEYOND THE MEAN BIAS: THE EFFECT OF WARNING AGAINST FAKING ON BIODATA ITEM VARIANCES

We studied the effects of faking biodata test items by randomly warning 214 of 429 applicants for a nurse's assistant position against faking. While the warning mitigated the propensity to fake, the specific warning effects depended on item transparency. For transparent items, warning reduced the extremeness of item means and increased item variances. For nontransparent items, warning did not have an effect on item means and reduced item variances. These faking effects were best predicted when transparency was operationalized in terms of item-specific job desirability in addition to the item-general social desirability. We also demonstrated a psychometric principle: The effect of warning on means at the item level is preserved in scales constructed from those items, but the effect on variances at the item level is masked at the scale level. These results raise new questions regarding the attenuating effects of faking on validity, and regarding the benefit of warning applicants against faking.     

WEB‐BASED AND PAPER‐AND‐PENCIL TESTING OF APPLICANTS IN A PROCTORED SETTING: ARE PERSONALITY,BIODATA, AND SITUATIONAL JUDGMENT TESTS COMPARABLE?

This quasi-experimental study compares the equivalence of proctored Web-based tests to paper-and-pencil tests in a selection setting. The predictor battery was composed of measures of Conscientiousness, Agreeableness, Emotional Stability, a biodata form, and a situational judgment test. Three samples were administered the same test items, but differed with respect to context and test format: 425 incumbents completed a paper-and-pencil version, 2,544 applicants completed a paper-and-pencil version, and 2,356 applicants completed an identical Web-based version (although these individuals were not randomly assigned to test format, the job and organization were the same for all samples). The results found effects for both context and test format. Relative to the applicants completing the paper-and-pencil measures, the Web-based measures showed (a) better distributional properties, (b) lower means, (c) more variance, (d) higher internal consistency reliabilities, and (e) stronger intercorrelations. Although the applicant context resulted in higher mean responses than the incumbent context, this difference was about twice as large for the paper-and-pencil test than for the Web-based test. All such differences were greater for personality measures than for biodata and situational judgment tests. Overall, these results suggest that proctored Web-based testing has some positive benefits relative to paper-and-pencil measures, and we identify several implications of these findings for research and practice.     

Predicting Voluntary Turnover from Job Applicant Biodata and Other Applicant Information

Research suggests that biodata (i.e., personal history information) is one of the best predictors of voluntary employee turnover. Given this fact, it is surprising that research on biodata has declined in recent years, and that biodata is not commonly used by employers for selecting employees. This article presents the results of a study that focused on the ability of biodata items and other information available on job applicants to predict voluntary turnover. With one exception, the hypotheses offered were confirmed. Specifically, applicant status (i.e., previously had applied for a job), submission of optional personal history information, employment status (i.e., job applicant was employed), and recruitment source (i.e., employee referral) all predicted voluntary turnover. The ability of applicant status, submission of a personal statement, and recruitment source to predict employee performance also was examined. Applicant status and recruitment source were found to predict subsequent performance ratings.     

Cohen's d Corrected for Case IV Range Restriction: A More Accurate Procedure for Evaluating Subgroup Differences in Organizational Research

Organizational and staffing researchers are often interested in evaluating whether subgroup differences exist (e.g., between Caucasian and African‐American individuals) on predictors of job performance. To investigate subgroup differences, researchers often will collect data from current employees to make inferences about subgroup differences among job applicants. However, the magnitude of subgroup differences (i.e., Cohen's d) within incumbent samples may be different (i.e., smaller) than the magnitude of subgroup differences in applicant samples because selection of applicants typically reduces the variance of scores on the predictors (i.e., because lower scoring applicants are not selected). If researchers seek to generalize a d value in an incumbent sample to the applicant population, they may use Bobko, Roth, and Bobko's (correcting the effect size of d for range restriction and unreliability, 2001) Case II or III correction. By extension, Hunter, Schmidt, and Le (implications of direct and indirect range restriction for meta‐analysis methods and findings, 2006) have proposed a Case IV correction, which is more realistic than Bobko et al.'s approach. Therefore, this paper develops a Case IV correction for d (i.e., d_c4). The simulation results showed that the d_c4 was generally accurate across 6,000 simulation conditions. Moreover, 2 published datasets were reanalyzed to show the influence of the Case IV correction on d. In addition, implications and future directions of the d_c4 are discussed.     

Letters of Recommendation for the Predoctoral Internship in Medical Schools and Other Settings: Do They Enhance Decision Making in the Selection Process?

Since the 1970s, letters of recommendation to medical and other internship settings have surfaced as important variables in the internship selection process. However, several studies have challenged their value in the selection process, specifically by pointing out that these letters have an overly positive bias and fail to address applicant weaknesses. Our study, using the Linguistic Inquiry and Word Count text analysis method, developed a quantitative profile for a sample of letters of recommendation, determined whether letters differentiated among applicants with regard to positive/negative attributions, evaluated letters for positive and negative bias, and investigated potential gender bias of writers toward applicants. Results indicated that writers apply positive and negative attributions homogeneously across applicants, thus, rendering applicant differentiation on this basis impossible. Also, results demonstrated that letters of male and female writers were not biased toward male or female applicants. These findings, in combination with previous studies, question the utility of letters of recommendation as presently structured. Possible modifications are discussed.     

APPLICANT WITHDRAWAL: THE ROLE OF TEST-TAKING ATTITUDES AND RACIAL DIFFERENCES

The role of test-taking attitudes in the decisions of applicants to withdraw from a selection process was examined. Measures of test-taking attitudes were administered to 3,290 police officer applicants. Interviews were conducted with 618 applicants who withdrew from the selection process. Comparative anxiety, motivation, and literacy scales were found to predict withdrawal, but the effects were quite small. African-Americans were more likely to withdraw. Small race differences were found on test attitude scales. The requirement of taking a test was not a major factor in applicant withdrawal; procedural fairness and several other factors appeared to play a greater role. A model of applicant withdrawal is proposed based on the qualitative data from applicants who withdrew.     

Comparison of the MMPI and MMPI-2 Among Job Applicants

The primary objective was to compare the MMPI-2 profiles for professional job applicants with corresponding profiles on the original MMPI. Male and female MMPI-2 profiles were also compared, and correlational and factor analyses were used to examine: A) Interscale correlations vis-à-vis item overlap, and B) The impact of K-corrections. The subjects were 82 auditor applicants, plus 212 sales and management applicants from an earlier study. The original MMPI was found to overpathologize applicant profiles. Male and female MMPI-2 profiles differed only on Mf. Scales L, K, and Hy formed a favorable cluster for job applicants, while F, Hs, Pt, Sc formed one unfavorable cluster—and D, Si a second. K-corrections consistently moved clinical scales in the direction of the favorable cluster.     

Contextualized personality measures in employee selection: Extending frame‐of‐reference research with job applicant samples

This study compared contextualized and non‐contextualized personality measures in operational hiring situations, unlike previous research which has largely relied on student or job incumbent samples. Comparisons were made with regard to (A) the frames‐of‐reference adopted by applicants when responding to the measures, (B) relations with subsequent employment interview scores, (C) applicant reactions, and (D) mean scores for the personality scales. The findings highlight potential concerns with using non‐contextualized personality measures for employee selection, as job applicants will likely adopt various frames‐of‐reference unrelated to the intended focus of measurement (i.e., work). Results also indicated that it may be premature to assume contextualized measures elicit improved applicant reactions despite their apparent job‐relatedness. The discussion concludes with suggested directions for future research.     

Applicant Reactions in Selection: Comprehensive meta‐analysis into reaction generalization versus situational specificity

This paper reports a comprehensive quantitative summary into applicant reactions to popular methods of employee selection. A detailed search of published and unpublished studies resulted in a final sample (k) of 38 independent samples covering 10 popular candidate assessment methods. Applicant reactions primary studies included samples from 17 countries internationally. Counter to previous suggestions for the situational specificity of applicant reactions, findings showed considerable similarity supporting the reaction generalizability hypothesis. Reaction favorability was structurally similar across countries and revealed a three‐tier clustering of overall favorability perceptions –most preferred (work samples, interviews), favorably evaluated (resumes, cognitive tests, references, biodata, personality inventories), and least preferred (honesty tests, personal contacts, graphology). Some differences in applicant reactions to dimension‐specific perceptions were found, however. Further analyses revealed strong positive correlations between favorability ratings and their validity and international usage. Implications for future research and ramifications for practice are considered in conclusion.     

Examination of ethnic group differential responding on a biodata instrument

Biodata is a selection device that is often cited for high criterion validity and low adverse impact, yet little research examines the latter point. Further, little is known about whether differential biodata item responding by ethnic group occurs, and if so, whether this impacts ethnic group differences, predictive bias, adverse impact, and criterion validity. Data from 5,242 applicants for a government agency position indicated that elimination of biodata item response options with differential ethnic group selection rates from scoring consideration resulted in a large decrease in ethnic group differences, no predictive bias, less potential adverse impact, and a nonsignificant decrease in criterion validity. Results are compared to a general mental ability measure. Implications and future research directions are discussed.     

APPLICANT REACTIONS TO SELECTION: DEVELOPMENT OF THE SELECTION PROCEDURAL JUSTICE SCALE (SPJS)

This paper describes research that fills a void in the applicant reactions literature by developing a comprehensive measure of Gilliland's (1993) procedural justice rules, called the Selection Procedural Justice Scale (SPJS). Five separate phases of scale development were conducted. In Phase 1 we generated and refined the items. For Phase 2 we reduced the items through exploratory factor analysis using data gathered from 330 applicants for the job of court officer and found higher-order factors consistent with Greenberg (1993a, 1993b). In Phase 3 we confirmed the factor structure using a separate sample of 242 applicants and trainees for the court officer job. In Phase 4 we assessed the initial convergent and divergent validity of the scale. In Phase 5 we tested the generalizability of these items in general and for those receiving positive and negative selection outcomes using 2 student samples. The results demonstrated the usefulness of the SPJS in differentiating each of Gilliland's procedural justice rules and relating them to outcomes included in his model of applicant reactions. A copy of the SPJS is included in the Appendix.     

CORRESPONDENCE BETWEEN ESTIMATES OF CONTENT AND CRITERION-RELATED VALIDITY VALUES

The correspondence between inferences made using two validation strategies–content and criterion-related–were examined in a specific personnel selection application. Empirical validity values and Law-she's (1975) content validity ratios (CVR) were obtained for items from three structured interview guides used in the selection of insurance agents. Ratings of each item by over 300 field managers were used to calculate the CVR values. Statistically significant, yet modest correlations were found between empirical item validities and content validities for an interview guide used to select applicants with prior insurance sales experience. No significant differences were found among these correlations by comparing job experts of different levels of managerial experience and experience in selection. Data for the interview guide used to select experienced applicants also indicated that a content validity approach can be useful in developing a selection instrument with an empirically valid composite rating. The hypotheses were not confirmed for interview guides used to select applicants with no prior insurance sales experience. The practical importance of these results are discussed, as are plans for future research.