THE IMPLICATIONS OF PROFESSIONAL AND LEGAL GUIDELINES FOR COURT DECISIONS INVOLVING CRITERION-RELATED VALIDITY: A REVIEW AND ANALYSIS

Title VII court cases litigated since 1978 were reviewed to assess the implications of the latest professional and legal guidelines for court deliberations regarding the criterion-related validity of paper-and-pencil tests with adverse impact. Major topics important to an understanding of predictor, criterion, procedural, data analysis, and interpretation issues were examined. Among the major findings were the heavy reliance placed on test development procedures and the reluctance of many judges to accept recent research findings, often contained in professional guidelines, which are inconsistent with those in the Uniform Guidelines on Employee Selection Procedures (1978). Suggestions are offered throughout to guide employers who are interested in successfully conducting or defending a criterion-related validation study.     

AGE DISCRIMINATION AND PERSONNEL PSYCHOLOGY: A REVIEW AND SYNTHESIS OF THE LEGAL LITERATURE WITH IMPLICATIONS FOR FUTURE RESEARCH

One hundred and fifty-two court cases were reviewed to determine the standards set by the courts for establishing a claim of age discrimination under the Age Discrimination in Employment Act, as amended. Fifty-three are discussed. Two major issues were examined in Part 1 of the review: (1) the type and extent of the evidence sufficient to establish a complaint of age discrimination and (2) available defenses against age discrimination complaints. Part 2 discussed employer strategies for averting and/or dealing with discrimination complaints. Also discussed were the research-related implications of the age discrimination case law, as well as the role of professionals in future research.     

RELATIONSHIPS BETWEEN GRADUATE TRAINING, PROFESSIONAL AFFILIATION, AND INDIVIDUAL PSYCHOLOGICAL ASSESSMENT PRACTICES FOR PERSONNEL DECISIONS

Survey data on 321 psychologists who conduct individual assessments were analyzed to compare practices and opinions of those with differing graduate training (I/O or non-I/O) and professional affiliation (SIOP or non-SIOP). Differences in practice on the basisof training and affiliation occur in licensing, types of assessments conducted, specific testing instruments used, and knowledge of legal guidelines governing selection practices. However, when purpose of assessment was controlled for by examining only managerial selection assessments, few practice differences related to training and affiliation emerged. Implications for the training of individual assessors are discussed, and several practice variations warranting further research are presented.     

PERFORMANCE APPRAISAL, PROMOTION AND THE COURTS: A CRITICAL REVIEW

Twenty-three Title VII court cases were reviewed in order to determine the standards set by the courts in their assessment of performance appraisal systems when used as the basis for promotion decisions. The topics covered were adverse impact determination, the courts' adjudication strategy, and the evidence needed to justify the performance appraisal procedures. Among the major findings was the courts': (1) failure to adhere to the "applicant flow technique" of adverse impact determination, (2) interest in assessing performance appraisal systems regardless of their adverse impact, (3) ignorance regarding acceptable validation procedures, and (4) focus on objectivity in lieu of validity. The discussion offered suggestions to employers for developing a professionally sound and legally defensible appraisal system.     

COMPARISONS OF MANAGERIAL AND EMPLOYEE SATISFACTION WITH A PERFORMANCE APPRAISAL SYSTEM

Questionnaires were administered to managers and employees in a large multinational corporation to assess satisfaction with various aspects of a performance appraisal system. Comparisons of managers' and employees' satisfaction were made by conducting factor analyses for each sample. The results indicate moderate similarity between the two groups. However, two significant differences were revealed by the analysis. First, there is evidence that employees perceive certain aspects of the appraisal system in a global way, whereas managers differentiate among various components and see them as distinct entities. Second, the relative importance of the factors differs between the two groups. The largest portion of variance accounted for in the employee sample pertained to general satisfaction with the system whereas for managers it pertained to the types of ratings made on the appraisal form. The results are discussed in terms of the different perspectives managers and employees have in the appraisal process.     

HONESTY TESTING FOR PERSONNEL SELECTION: A REVIEW AND CRITIQUE

Paper and pencil predictors of employee theft are described and studies of validity, reliability, and adverse impact of these tests are examined. Validity studies for 10 tests were grouped into 5 categories: comparisons with polygraph examination results, correlations with admissions of past theft, predictive studies using future job behaviors as criteria, comparisons of shrinkage rates before and after the introduction of a testing program, and comparisons of test scores of groups known to be dishonest with groups representing the general population. While positive correlations were consistently found, a variety of methodological differences between studies were identified which make the direct comparison of test validities suspect. High reliabilities are consistently reported, and test score comparisons by race and sex generally report no differences. Ethical issues in honesty test usage are considered and future research needs are identified.     

PERSONNEL SELECTION FOR PHYSICALLY DEMANDING JOBS: REVIEW AND RECOMMENDATIONS

Improvement in personnel selection systems for physically demanding jobs is needed due to equal employment opportunity (EEO) considerations, concern for worker physical well-being, and the lack of alternative procedures. After addressing the special EEO sensitivities of physical abilities selection, the literature is reviewed from a variety of disciplines on: (1) the physiological background underlying the selection strategies, (2) the assessment of human physical abilities, (3) the measurement of physical requirements of jobs, and (4) the physical abilities personnel selection studies reported in the literature. Conclusions are provided in the form of recommendations for future research.     

EFFECTS OF JOB ATTRIBUTES AND RECRUITING PRACTICES ON APPLICANT DECISIONS: A COMPARISON1

Previous research on the factors which affect applicant decisions concerning jobs has focused on the effects of either job attributes or recruiting practices. The present study examined the simultaneous impact of job attributes and recruiting practices on the likelihood of job acceptance by actual job applicants. Path analysis revealed that job attributes but not recruiting practices influenced the likelihood of job acceptance by graduating college students (N= 200). Given that college students might be expected to be the job applicants most swayed by recruiting practices because of their lack of work experience, the results suggest that the emphasis placed on recruiting practices as determinants of applicant decisions may be overstated in the literature.     

PERFORMANCE APPRAISAL OF BEHAVIOR-BASED COMPETENCIES: A RELIABLE AND VALID PROCEDURE

A new performance appraisal system, developed for promotions in the Royal Canadian Mounted Police non-commissioned officer ranks, fairly differentiated among candidates. Members ( N = 6,571) illustrated their performance on core competencies with behavioral examples. Supervisors and then review boards used a BARS procedure to reliably rate performance. Both candidates and supervisors supported the system. The performance appraisal scores predicted career advancement in the organization.     

WORK ENGAGEMENT: A QUANTITATIVE REVIEW AND TEST OF ITS RELATIONS WITH TASK AND CONTEXTUAL PERFORMANCE

Many researchers have concerns about work engagement's distinction from other constructs and its theoretical merit. The goals of this study were to identify an agreed‐upon definition of engagement, to investigate its uniqueness, and to clarify its nomological network of constructs. Using a conceptual framework based on Macey and Schneider (2008; Industrial and Organizational Psychology, 1, 3–30), we found that engagement exhibits discriminant validity from, and criterion related validity over, job attitudes. We also found that engagement is related to several key antecedents and consequences. Finally, we used meta‐analytic path modeling to test the role of engagement as a mediator of the relation between distal antecedents and job performance, finding support for our conceptual framework. In sum, our results suggest that work engagement is a useful construct that deserves further attention.     

THE IMPLICATIONS FOR PERSONNEL SELECTION OF APPARENT DECLINES IN PREDICTIVE VALIDITIES OVER TIME: A CRITIQUE OF HULIN, HENRY, AND NOON

Hulin, Henry, and Noon (1990) reviewed evidence from a number of studies which supported, in their view, the position that predictive validities decreased over time. If correct, their results would have significant implications for personnel selection practice and research. However, further analysis of their evidence suggested that their results may have only limited generalizability. More specifically, few of the studies they used to support their claim of decreasing predictive validities were field studies of prediction-criterion pairs. Furthermore^ reported data on lagged intercorrelations were of limited relevance to the question of decreasing validities. Finally, a large body of data relevant to the issue of time-lagged validities in a personnel selection context were omitted because the data did not meet Hulin et al.'s restrictive criteria.     

INCREASED ACCURACY FOR RANGE RESTRICTION CORRECTIONS: IMPLICATIONS FOR THE ROLE OF PERSONALITY AND GENERAL MENTAL ABILITY IN JOB AND TRAINING PERFORMANCE

A recently developed procedure produces substantial improvements in the accuracy of corrections for range restriction and reveals that predictive validities of employment selection methods are higher than previously thought. This study applied this procedure to meta‐analytic validities of 2 personality measures (Conscientiousness and Emotional Stability) and general cognitive ability measures. Results show that the increases in validity estimates are greater for cognitive ability than for personality, reducing the relative validity of personality. In addition, the incremental validity of these personality measures over that provided by cognitive ability is smaller than previously estimated. Path analyses based on the more accurate data reveal relatively smaller path coefficients from these personality traits to job and training performance. These findings have implications for theories of job performance and for the practical value in selection of personality measures relative to cognitive ability measures.     

SOCIAL COMPARISON THEORY AND THE EVALUATION OF PEER EVALUATIONS: A REVIEW AND SOME APPLIED IMPLICATIONS

Some of the more fundamental characteristics of peer evaluations were reviewed and their striking industrial validity was noted. Subsequently, the theories put forth to account for the validity of peer evaluations were examined and found to be wanting. A review of the conditions under which peer evaluations yield strong validity coefficients as well as certain factor analytic and experimental studies indicated that social comparison theory might provide a useful framework for elucidating the nature of peer evaluation process. This theory was then related to the characteristics of peer evaluation and certain applied implications were suggested. It was concluded that further empirical research along these lines might have some value.     

DEVELOPMENT AND VALIDATION OF A COMPUTERIZED INTERPRETATION SYSTEM FOR PERSONNEL TESTS

A computerized system was developed for generating narrative interpretations of scores from a battery of personnel screening tests. The report structure and interpretive statement library were designed to capture the test expertise and interpretive strategies of a panel of testing experts. This was accomplished by enumerating the questions that the experts believed the battery could answer, developing answers to these questions, and devising rules for selecting the appropriate answers based on test-battery scores. The accuracy, thoroughness, readability, and coherence of the computer-generated reports were evaluated in comparison to reports generated by human experts for the same examinees. Results of the evaluation showed the computerized reports to be more accurate and thorough, as readable, and somewhat less coherent than interpretations generated by the typical human expert. The computerized system development and validation strategies described are useful for other applications in which numbers are interpreted in a narrative report format.     

TRANSFER OF TRAINING: A REVIEW AND DIRECTIONS FOR FUTURE RESEARCH

Transfer of training is of paramount concern for training researchers and practitioners. Despite research efforts, there is a growing concern over the "transfer problem." The purpose of this paper is to provide a critique of the existing transfer research and to suggest directions for future research investigations. The conditions of transfer include both the generalization of learned material to the job and the maintenance of trained skills over a period of time on the job. The existing research examining the effects of training design, trainee, and work-environment factors on conditions of transfer is reviewed and critiqued. Research gaps identified from the review include the need to (1) test various operationalizations of training design and work-environment factors that have been posited as having an impact on transfer and (2) develop a framework for conducting research on the effects of trainee characteristics on transfer. Needed advancements in the conceptualization and operationalization of the criterion of transfer are also discussed.