ON THE NEED FOR QUALITY CONTROL IN VALIDATION RESEARCH

The aptitude tests used to help make personnel decisions about military recruits were validated against hands-on tests of job performance in two Marine Corps occupational specialties, radio repairers and automotive mechanics. The tests were administered by Marine Corps noncommissioned officers. Marine Corps units provided the test administrators, testing facilities, and examinees. Data collected under such conditions are filled with errors that reduce the accuracy of validity coefficients. This paper shows how validity coefficients can be made more accurate by exercising quality control during the statistical analysis.     

EFFECTS OF APPLICANT RACE, SEX, SUITABILITY, AND ANSWERS ON INTERVIEWER'S QUESTIONING STRATEGY AND RATINGS

Impression formation research (Snyder & Swann, 1978) suggests that people engage in information seeking strategies designed to confirm their impressions of others. In an extension of this work to the selection interview, Sackett (1982) found, however, that subjects did not consistently adopt a confirmatory information seeking strategy. This study is a further examination of Snyder and Swann's theory. In simulated interviews 170 subjects (1) read resumes of hypothetical applicants, (2) selected ten questions to ask each applicant, (3) received written responses to each question immediately on its selection, and (4) rated each applicant's suitability. An interaction accounting for 73% of variance indicated that applicant ratings were based primarily on the content of applicants' responses, but initial impressions also played a part. No clear-cut, confirmatory information seeking strategy was observed.     

CONVERGENCE AMONG DATA SOURCES, RESPONSE BIAS, AND RELIABILITY AND VALIDITY OF A STRUCTURED JOB ANALYSIS QUESTIONNAIRE

Research examined both the Position Analysis Questionnaire (PAQ) and the possible analyst sources for gathering job analysis information. 25 state government jobs were examined using job incumbents, supervisors, job analysts, and a comparison group of college students. Reliability and validity (judge convergence and prediction of present pay levels) information was determined for each of the four analyst categories. Results indicate that there is little difference between analyst sources, including students, in terms of their ability to reliably analyze a job using the PAQ. Convergent validity results showed a high degree of agreement among all judge categories when summing item frequencies across all 25 jobs. The prediction of present pay levels was significant for all judge categories but was noticeably smaller than previously reported studies. This seemed to be primarily due to the restriction in salary range of the present study. An analysis of judge response bias showed that supervisors and incumbents rate all or most PAQ items higher than their analyst counterparts. These findings suggest that who furnishes responses to a job analysis inventory makes little practical difference. The exception is that the determination of pay levels and human requirements for test construction purposes should be viewed with caution when different analyst sources are utilized for different jobs.     

A METHODOLOGICAL APPROACH TO JOB CLASSIFICATION FOR PERFORMANCE APPRAISAL PURPOSES

This paper illustrates a methodological approach to classifying jobs for the purpose of developing performance evaluation instruments. The approach was to administer a worker-oriented job inventory to 2,023 incumbents across several jobs and several levels of responsibility (rank) in the U. S. Coast Guard. The data from the job inventory were then analyzed using Tucker's Three-Mode Factor Analysis. Output from the three-mode factor analysis was used to identify combinations of jobs and ranks for which separate appraisal instruments could be developed. In addition, output from the three-mode factor analysis was used to suggest the content of the various appraisal instruments. Advantages, applications, and limitations of this approach are discussed.