A comprehensive meta-analysis of the predictive validity of the graduate record examinations: implications for graduate student selection and performance

This meta-analysis examined the validity of the Graduate Record Examinations (GRE) and undergraduate grade point average (UGPA) as predictors of graduate school performance. The study included samples from multiple disciplines, considered different criterion measures, and corrected for statistical artifacts. Data from 1,753 independent samples were included in the meta-analysis, yielding 6,589 correlations for 8 different criteria and 82,659 graduate students. The results indicated that the GRE and UGPA are generalizably valid predictors of graduate grade point average, 1st-year graduate grade point average, comprehensive examination scores, publication citation counts, and faculty ratings. GRE correlations with degree attainment and research productivity were consistently positive; however, some lower 90% credibility intervals included 0. Subject Tests tended to be better predictors than the Verbal, Quantitative, and Analytical tests.     

Graduate Student Selection: Graduate record examination,socioeconomic status,and undergraduate grade point average as predictors of study success in a western European University

Graduate students' mobility has increased within Europe. Yet, empirical evidence on the validity of standardized admission tests in Europe is still scarce. The aim of the present study was therefore to investigate the incremental validity of the GRE® revised general test above undergraduate grade point average (U‐GPA) by focusing on a multinational sample of master students (N = 282) enrolled at a Dutch university. Results indicated that the Analytical Writing part of the GRE predicts graduate grade point average above and beyond U‐GPA. Furthermore, the results suggest that the relationship is independent of students' socioeconomic status as indexed by parental education. Implications for graduate student selection in a European context are discussed.     

The validity of the general ability measure for adults: comparison with WAIS-R IQ scores in a sample of college students with academic difficulties

The present study investigated the validity of the General Ability Measure for Adults (GAMA) by comparing it to the WAIS-R using a sample of 80 college students reporting learning difficulties. Results indicated that the mean GAMA IQ score did not deviate significantly from the mean WAIS-R IQ scores. The GAMA Full Scale IQ correlated significantly with the WAIS-R PIQ, VIQ, and FSIQ scores. However, the obtained correlation coefficient for the GAMA and WAIS-R PIQ significantly differed from the observed correlation coefficient between the GAMA and WAIS-R VIQ, suggesting that the GAMA was more clearly associated with perceptual skills than verbal abilities. When the correlation coefficients between the GAMA and WAIS-R scores were corrected for the effects of range restriction, the correlation coefficients increased, yet demonstrated the same pattern (e.g., GAMA/PIQ,.69; GAMA/VIQ,.36; GAMA/FSIQ,.60). The GAMA s accuracy in predicting individual student performance on the WAIS-R FSIQ also was examined.     

Predictive Validity of the Air Force Officer Qualifying Test for Non-Rated Officer Specialties

The Air Force Officer Qualifying Test (AFOQT) is used to qualify applicants for officer commissioning and aircrew training programs. The current study examined its predictive validity for 14 officer technical training courses for which there are no additional AFOQT minimum qualification requirements beyond those for officer commissioning. Sample sizes ranged from 16 to 2,190 with a mean and median size of 753 and 319 officers. Ninety percent (63 of 70) of the observed correlations between the AFOQT composites and average technical training grades were statistically significant. Meta-analyses were conducted to determine whether the AFOQT validities were generalizable across training specialties. Analyses were done on the observed data, after correction for range restriction, and after correction for both range restriction and criterion unreliability. The pattern of validities was similar for all three meta-analyses. The lower bound of the 95% confidence interval and the 95% credibility interval around the weighted mean validities were greater than zero for all five AFOQT composites supporting its utility for making personnel selection decisions for these jobs. The Verbal composite had the lowest and the Academic Aptitude composite had the highest weighted mean validity. The weighted mean of the validity coefficients across training specialties ranged from .261 to .326 for the observed data, .322 to .387 for the range-restriction corrected data, and .360 to .433 for the fully corrected data. Additional efforts are required to examine the generalizability of the results for a broader range of occupational specialties and to set minimum qualifying scores.     

A Case Illustrating the Concept of the Alpha Schizophrenic

The construct validity of the Academic Self-Concept Scale (Reynolds, in press) as a measure of an academic facet of general self-concept in college students was examined. Subjects were 589 undergraduate students from three colleges. Convergent validity was demonstrated in the form of correlations between the academic self-concept measure and subjects' grade point average (GPA), and scores on measures of general self-concept and locus of control. A multiple regression analysis of the Academic Self-Concept Scale produced a multiple correlation of .59 with GPA and general self-concept. Discriminant validity was shown via low correlations between academic self-concept and measures of mental ability and social desirability. As expected, differences in academic self-concept between year levels in college were also found. Factor analysis of the Academic Self-Concept Scale and subsequent oblique rotation resulted in seven interpretable factors. The results of this investigation lend support for the construct validity of the Academic Self-Concept Scale.     

Five Factor Biodata Inventory: resistance to faking

Faking on a biographical inventory compared to a traditional personality inventory was assessed in measuring the Five Factor Model of Personality. 705 subjects were randomly assigned to either an Answer Honestly or Faking condition. All subjects were recruited from psychology classes at two New Jersey State colleges. Women comprised 68.6% of the participants. The average age of the subjects was 24 yr. 383 subjects took part in the Answer Honestly condition. 322 participated in the Faking condition. In the Faking condition, subjects responded as if applying for the position of librarian. All subjects completed a biodata inventory, the NEO-Five Factor Inventory, a social desirability scale, a letter-cancellation task, and self-reported their grade point average. Criterion-related validity was assessed for both test scores across samples. Comparisons between samples indicated that subjects inflated scores on both inventories in socially desirable directions. Biodata Inventory scores were less elevated under the Faking conditions than the NEO-Five Factor Inventory scores.     

Influence of Irrelevant Speech on Standardized Test Performance

The aim of this study was to estimate the impact of distraction on standardized test performance. The distraction investigated here was from fellow examinees who were taking a speaking test. Study participants were volunteers (N = 171) who had previously taken the Graduate Management Admission Test (GMAT), the Graduate Record Examinations (GRE) General Test, or the Test of English as a Foreign Language (TOEFL). They were invited to retake a different form of the same test under either distracting conditions or standard, distraction-free conditions. Test takers expressed strong negative perceptions about the distraction caused by fellow test takers. The impact on actual test performance, however, was slight in the GMAT sample and negligible in both the GRE and TOEFL samples. Moreover, the influence of distraction was no greater than that associated with other common, undesirable influences.     

Air force officer qualifying test validity for predicting pilot training performance

The AFOQT was validated for the prediction of pilot training criteria. Subjects were 7,563 men and women selected for pilot training on the basis of educational attainment and AFOQT scores. Criterion variables included daily flight training grades, check flight grades in subsonic and transonic aircraft, and overall academic performance in the 53 week pilot training course. Test validities were presented as observed, corrected for multivariate range restriction, and corrected for multivariate range restriction and unreliability. The Aviation Information and Instrument Comprehension tests, measures of job knowledge, were most predictive of daily and check flights in the initial subsonic jet aircraft. This reflects the relative greater importance of prior job knowledge early in training. The Scale Reading test, a measure of perceptual speed, was most predictive for daily and check flights in the advanced transonic training aircraft. The Arithmetic Reasoning test, a good measure of general cognitive ability, was most predictive of aeronautics in ground school. The development of an improved pilot selection composite is suggested by the results of the validity analyses.The views expressed are those of the authors and not necessarily those of the Department of the Air Force, Department of Defense, or the Government of the United States.     

Perceptions of Overt and Personality-based Integrity Tests

Differences in test-taker perceptions between overt and personality-based integrity tests were examined. Following administration of both types of integrity tests, 255 undergraduate students provided ratings of perceived face validity and perceived predictive validity. Following receipt of actual test scores, 126 test takers participated in a second phase of the study in which they reported perceptions of distributive justice. Test takers perceived overt integrity tests as having greater face validity and predictive validity than personality-based integrity tests. Perceptions of job-relatedness were not strongly related to test performance on either test type. Distributive justice perceptions were related to test performance, but not type of integrity test.     

Test anxiety inventory: 30 years later

Abstract Research suggests that test anxiety is associated with a number of maladaptive factors. The majority of test anxiety research includes the Test Anxiety Inventory (TAI) as a primary outcome variable. However, the TAI was normed on college undergraduates in 1980. The academic landscape has altered in a variety of ways in the past 30 years, which may result in out-of-date norms. This study examined changes in TAI scores in college undergraduates (n =437) as well as convergent validity with measures of trait anxiety and academic performance. Results indicated increases in TAI scores for females while holding constant for males. Additionally, females and males displayed positive correlations between the TAI and state-trait anxiety inventory, while only females displayed a significant negative correlation between the TAI and grade point average. Data provide evidence of changes in TAI scores. As a result, researchers should be careful when drawing conclusions based on original TAI norms, especially in the case of female undergraduates.     

Personality,achievement test scores,and high school percentile as predictors of academic performance across four years of coursework

This research examined the role of three Big Five-related personality traits (Prudence, Sociability, and Ambition) as unique predictors of academic performance. Two successive undergraduate classes (entering in 1998 and 1999) completed personality inventories upon entering college. The authors obtained grade point averages (GPA’s) for all students in the Spring of 2003. Two of the traits, Prudence and Sociability, contributed unique variance to GPA when the effects of high school percentile rank and achievement test scores were controlled for both classes. There was a decline in incremental validity for these two personality traits as predictors of GPA later in student tenure. Ambition was unrelated to academic performance for both classes.     

Fear of Success,Need Achievement,and Fear of Failure: A factor analytic study

Several tests of Fear of Success (FOS), Fear of Failure (FOF), and Need Achievement (nAch), plus Taylor's Manifest Anxiety Scale and a social desirability scale were administered to 104 males and 101 females. FOS, FOF, and nAch did not show factor validity. Results for predictive validity were also poor. Anxiety defined the first factor in the factor analysis. All first-factor tests negatively predicted female grade point average and American College Test scores. All the first-factor tests showed sex differences, with females scoring higher. A clarifying role for anxiety in the study of sex differences in achievement is indicated.     

Intra- and intercultural comparisons of the personality profiles of medical students in Argentina and the United States

This study was designed to compare the personality profiles of medical students in Argentina and the United States. The ultimate purpose of the research was to study the value of personality measures in predicting academic and professional performances. Participants were 421 medical students in Argentina (254 women, 167 men) and 623 medical students in the United States (207 women, 416 men). Eight personality measures were administered: Perception of Stressful Life Events, Test Anxiety, General Anxiety, Loneliness, Self-Esteem, Locus of Control, Extraversion, and Neuroticism. Intracultural comparisons showed some minor gender differences in personality profiles within each culture (e.g., in the United States, women scored higher than men on the Perception of Stressful Life Events and General Anxiety scales, and in Argentina, women scored higher on the Test Anxiety scale). Intercultural comparisons of personality profiles showed that Argentine medical students obtained higher average scores than did their American counterparts on the Perception of Stressful Life Events, Test Anxiety, General Anxiety, External Locus of Control, Extraversion, and Neuroticism scales. Argentine students scored lower on the Loneliness scale than did their American counterparts. Psychometric findings supported the measurement properties of the personality measures in the two cultures (e.g., construct validity, and internal consistency aspect of reliability). Further study of the implications of the findings in predicting academic attainment in medical school and to physician performance is recommended.     

Prediction of job satisfactoriness for workers with severe handicaps from aptitudes,personality, and training ratings

The predictive validity of General Aptitude Test Battery and Sixteen Personality Factor Questionnaire scores were compared to standard training ratings made by vocational instructors against the criterion of work performance measured by the Minnesota Satisfactoriness Scales for a sample of 106 employees with severe handicaps. The psychometric test variables were not correlated with the criterion; however, the training ratings were consistently predictive of the job satisfactoriness scores. These results suggest that the employment potential of job applicants with disabilities can be assessed more accurately using situational training ratings, as opposed to standardized psychometric test scores.     

Validity Stability Across Entering College Cohorts: Exploring the temporal generalizability of local validity estimates

Local validity studies rely on the assumption that validity estimates from one incumbent sample approximate validity for future applicant pools. We test this assumption using SAT scores and high school grades as predictors of first year college grade point average across multiple college applicant pools for over 100 schools. We present evidence for substantial absolute and rank order consistency in validity estimates. However, this consistency is far less than perfect, resulting in potentially meaningful utility differences over time. In addition, observed fluctuations are not fully explained by sampling error alone.     

Validity of the Rey Visual Design Learning Test in primary and secondary school children.

The Rey Visual Design Learning Test (Rey, 1964, cited in Spreen & Strauss, 1991, Wilhelm, 2004) assesses immediate memory span, new learning, delayed recall and recognition for nonverbal material. Two studies are presented that focused on the construct validity of the RVDLT in primary and secondary school children. In the first study, primary school children performed the RVDLT and the Biber Figural Learning Test, as well as the WISC-R Block design Test, Boston Naming Test, and the Trailmaking Test, to assess discriminant validity. In the second study, the age range was expanded and the subtest Visual Reproductions of the Wechsler Memory Scale with a Delayed recall phase was used to assess the construct validity. A test for visual-motor integration and a test for attention, concentration, and speed of information processing were also added to complete the test battery for assessing discriminant validity. Moderate to high correlations were found between scores on the RVDLT and the tests used to assess construct validity. The correlational pattern of RVDLT scores and the scores on the discriminant tests is discussed.     

Construct Validity of Raven's Advanced Progressive Matrices for African and Non‐African Engineering Students in South Africa

We test the hypothesis that the Raven's Advanced Progressive Matrices has the same construct validity in African university students as it does in non‐African students by examining data from 306 highly select 17‐ to 23‐year olds in the Faculties of Engineering and the Built Environment at the University of the Witwatersrand (177 Africans, 57 East Indians, 72 Whites; 54 women, 252 men). Analyses were made of the Matrices scores, an English Comprehension test, the Similarities subscale from the South African Wechsler Adult Intelligence Scale, end‐of‐year university grades, and high‐school grade point average. Out of the 36 Matrices problems, the African students solved an average of 23; East Indian students, 26; and White students, 29 (p<.001), placing them at the 60th, 71st, and 86th percentiles, respectively, and yielding IQ equivalents of 103, 108, and 118 on the 1993 US norms. The same pattern of group differences was found on the Comprehension Test, the Similarities subscale, university course grades, and high‐school grade‐point average. The items on the Matrices ‘behaved’ in the same way for the African students as they did for the non‐African students, thereby indicating the test's internal validity. Item analyses, including a confirmatory factor analysis, showed that the African/non‐African difference was most pronounced on the general factor of intelligence. Concurrent validity was demonstrated by correlating the Matrices with the other measures, both individually and in composite. For the African group, the mean r=.28, p<.05, and for the non‐African group, the mean r=.27, p<.05. Although the intercepts of the regression lines for the two groups were significantly different, their slopes were not. The results imply that scores on the Raven's Matrices are as valid for Africans as they are for non‐Africans.     

Testing Examinees With Handicaps for College and Graduate Admissions

Many colleges and universities require handicapped students to submit scores from admissions tests such as the SAT, ACT, and GRE as part of the application process. Testing handicapped students for admissions purposes raises several critical issues. These issues relate to the accessibility of the admissions testing process, the content of admissions tests, their predictive validity, and the preadmission identification of disabled students.     

Worry and emotionality components of test anxiety

The purpose of the present study was to investigate the relationship between trait measures of Worry and Emotionality and test anxiety related variables. Ninety-eight undergraduate students completed the Test Anxiety Inventory (TAI) and the following self-report measures toward the end of the psychology exam: Checklist of Positive and Negative Thought, Checklist of Body Sensations (CBS), and Subjective Units of Discomfort Scale (SUDS). Further, the students' scores on the psychology exam and the students' grade point averages (GPA) served as outcome measures of test anxiety. In general the results supported the discriminant validity of the Worry and Emotionality subscales of the TAI. Worry was found to be correlated with measures of internal dialogue and Emotionality was found to be correlated with the self-reported arousal measures.     

An evaluation of the yellow brick road test through a full prediction-performance comparison matrix

The purpose of the present study was to evaluate the predictive validity of the Yellow Brick Road Test (YBRT) relative to performance in reading as measured by the Metropolitan Achievement Test (MAT). A full prediction-performance comparison matrix was employed to compare three different cutoff scores for the YBRT for the identification of “at-risk” children. Four hundred and fourteen kindergarten-aged children were administered the YBRT, 214 of whom were followed to the end of second grade. It was concluded that the YBRT is a moderately valid instrument for predicting the level of reading achievement in a population with a different average performance level from the normative sample. However, the results of this study failed to support the utility of the test author's suggested use of a score of 120 as the cutoff score. Finally, it is recommended that a prediction-performance comparison study be considered standard methodology for the evaluation of the validity and efficiency of screening instruments.