Letters of Recommendation: Controversy and consensus from expert perspectives

Letters of recommendation (LORs) are a widely used selection tool with many issues associated with their use. To address some of these issues, 575 professionals in personnel‐related professions reported their experiences with LORs. We separated items into consensus, polarized, or neither categories. Experts reached consensus that letter inflation is a problem that may never be alleviated and that more weight is placed on letters written by someone the reader knows or from a prestigious institution or organization. Most items were polarized, suggesting substantial controversy in the field regarding LORs. Some items originally polarized reached consensus within profession (academic vs applied). Academic professionals reported using LORs more and placing more weight on their contents than applied professionals. Implications discussed include recommendations for future research and practice, such as the appropriate use of LORs, LOR formats, and training.     

Conservatism and cognitive ability

Conservatism and cognitive ability are negatively correlated. The evidence is based on 1254 community college students and 1600 foreign students seeking entry to United States' universities. At the individual level of analysis, conservatism scores correlate negatively with SAT, Vocabulary, and Analogy test scores. At the national level of analysis, conservatism scores correlate negatively with measures of education (e.g., gross enrollment at primary, secondary, and tertiary levels) and performance on mathematics and reading assessments from the PISA (Programme for International Student Assessment) project. They also correlate with components of the Failed States Index and several other measures of economic and political development of nations. Conservatism scores have higher correlations with economic and political measures than estimated IQ scores.     

Knowledge and the value of cognitive ability

We challenge a line of thinking at the fore of recent work on epistemic value: the line (suggested by Kvanvig in The value of knowledge and the pursuit of understanding, 2003 and others) that if the value of knowledge is “swamped” by the value of mere true belief, then we have good reason to doubt its theoretical importance in epistemology. We offer a value-driven argument for the theoretical importance of knowledge—one that stands even if the value of knowledge is “swamped” by the value of true belief. Specifically, we contend that even if knowledge itself has no special epistemic value, its relationship to other items of value—cognitive abilities—gives ample reason to locate the concept at the very core of epistemology.     

Situational effects may account for gain scores in cognitive ability testing: A longitudinal SEM approach

Mean gain scores for cognitive ability tests between two sessions in a selection setting are now a robust finding, yet not fully understood. Many authors do not attribute such gain scores to an increase in the target abilities. Our approach consists of testing a longitudinal SEM model suitable to this view. We propose to model the scores' changes of a battery of tests between two sessions with a single factor, namely the change in the situational component of the scores. The situational component encompasses all effects due to the specificity of the state of the person in the current situation (e.g., anxiety level, tiredness, test-taking practice) and is allowed to vary from one session to another. By definition, this single component is supposed to influence all tests at a given session. In particular cases such as high-stake selection settings, where applicants are likely to train themselves before retaking the tests, situational factors might even suffice to explain mean score increases. Empirically, our latent change model closely fitted the scores of 752 applicants for entry into the French Aircraft Pilot Training, gathered on a set of three tests (visual perception, mechanical comprehension, and selective attention). Gain scores of moderate to strong effect sizes could be explained by common situational effects, with no need for admitting change on ability components. Therefore, gain scores may be understood as construct-irrelevant changes.     

Assessing familiality of cognitive ability

In a recent paper in this journal, Benbow, Zonderman, and Stanley (1983) conclude that intellectually precocious children resemble their parents to a lesser extent than do children of lesser ability. In reply, Vining (1985) asserts that Benbow, Zonderman, and Stanley's results are artifacts of selection and their statistical methodology, and that a more appropriate statistical methodology yields quite the opposite conclusion. The present study has two purposes: (1) To show that Vining's criticism is misdirected, stemming from a misunderstanding of how Benbow, Zonderman, and Stanley selected their subjects, and (2) to point out some problems in the model, indices of familiarity, and design used by Benbow, Zonderman, and Stanley that need to be addressed before future comparative studies of familiality are attempted.     

Individual differences in religiosity as a function of cognitive ability and cognitive style

The current study examines the degree to which individual differences in cognitive ability and cognitive style (rational thinking vs. experiential thinking) uniquely and jointly account for differences in religiosity. Using an array of measures of religiosity, results show that cognitive ability has a medium to large negative effect on various aspects of religiosity. Though also negatively related to religiosity, rational thinking style did not add significant unique effects, nor did it convey a significant indirect effect from cognitive ability. Experiential thinking was generally unrelated to ability but was positively related to some aspects of religiosity. Overall the results confirm that those with higher cognitive ability are less likely to accept religious doctrine or engage in religious behaviors and those with lower ability are more likely to accept religious doctrine and exhibit higher levels of fundamentalism. Cognitive style appears to play a lesser role in explaining individual differences in religiosity than cognitive ability.     

Sex differences in the personality and cognitive correlates of spatial ability

Ozer (1987) found that spatial visualization correlated with personality and verbal ability for girls but not for boys. To account for this finding, he hypothesized that "the causes of individual differences in spatial visualization [may be] different for the two sexes" (Ozer, 1987, p. 134). Although Ozer's conclusion is an intriguing one, it should be viewed with at least some skepticism until the discrepancies between his data and those of other researchers have been reconciled.     

Condom use among high-risk adolescents: testing the influence of alcohol use on the relationship of cognitive correlates of behavior.

The goal of this study was the exploration of distal effects of alcohol use on condom use. Criminally involved adolescents completed an initial measure of attitudes, beliefs, and prior behavior. Of the 300 who completed the initial measurement, 267 (89%) completed a behavioral assessment 6 months later. Analyses validated a theoretical model of condom use intentions and indicated that intentions and attitudes measured at baseline were significant predictors of condom use behavior 6 months later. Neither alcohol use nor alcohol problems moderated relationships among model variables or the influence of intentions and attitudes on behavior. The findings do not support a distal role for alcohol use in altering the cognitive correlates of condom use intentions and behavior among high-risk adolescents.     

Recent criticisms of group ability testing in the schools

In a series of articles in Education, the journal of the N.S.W. Teachers' Federation, from March to July 1983, Poulos, Junor and Andreoni severely criticized the use of group verbal general ability tests by the N.S.W. Department of Education. Their criticisms were manifold. Poulos in particular objects to “streaming” or ability grouping. He and Junor condemn general ability tests because they were so often sponsored in the U.S.A. in the second, third and fourth decades of this century by hereditarians and racists and were thought by such sponsors to measure an innate hereditary trait. They fail to mention contrary views propounded in the same period by other psychologists. They quite often follow Kamin in characterizing their bětes noites. All three authors deny the short-term predictive value in an educational setting of group ability tests and exaggerate the long-term influence in N.S.W. of pssessed IQs established in Year 6. The ACER which produces the tests under attack is subjected to a vicious ideological assault. A not very convincing case is made for parents being asked to give permission for the testing of their children; a better case is made for the giving of the results of the testing to the parents (though this is a task for the counsellor rather than the teacher). Andreoni, though rather cavalier with the evidence she uses, probably has a case in urging that the administration of verbal general ability tests to “migrant” children after only 4 years in the local schools is too soon for a valid assessment of current intellectual functioning without taking many other factors into account.     

Test bias and ability level testing

The average grade equivalent reading comprehension scores of students in 50 black schools were compared to those of students in 50 white schools under two forms of test administration. Traditional grade level testing resulted in smaller group differences between blacks' and whites' scores than the apparently more valid ability level testing technique. It was concluded that the use of grade level testing with the Iowa Tests of Basic Skills is biased in favor of low scoring subgroups.     

Educational attainment as a proxy for cognitive ability in selection: effects on levels of cognitive ability and adverse impact

The authors examined the differences in mean level of cognitive ability and adverse impact that can be expected when selecting employees solely on educational attainment as a proxy for cognitive ability versus selecting employees directly on cognitive ability. Selection using cognitive ability worked as a more efficient cognitive screen. Imposing an educational attainment standard of at least 1 year of college, though, did result in noticeably higher levels of cognitive ability in potential applicant pools than did random selection, meaning that educational attainment does work as a cognitive screen. These results held not only in a nationally representative sample but also within and across 6 different occupational groups. Finally, adverse impact is examined for selection using educational attainment, compared with selection on the basis of cognitive ability.     

Comparison of the WISC-R and Woodcock-Johnson tests of cognitive ability

The relationship between the Woodcock-Johnson Tests of Cognitive Ability (WJTCA) and Wechsler Intelligence Scale For Children-Revised (WISC-R) was investigated in a referral sample of 52 elementary aged students. The WJTCA and WISC-R provided comparable ability estimates as reflected by a .74 correlation between the respective Full Scale scores and no significant (.05 level) mean difference (WJTCA Broad Cognitive Ability mean = 97.1, WISC-R Full Scale mean = 96.8). The comparability of the WJTCA/WISC-R global ability estimates is contrary to lower WJTCA scores found in recent studies with learning disabled students, with the current findings supporting the criticism of those studies as advanced by the author of the WJTCA.     

On the relative independence of thinking biases and cognitive ability

In 7 different studies, the authors observed that a large number of thinking biases are uncorrelated with cognitive ability. These thinking biases include some of the most classic and well-studied biases in the heuristics and biases literature, including the conjunction effect, framing effects, anchoring effects, outcome bias, base-rate neglect, "less is more" effects, affect biases, omission bias, myside bias, sunk-cost effect, and certainty effects that violate the axioms of expected utility theory. In a further experiment, the authors nonetheless showed that cognitive ability does correlate with the tendency to avoid some rational thinking biases, specifically the tendency to display denominator neglect, probability matching rather than maximizing, belief bias, and matching bias on the 4-card selection task. The authors present a framework for predicting when cognitive ability will and will not correlate with a rational thinking tendency.     

Comparing single-pool and multiple-pool designs regarding test security in computerized testing

This article compares the use of single- and multiple-item pools with respect to test security against item sharing among some examinees in computerized testing. A simulation study was conducted to make a comparison among different pool designs using the item selection method of maximum item information with the Sympson-Hetter exposure control and content balance. The results from the simulation study indicate that two-pool designs have a better degree of resistance to item sharing than do the single-pool design in terms of measurement precision in ability estimation. This article further characterizes the conditions under which employing a multiple-pool design is better than using a single, whole pool in terms of minimizing the number of compromised items encountered by examinees under a randomized item selection method. Although no current computerized testing program endorses the randomized item selection method, the results derived in this study can shed some light on item pool designs regarding test security for all item selection algorithms, especially those that try to equalize or balance item exposure rates by employing a randomized item selection method locally, such as the a-stratified-with-b-blocking method.