The prediction of intelligence in preschool children using alternative models to regression

Statistical prediction of an outcome variable using multiple independent variables is a common practice in the social and behavioral sciences. For example, neuropsychologists are sometimes called upon to provide predictions of preinjury cognitive functioning for individuals who have suffered a traumatic brain injury. Typically, these predictions are made using standard multiple linear regression models with several demographic variables (e.g., gender, ethnicity, education level) as predictors. Prior research has shown conflicting evidence regarding the ability of such models to provide accurate predictions of outcome variables such as full-scale intelligence (FSIQ) test scores. The present study had two goals: (1) to demonstrate the utility of a set of alternative prediction methods that have been applied extensively in the natural sciences and business but have not been frequently explored in the social sciences and (2) to develop models that can be used to predict premorbid cognitive functioning in preschool children. Predictions of Stanford–Binet 5 FSIQ scores for preschool-aged children is used to compare the performance of a multiple regression model with several of these alternative methods. Results demonstrate that classification and regression treesprovided more accurate predictions of FSIQ scores than does the more traditional regression approach. Implications of these results are discussed.     

Updating the trainability tests literature on Black-White subgroup differences and reconsidering criterion-related validity

A number of applied psychologists have suggested that trainability test Black–White ethnic group differences are low or relatively low (e.g., Siegel & Bergman, 1975), though data are scarce. Likewise, there are relatively few estimates of criterion-related validity for trainability tests predicting job performance (cf. Robertson & Downs, 1989). We review and clarify the existing (and limited) literature on Black–White group differences on trainability tests, provide new trainability test data from a recent video-based trainability exam, and present archival data about how trainability test scores relate to cognitive ability, Black–White differences, and job performance. Consistent with hypotheses, our results suggest large correlations of trainability tests with cognitive ability (e.g., .80) and larger standardized ethnic group differences than previously thought (ds of 0.86, 1.10, and 1.21 for 3 samples). Results also suggest that trainability tests have higher validity than previously thought. Overall, our analysis provides a substantial amount of data to update our understanding of the use of trainability tests in personnel selection.     

A moderate financial incentive can increase effort,but not intelligence test performance in adult volunteers

A positive correlation between self-reported test-taking motivation and intelligence test performance has been reported. Additionally, some financial incentive experimental evidence suggests that intelligence test performance can be improved, based on the provision of financial incentives. However, only a small percentage of the experimental research has been conducted with adults. Furthermore, virtually none of the intelligence experimental research has measured the impact of financial incentives on test-taking motivation. Consequently, we conducted an experiment with 99 adult volunteers who completed a battery of intelligence tests under two conditions: no financial incentive and financial incentive (counterbalanced). We also measured self-reported test-taking importance and effort at time 1 and time 2. The financial incentive was observed to impact test-taking effort statistically significantly. By contrast, no statistically significant effects were observed for the intelligence test performance scores. Finally, the intelligence test scores were found to correlate positively with both test-taking importance (r_c = .28) and effort (r_c = .37), although only effort correlated uniquely with intelligence (partial r_c = .26). In conjunction with other empirical research, it is concluded that a financial incentive can increase test-taking effort. However, the potential effects on intelligence test performance in adult volunteers seem limited.     

PROJECT A VALIDITY RESULTS: THE RELATIONSHIP BETWEEN PREDICTOR AND CRITERION DOMAINS

A predictor battery of cognitive ability, perceptual-psychomotor ability, temperament/personality, interest, and job outcome preference measures was administered to enlisted soldiers in nine Army jobs. These measures were summarized in terms of 24 composite scores. The relationships between the predictor composite scores and five components of job performance were analyzed. Scores from the cognitive and perceptual-psychomotor ability tests provided the best prediction of job-specific and general task proficiency, while the temperament/personality composites were the best predictors of giving extra effort, supporting peers, and exhibiting personal discipline. Composite scores derived from the interest inventory were correlated more highly with task proficiency than with demonstrating effort and peer support. In particular, vocational interests were among the best predictors of task proficiency in combat jobs. The results suggest that the Army can improve the prediction of job performance by adding non-cognitive predictors to its present battery of predictor tests.     

Introduction: A Festschrift in Honor of Earl A. Alluisi

A battery of 10 traditional paper-and-pencil aptitude tests and a battery of 25 cognitive-components-based tests were administered to 298 men and women to investigate the common sources of variance in those batteries. Earlier confirmatory factor analyses showed each battery to have a hierarchical structure, each with a single higher order factor. The higher order factor in the paper-and-pencil battery had previously been identified as general cognitive ability, or g. The higher order factor from the cognitive-components battery had been identified as working memory. The intercorrelation of the higher order factors from the two batteries was .994, indicating that both measured g. The proportion of common variance because of g was greater in the cognitive-components battery than in the paper-and-pencil battery. The correlations between each factor based on cognitive components and g averaged .946. Despite theoretical foundations and arguments, cognitive components tests appear to measure much the same thing as traditional paper-and-pencil tests.     

Using SAT-grade and ability-job performance relationships to test predictions derived from stereotype threat theory

To examine the generalizability of stereotype threat theory findings from laboratory to applied settings, the authors developed models of the pattern of relationships between cognitive test scores and outcome criteria that would be expected if the test scores of women and minority group members were affected by stereotype threat. Two large data sets were used to test these models, one in an education setting examining SAT-grade relationships by race and gender and the other in a military job setting examining Armed Services Vocational Aptitude Battery-job performance relationships by race. Findings were not supportive of the predictions arising from stereotype threat theory, suggesting caution in positing threat as a key determinant of subgroup mean test score differences in applied settings.     

Threat of bodily harm has opposing effects on cognition

Several theoretical frameworks have suggested that anxiety/stress impairs cognitive performance. A competing prediction is made by attentional narrowing models that predict that stress decreases the processing of task-irrelevant items, thus benefiting performance when task-irrelevant information interferes with behavior. Critically, previous studies have not evaluated these competing frameworks when potent emotional manipulations are involved. Here, we used threat of bodily harm preceding a color-word Stroop task to test these claims. We found a basic effect of threat consisting of a slowing down of performance during neutral Stroop trials. Furthermore, both facilitation and interference scores were affected by threat of shock in a way that was consistent with a reduced-distractor effect. Taken together, we interpret our findings in terms of two opposing effects of stress on cognitive performance. Although partly consistent with the attentional narrowing hypothesis, both resource models and cognitive breadth models require revision in order to account for the results.     

Assessing the validity of inferences from scores on the cognitive reflection test

Decision‐making researchers purport that a novel cognitive ability construct, cognitive reflection, explains variance in intuitive thinking processes that traditional mental ability constructs do not. However, researchers have questioned the validity of the primary measure because of poor construct conceptualization and lack of validity studies. Prior studies have not adequately aligned the analytical techniques with the theoretical basis of the construct, dual‐processing theory of reasoning. The present study assessed the validity of inferences drawn from the cognitive reflection test (CRT) scores. We analyzed response processes with an item response tree model, a method that aligns with the dual‐processing theory in order to interpret CRT scores. Findings indicate that the intuitive and reflective factors that the test purportedly measures were indistinguishable. Exploratory, post hoc analyses demonstrate that CRT scores are most likely capturing mental abilities. We suggest that future researchers recognize and distinguish between individual differences in cognitive abilities and cognitive processes.     

Motor assessment of preschool aged children: A preliminary investigation of the validity of the Bruininks–Oseretsky test of motor proficiency – Short form

The aim of this study was to examine the validity of the Bruininks–Oseretsky test of motor proficiency – short form (BOTMP-SF) [Bruininks, R. (1978). Bruininks–Oseretsky test of motor proficiency: Examiners manual. Circle Pines, MN] for the assessment of preschool aged children. Three-hundred and eighteen children 48–71 months old (M = 58.97 months, SD = 6.73) participated in the study. For the data analysis both an ANOVA and a MANOVA were applied with the total battery score and the 14 item scores being the dependent variables, respectively. Age was found to have a significant effect on both children’s total battery score (F(3, 314) = 110.65, p < .001, η² = .68) and 13 item scores (minimum F(3, 314) = 8.75, p < .001, minimum η² = .145). Although the aforementioned results represent an evidence for the validity of the BOTMP-SF, a closer study on the score of each item revealed a high percentage of zero scores on four items. Task difficulty has caused a floor effect, constituting a threat to the validity of the preschoolers’ motor assessment with the above battery. A modification of the battery items is suggested, so that the BOTMP-SF will give valid results for children 4–6 years old.     

Towards a theory of algorithm-determined cognitive test construction

A rationale for, and data from, a trial of a theory of item generation by algorithms whose origins are cognitive models of task performance are presented. Since Spearman (1904), intelligence has been operationally defined and assessed in human subjects by administering identical test items whose content and order have been fixed only after empirical iterations. In our approach, intelligence is ostensively defined by theoretically determined algorithms used for item construction and presentation. Knowledge of what cognitive factors limit human performance makes it possible to vary within tightly specified parameters those features of the tasks that contribute to difficulty, which we call radicals, to let those components of the tasks that do not contribute to difficulty vary randomly, and to counterbalance aspects of answer production that might induce biases of response. Empirical data are based on the generation of five different short tests demanding only functional literacy as a prerequisite for their execution. Four parallel forms of each test were administered to young male Army recruits whose scores were collated with their Army Entrance Test results, which were not previously known to us. Results show that the parallel, algorithm-generated item sets are statistically invariant, which item generation theory demands; and that the individual tests differentially predict Army Entrance Test scores. We conclude that IQ test performances are parsimoniously explained by individual differences in encoding, comparison and reconstructive memory processes.     

Chewing gum while studying: Effects on alertness and test performance

Recent research has demonstrated chewing gum can enhance various cognitive processes associated with learning, but most studies have used cognitive functioning tasks (e.g., selective attention and working memory) as outcomes. Across two experiments, we investigated effects of chewing gum on self‐reports of alertness and test performance following study of realistic educational materials. In Experiment 1 (n = 40), adult participants who chewed gum while studying a 20‐min physiology lesson outperformed a nonchewing condition on subsequent terminology and comprehension tests, but did not report higher levels of postlesson alertness as hypothesised. In Experiment 2 (n = 39), adult participants who chewed gum while studying a 9‐min lesson on a mental mathematics strategy outperformed a nonchewing condition on a subsequent problem‐solving test, whereas also reporting higher levels of postlesson alertness. The results provide initial support for chewing gum while studying realistic educational materials across a range of topics and study durations.     

Showing that the race model inequality is not violated

When participants are asked to respond in the same way to stimuli from different sources (e.g., auditory and visual), responses are often observed to be substantially faster when both stimuli are presented simultaneously (redundancy gain). Different models account for this effect, the two most important being race models and coactivation models. Redundancy gains consistent with the race model have an upper limit, however, which is given by the well-known race model inequality (Miller, 1982). A number of statistical tests have been proposed for testing the race model inequality in single participants and groups of participants. All of these tests use the race model as the null hypothesis, and rejection of the null hypothesis is considered evidence in favor of coactivation. We introduce a statistical test in which the race model prediction is the alternative hypothesis. This test controls the Type I error if a theory predicts that the race model prediction holds in a given experimental condition.     

PREDICTING TRAINING SUCCESS: NOT MUCH MORE THAN g

The roles of general ability ( g ) and specific abilities ( s₁ … s_g ) were investigated in prediction of job-training-school grades. Subjects were 78,041 Air Force enlistees in 82 jobs. General ability and specific abilities were defined by scores on the first and subsequent unrotated principal components of the enlistment selection and classification test, the Armed Services Vocational Aptitude Battery. Linear models analyses revealed that s₁ … s ₉ added little to the prediction afforded by g. It was also determined that a common prediction equation for all jobs was almost as predictive as an equation for each job.     

Comparing neurocognitive impairment in schizophrenia and bipolar I disorder using the Screen for Cognitive Impairment in Psychiatry Scale

The purpose of this study was to compare the psychometric properties of the Screen for Cognitive Impairment in Psychiatry (SCIP) when applied to patients diagnosed with schizophrenia (n = 126) or bipolar I disorder (n = 76), and also to compare the cognitive impairment in both samples of patients and a control group (n = 83) using the SCIP and a complete neuropsychological battery. The SCIP is a scale intended to quickly and easily assess cognitive impairment in patients with severe psychiatric disorders. The results showed firstly that, in terms of internal consistency, temporal stability, dimensional structure, and criterion-referenced validity, the SCIP provides reliable and valid scores at an equivalent level in both schizophrenia and bipolar I disorder samples. Secondly, it showed that differential cognitive impairment between the two patient groups occurs only in verbal memory, although the effect size of the difference is small. Finally, compared with the control group, cognitive impairment was present at all levels in both groups of patients using both the SCIP and the neuropsychological battery, which indicates that the SCIP is a good screening tool for cognitive deficits in schizophrenia and bipolar and useful in clinical practice for healthcare professionals.     

Effects of hypertension on attention, memory, and executive function in older adults.

Hypertension is associated with impairments in cognitive function in older adults, but the nature and extent of these deficits are unclear. Brief cognitive measures lack sensitivity, whereas comprehensive assessments produce numerous variables that are difficult to interpret. The authors performed a principal-components analysis using a computerized cognitive assessment battery and neuropsychological measures of executive function in 506 hypertensive and normotensive older participants. Composite factor scores were used to reanalyze data from 223 untreated participants without vascular complications. The hypertensive group had deficits in Speed of Cognition, Episodic and Working Memory, and Executive Function but not Continuity of Attention. Using composite scores simplified data interpretation and suggested differential effects of hypertension on cognitive performance not clearly evident in individual test results.     

Competing tasks as an index of intelligence

Most studies involving competing (or dual) tasks have been concerned with the investigation of models of attention and have stressed the importance of task characteristics in determining competing-task performance. The relatively few studies which have looked at individual differences in competing-task performance suggest that measures of this performance could reflect operations which are central to cognitive functioning. This paper examines two key questions which stem from this research: is there a separate ability involved in competing-task performance? Is competing-task performance more indicative of general intellectual functioning?A battery composed of both single and competing tasks was presented to 91 Ss. Two sets of scores, ‘primary’ and ‘secondary’, were obtained from the competing tasks. The results indicate that ‘single’ and ‘primary’ scores are basically measuring the same thing but that ‘secondary’ scores measure what is perhaps a time-sharing factor. There is also some evidence that primary and secondary scores are more indicative of the general factor, as measured by this battery, than their single counterparts.     

Are apparent sex differences in mean IQ scores created in part by sample restriction and increased male variance?

This study investigated the possibility that apparent sex differences in IQ are at least partly created by the degree of sample restriction from the baseline population. We used a nationally representative sample, the 1970 British Cohort Study. Sample sizes varied from 6518 to 11,389 between data-collection sweeps. Principal components analysis of scores obtained on four cognitive tests administered at age 10 was used to obtain estimates that we name ‘IQ’. These age-10 scores were then used to estimate the sex differences at age 10, and also among participants in the two later waves, at age 26 and 30. At age 10, there was a small but significant advantage for boys (Cohen’s d = 0.081). Boys had greater variability in these IQ scores. We then investigated how this very small male advantage at 10 changed with sample restriction. We used the same IQs obtained at age 10, but considered only those subjects who returned for data-collection sweeps at ages 26 and 30 years. Subjects returning at age 26 and 30 were more likely to be females and to have higher age-10 IQ scores. Attrition at age 30 was 28% and the male advantage in IQ scores increased by 15%. Attrition at age 26 was 43% and the male advantage in IQ scores increased by 48%. The findings underline the importance of monitoring attrition in longitudinal studies, as well as emphasising the need for representative samples in studying sex differences in intelligence. A proportion of the apparent male advantage in general cognitive ability that has been reported by some researchers might be attributable to the combination of greater male variance in general cognitive ability and sample restriction, though this remains to be tested in a sample with an appropriate mental test battery.     

Scores composites CHC pour le WISC-IV : normes francophones

Although the WISC-IV references the Cattell–Horn–Carroll (CHC) theory in the manual, the composite scores of this battery (VCI, PRI, WMI, and PSI) were not defined according to this model. Nevertheless, we recommend examining the subtests scores of the WISC-IV in reference to the nomenclature of the cognitive abilities proposed in the CHC theory, so as to bring additional insight on the observed performance. The objective of the first part of this paper is to provide normative tables for five CHC cognitive abilities of the French WISC-IV: fluid reasoning (Gf), comprehension–knowledge (Gc), visual processing (Gv), short-term-memory (Gsm), and processing speed (Gs); these tables were created using a statistical approximation procedure. The objective of the second part is to test the validity of these tables with data obtained from 250 children. Correlation between the standard indices of the WISC-IV (VCI, PRI, WMI, and PSI) and the CHC composite scores were high, demonstrating the validity of these CHC scores. These tables, for the French version of the WISC-IV, allow using the CHC composite scores as complementary measures, in order to conduct normative and ipsative analyses.