Typical intellectual engagement,Big Five personality traits,approaches to learning and cognitive ability predictors of academic performance

Background Both ability (measured by power tests) and non‐ability (measured by preference tests) individual difference measures predict academic school outcomes. These include fluid as well as crystalized intelligence, personality traits, and learning styles. This paper examines the incremental validity of five psychometric tests and the sex and age of pupils to predict their General Certificate in Secondary Education (GCSE) test results. Aims The aim was to determine how much variance ability and non‐ability tests can account for in predicting specific GCSE exam scores. Sample The sample comprised 212 British schoolchildren. Of these, 123 were females. Their mean age was 15.8 years (SD 0.98 years). Method Pupils completed three self‐report tests: the Neuroticism–Extroversion–Openness‐Five‐Factor Inventory (NEO‐FFI) which measures the ‘Big Five’ personality traits, ( Costa & McCrae, 1992 ); the Typical Intellectual Engagement Scale ( Goff & Ackerman, 1992 ) and a measure of learning style, the Study Process Questionnaire (SPQ; Biggs, 1987 ). They also completed two ability tests: the Wonderlic Personnel Test ( Wonderlic, 1992 ) a short measure of general intelligence and the General Knowledge Test ( Irving, Cammock, & Lynn, 2001 ) a measure of crystallized intelligence. Six months later they took their (10th grade) GCSE exams comprising four ‘core’ compulsory exams as well as a number of specific elective subjects. Results Correlational analysis suggested that intelligence was the best predictors of school results. Preference test measures accounted for relatively little variance. Regressions indicated that over 50% of the variance in school exams for English (Literature and Language) and Maths and Science combined could be accounted for by these individual difference factors. Conclusions Data from less than an hour's worth of testing pupils could predict school exam results 6 months later. These tests could, therefore, be used to reliably inform important decisions about how pupils are taught.     

Prediction of job satisfactoriness for workers with severe handicaps from aptitudes,personality, and training ratings

The predictive validity of General Aptitude Test Battery and Sixteen Personality Factor Questionnaire scores were compared to standard training ratings made by vocational instructors against the criterion of work performance measured by the Minnesota Satisfactoriness Scales for a sample of 106 employees with severe handicaps. The psychometric test variables were not correlated with the criterion; however, the training ratings were consistently predictive of the job satisfactoriness scores. These results suggest that the employment potential of job applicants with disabilities can be assessed more accurately using situational training ratings, as opposed to standardized psychometric test scores.     

Responsible Teenage Fatherhood in a South African Historically Disadvantaged Community

The study analysed the relationships between teacher ratings and standardised test scores in Reading and Mathematics for students receiving learning support for learning difficulties (n = 60, mean age= 16.3). Teacher ratings were found to significantly predict pupils' performance in Reading. Students with extended learning support had significantly higher teacher attainment ratings in both Reading and Mathematics. Teacher judgment ratings combined with standardized test scores were reliable for assessing instructional effectiveness. Standardised tests were more reliable than teacher ratings in assessing academic attainment.     

Test anxiety in elementary school and junior high school students in japan

This study describes the sex difference, developmental trends of a modified Japanese version of the Test Anxiety Scale for Children (TASC, Sarason, Davidson, Frederick, & Waite, 1960) and presents evidence of negative effects of test anxiety on academic achievement. It was found that (a) the mean TASC scores for girls were found to be higher than boys across grades; (b) the developmental trends of anxiety scores showed the inversed V curves with the peak of 4th grades in elementary school, but in junior high school the curves were V shaped. Those results were explained in reference with selfdefensiveness, learning of anxiety, resistance to anxiety, and acquisition of self-concept; (c) the test anxiety was shown to have a negative effect on both academic achievement and intelligence test scores. Furthermore, it was to be also found in teacher-made tests that higher test anxiety students were inferior to lower in all the school subjects.     

Text anxiety on cognitive ability test can result in differential predictive validity of academic performance

This paper investigates whether test anxiety leads to differential predictive validity in academic performance. Our results show that the predictive validity of a cognitive ability test, using final exam performance as a criterion, decreased a small amount as Worry (the cognitive aspect of anxiety) increased but was unaffected by Emotionality (the physiological aspect of anxiety). These results suggest that cognitive ability tests may be more useful as predictors of performance for low anxiety test-takers. These findings are discussed in the context of the interference and deficit perspectives of test anxiety.     

对学前儿童语言学习能力诊断量表的效度评价

以所编制的量表为工具 ,对采集的数据进行效度分析 ,结果表明各分测验与全量表有较好的相关 ,说明量表的内容效度是比较高的。使用因素分析的方法 ,将全部变量作系统分类 ,研究量表的结构 ,绝大部分分测验在所得的四个因素上的共通性都大于 0 .70 ;保留下的分测验与所属因素的相关系数在0 .5 3 -0 .84之间 ,它们在各个因素上有较高的负荷量 ,说明量表有较好的结构效度。从效度分析的结果看 ,本量表的测量结果应该是准确的。另外 ,还根据因素分析结果指示的方向 ,调整了分测验 ,调整后的量表结构不但与假设的量表结构十分吻合 ,而且更条理化。     

Consistency in reasoning test scores over time

Background: UK schools have a long history of using reasoning tests, most frequently of Verbal Reasoning (VR), Non‐Verbal Reasoning (NVR), and to a lesser extent Quantitative Reasoning (QR). Results are used for identifying students' learning needs, for grouping students, for identifying underachievement, and for providing indicators of future academic performance. Despite this widespread use there are little empirical data on the long‐term consistency of VR, QR and NVR as discrete abilities. Aims: To evaluate and compare the consistency of VR, QR and NVR scores over a 3‐year period, and to explore the influence of the secondary school on pupils' progress in the tests. Sample: Data were collected on a longitudinal sample of over 10,000 pupils who completed the Cognitive Abilities Test Second Edition in year 6 (age 10+) and year 9 (age 13+), and GCSE public examinations in year 11 (age 15+). Methods: Correlation coefficients and change scores for individual pupils are calculated. Multilevel modelling is used to determine school effects on reasoning scores and GCSE public examination results. Results: The results reveal high correlations in scores over time, ranging from .87 for VR to .76 for NVR, but also show around one‐sixth of pupils on the VR test and one‐fifth of pupils on the QR and NVR tests change their scores by 10 or more standard score points. Schools account for only a small part of the total variation in reasoning score, although they account for a much greater proportion of the variation in measures of attainment such as GCSE. School effects on pupils' progress in the reasoning tests between age 10 and age 13 are relatively modest. Conclusions: Reasoning tests make excellent baseline assessments for secondary schools. Some practical and policy implications for schools are discussed.     

Performance of students with learning disabilities on validity indexes of the Perception of Ability Scale for Students.

The present study examined the performance of 78 students with learning disabilities and 71 normally achieving students in regular Form 1 (Grade 6) classes on three validity indexes of the Perception of Ability Scale for Students, a measure of academic self-concept. The three indexes assess consistency of responding, negative or positive response biases, and misrepresentation of self-perceptions in terms of unrealistic perceptions of perfection in school. Analysis showed that learning disabled students obtained significantly lower Full Scale scores than the normal students, but no significant differences appeared on the three validity indexes. Users of the test can be confident that learning disabled students respond to items in as valid a manner as other students. Having specific learning problems in school should not interfere with response patterns on this scale.     

Air force officer qualifying test validity for predicting pilot training performance

The AFOQT was validated for the prediction of pilot training criteria. Subjects were 7,563 men and women selected for pilot training on the basis of educational attainment and AFOQT scores. Criterion variables included daily flight training grades, check flight grades in subsonic and transonic aircraft, and overall academic performance in the 53 week pilot training course. Test validities were presented as observed, corrected for multivariate range restriction, and corrected for multivariate range restriction and unreliability. The Aviation Information and Instrument Comprehension tests, measures of job knowledge, were most predictive of daily and check flights in the initial subsonic jet aircraft. This reflects the relative greater importance of prior job knowledge early in training. The Scale Reading test, a measure of perceptual speed, was most predictive for daily and check flights in the advanced transonic training aircraft. The Arithmetic Reasoning test, a good measure of general cognitive ability, was most predictive of aeronautics in ground school. The development of an improved pilot selection composite is suggested by the results of the validity analyses.The views expressed are those of the authors and not necessarily those of the Department of the Air Force, Department of Defense, or the Government of the United States.     

Comparison of the Luria-Nebraska Neuropsychological Battery-Children's Revision and the WISC-R with learning disabled children

This investigation concerned the relationship between the Luria-Nebraska Neuropsychological Battery--Children's Revision and the WISC--R for a sample of 32 children identified as learning disabled. The children's mean age was 9 yr., 11 mo.; they were identified as learning disabled on the basis of ability (WISC--R)/achievement discrepancy test scores. The sample was of low average intellectual ability according to the WISC--R and the Luria-Nebraska T-scores. Intercorrelations between scores on the WISC--R and Luria-Nebraska lists were generally nonsignificant, with the exception of language and arithmetic measures on each test. Also, 84% or 27 of the present sample of 32 were correctly identified as learning disabled using a criterion of three or more Luria-Nebraska subscale scores greater than one SD above the mean.     

JOB-RELATEDNESS OF A POSTTRAINING JOB KNOWLEDGE CRITERION USE TO ASSESS VALIDITY AND TEST FAIRNESS

Significant job-relatedness was found for a posttraining job knowledge test criterion using an application of Lawshe's content validity method. The aide test was used as a criterion to assess the predictive validity of a vocabulary test and a civil service test with samples of black ( N = 43) and white ( N = 62) psychiatric aides. Significant validities were found on both tests, but a vocabulary test proved to be the better predictor of the criterion in both samples. The obtained validities were discussed in terms of differential validity, test fairness, and sample size. This study demonstrated that a content validity method could be applied to criteria as well as selection tests. It was concluded that content validity methods may be able to help solve the problem of criterion relevance in validation research by providing quantitative evidence of the job-relatedness of criteria.     

Do subjective measures of attention and memory predict actual performance? Metacognition in older couples

Older individuals who recognize their cognitive difficulties are more likely to adjust their everyday life to their actual cognitive functioning, particularly when they are able to estimate their abilities accurately. We assessed self- and spouse-ratings of memory and attention difficulties in everyday life of healthy, older individuals and compared them with the respective test performance. Eighty-four older individuals (women's age, M = 67.4 years, SD = 5.2; men's age, M = 68.5 years, SD = 4.9) completed both the self and the spouse versions of the Attention Deficit Questionnaire and the Everyday Memory Questionnaire and completed two neuropsychological tests. Using the residual score approach, subjective metacognitive measures of memory and attention were created and compared with actual test performance. Significant associations between subjective and objective scores were found only for men and only for episodic memory measures. Men who underreported memory difficulties performed more poorly; men who overreported memory difficulties performed better. Men's recognition performance was best predicted by subjective measures (R2 = .25), followed by delayed recall (R2 = .14) and forgetting rate (R2 = .13). The results indicate gender-specific differences in metacognitive accuracy and predictive validity of subjective ratings.     

Converging on quality: Examining multiple measures of teaching effectiveness

The present study explores the convergent and predictive validity for several widely used measures of teaching quality from the Measures of Effective Teaching Project (Bill and Melinda Gates Foundation, 2009-2011). Specifically, the Classroom Assessment Scoring System (CLASS; Pianta, Hamre, & Mintz, 2012), the Framework for Teaching (FFT; Danielson Group, 2013), and the Tripod Student Perceptions Scale (Tripod; Ferguson, 2008) were examined. Correlations among measures were assessed by developmental level and content area (elementary mathematics N = 70; elementary English language arts N = 101; middle school mathematics N = 291, middle school English language arts N = 280). Both average scores and score variability (i.e., coefficient of variation) for the CLASS, FFT, and Tripod were used to predict value-added models (VAM), a high-stakes measure of students' academic growth. For elementary mathematics and ELA, findings indicated the CLASS and FFT exhibited moderate convergent validity while divergent validity was found between the Tripod and the CLASS and FFT. Across content areas in middle school grades, the CLASS, FFT, and Tripod exhibited moderate to high-moderate convergent validity. Average student and observer scores were positively related to VAM scores, whereas variability in scores demonstrated negative relations to VAM scores. Implications of findings for teacher evaluation and professional development are discussed.     

Validity of the Rey Visual Design Learning Test in primary and secondary school children.

The Rey Visual Design Learning Test (Rey, 1964, cited in Spreen & Strauss, 1991, Wilhelm, 2004) assesses immediate memory span, new learning, delayed recall and recognition for nonverbal material. Two studies are presented that focused on the construct validity of the RVDLT in primary and secondary school children. In the first study, primary school children performed the RVDLT and the Biber Figural Learning Test, as well as the WISC-R Block design Test, Boston Naming Test, and the Trailmaking Test, to assess discriminant validity. In the second study, the age range was expanded and the subtest Visual Reproductions of the Wechsler Memory Scale with a Delayed recall phase was used to assess the construct validity. A test for visual-motor integration and a test for attention, concentration, and speed of information processing were also added to complete the test battery for assessing discriminant validity. Moderate to high correlations were found between scores on the RVDLT and the tests used to assess construct validity. The correlational pattern of RVDLT scores and the scores on the discriminant tests is discussed.     

Global-local processing impacts academic risk taking

Research has shown that academic risk taking—the selection of school tasks with varying difficulty levels—affords important implications for educational outcomes. In two experiments, we explored the role of cognitive processes—specifically, global versus local processing styles—in students’ academic risk-taking tendencies. Participants first read a short passage, which provided the context for their subsequent academic risk-taking decisions. Following which, participants undertook the Navon’s task and attended to either global letters or local letters only, i.e., were either globally or locally primed. The effects of priming on academic risk taking were then assessed using a perception-based measure (Experiment 1) and a task-based measure (Experiment 2). Experiment 1 provided preliminary evidence, which Experiment 2 confirmed, that globally focused individuals took more academic risk than did locally focused individuals after controlling for participants’ need for cognition (how much they enjoy effortful cognitive activities). Additionally, the inclusion of and comparisons with a control group in Experiment 2 revealed that locally focused participants drove the observed effects. The theory of predictive and reactive control systems (PARCS) provides a cogent account of our findings. Future directions and practical applications in education are discussed.     

网络测验和纸笔测验的测量不变性研究：以生活满意度量表为例

以生活满意度量表为例,运用实证性因素分析,考察在中国文化下网络测验和传统纸笔测验之间的测量不变性。结果显示,网络测验和纸笔测验之间存在弱不变性,即网络测验和纸笔测验有着相同的测量单位;但网络测验和纸笔测验只存在部分的强不变性和部分的严格不变性,测验实施环境对结果的影响不可忽视。该研究表明,恰当设计的网络测验是可靠的,同时还提示,当一个测验在不同情境下运用时,检验测量不变性十分必要     

RETEST EFFECTS IN OPERATIONAL SELECTION SETTINGS: DEVELOPMENT AND TEST OF A FRAMEWORK

This study proposes a framework for examining the effects of retaking tests in operational selection settings. A central feature of this framework is the distinction between within-person and between-person retest effects. This framework is used to develop hypotheses about retest effects for exemplars of 3 types of tests (knowledge tests, cognitive ability tests, and situational judgment tests) and to test these hypotheses in a high stakes selection setting (admission to medical studies in Belgium). Analyses of within-person retest effects showed that mean scores of repeat test takers were one-third of a standard deviation higher for the knowledge test and situational judgment test and one-half of a standard deviation higher for the cognitive ability test. The validity coefficients for the knowledge test differed significantly depending on whether examinees' test scores on the first versus second administration were used, with the latter being more valid. Analyses of between-person retest effects on the prediction of academic performance showed that the same test score led to higher levels of performance for those passing on the first attempt than for those passing on the second attempt. The implications of these results are discussed in light of extant retesting practice.     

中学生学习适应性状况的研究

本研究对广东省五所中学2816名初高中学生进行了学习适应性的问卷调查,结果表明(1)在学习期望和学习意志力分测验中学习适应性不良的检出率高于全国理论比率;(2)在学习动机、学习方法、学校环境、家庭环境以及学习期望上存在显著的性别差异;(3)在学习期望、学校环境、学习意志力分测验中高中生学习适应性不良的检出率显著高于初中生;(4)二类学校学生在各测验中的得分高于一类学校和三类学校,而且检出率低于一类学校和三类学校。     

The team role test: development and validation of a team role knowledge situational judgment test

The main objectives in this research were to introduce the concept of team role knowledge and to investigate its potential usefulness for team member selection. In Study 1, the authors developed a situational judgment test, called the Team Role Test, to measure knowledge of 10 roles relevant to the team context. The criterion-related validity of this measure was examined in 2 additional studies. In a sample of academic project teams (N = 93), team role knowledge predicted team member role performance (r = .34). Role knowledge also provided incremental validity beyond mental ability and the Big Five personality factors in the prediction of role performance. The results of Study 2 revealed that the predictive validity of role knowledge generalizes to team members in a work setting (N = 82, r = .30). The implications of the results for selection in team environments are discussed.