Maximizing the discriminating power of a multiple-score test

Maximizing the discriminating power of a multiple-score test involves maximizing the homogeneity of each subtest and minimizing the correlations between subtests. A method is presented for constructing such tests from items whose intercorrelations are not too high. Under certain restrictions the saturation, defined as the proportion of inter-item covariance to total variance, is maximized for each subtest. The nucleus of each subtest is three items with high covariancesinter se. All items which will lower the saturation are discarded; the one item is added which will maximize the saturation of the resultant test. This process is repeated until all the items are included or discarded for that subtest. If the correlation between any such subtests approaches the geometric mean of their saturations, their items form a new pool for one or more subtests. Formulas are presented for deciding which items to eliminate in order to reduce further the correlations between subtests.This research was supported in part by the United States Air Force under Contract AF 33(038)-10588 with Human Resources Research Center, Lackland Air Force Base, San Antonio, Texas. Permission is granted for reproduction, translation, publication use and disposal in whole and in part by or for the United States Government.     

Decreasing scoring errors on Wechsler Scale Vocabulary, Comprehension, and Similarities subtests: a preliminary study

Studies of graduate students learning to administer the Wechsler scales have generally shown that training is not associated with the development of scoring proficiency. Many studies report on the reduction of aggregated administration and scoring errors, a strategy that does not highlight the reduction of errors on subtests identified as most prone to error. This study evaluated the development of scoring proficiency specifically on the Wechsler (WISC-IV and WAIS-III) Vocabulary, Comprehension, and Similarities subtests during training by comparing a set of 'early test administrations' to 'later test administrations.' Twelve graduate students enrolled in an intelligence-testing course participated in the study. Scoring errors (e.g., incorrect point assignment) were evaluated on the students' actual practice administration test protocols. Errors on all three subtests declined significantly when scoring errors on 'early' sets of Wechsler scales were compared to those made on 'later' sets. However, correcting these subtest scoring errors did not cause significant changes in subtest scaled scores. Implications for clinical instruction and future research are discussed.     

Abbreviated form of the Wechsler Memory Scale-III Faces subtest

A major criticism of the Wechsler Memory Scale-III Faces subtest is the number of items, which can be daunting and time-consuming for an impaired client or boring for a normal client. An analysis of several versions, with data from a sample of 50 clinical referrals, revealed that a 32-item subtest was best overall. Using this version, 100% (Faces I) and 94% (Faces II) of the predicted raw scores were less than 4 points away from the actual raw scores, whereas 66% (Faces I) and 70% (Faces II) were less than 2 points away. Limitations of this procedure are discussed.     

WAIS-III task perception and performance as a function of stopwatch display

Examiner's instructions for the Wechsler Adult Intelligence Scale-Third Edition indicate that stopwatch display is optional. Since verbal instructions do not consistently inform examinees whether a subtest is timed, examinees may rely upon the visual cue of the stopwatch. Furthermore, examinees' understanding of the actual subtest demands may affect their performances. In the present study, 58 volunteers were randomly assigned to two groups, one in which the examiner displayed the stopwatch and one in which the stopwatch was concealed. Participants were administered the first five subtests. Predictions were that (1) participants to whom the stopwatch was displayed would more accurately perceive whether the subtests were timed, and (2) participants who accurately understood the demands of subtests would score higher than those who did not. Results supported the first hypothesis for three subtests out of five, one remaining inconclusive due to a ceiling effect. The second hypothesis was not supported. Both the implications and limitations of the present study are discussed.     

Test reliability and effective test length

Measures of effective test length are developed for speeded and power tests, which are independent of the number of items in the test or of the time required for administration. These measures are used in determining reliability for (1) speeded and power tests, where a separately timed short parallel form is administered in addition to the full-length test; (2) power tests, where a subset of items is imbedded within the total test, parallel to the total test; and (3) power tests, where the subset of items is correlated with the complementary parallel subset in the test.     

非词复述测验的编制和初步分析

本研究编制了一种用于测量儿童语音工作记忆的测验———非词复述测验,并在48名四年级小学生中初步进行信度、效度检验和项目分析。结果表明:(1)该测验有较高的重测信度;(2)该测验具有较高的结构效度和效标效度;(3)分测验二的项目难度分布比较合理,多数项目鉴别力较高,而分测验一的项目难度分布和项目鉴别力则有待于在今后的研究中进一步提高。     

Statistical significance and reliability of single-subtest and cluster deviation scores

Suppose one has a battery of K subtests and a composite for the battery is defined as the mean of the K standardized subtest scores. An individual's single-subtest deviation score is the difference between the individual's score on any single subtest and his composite score. A cluster deviation score is the difference between an examinee's average for a small set (cluster) of subtests and his composite. Formulas are given for the test of statistical significance of the individual's subtest or cluster deviation score and the internal consistency reliability of such deviation scores.     

Speed of emotional information processing and emotional intelligence

The present study aimed to investigate the relationship between the speed of emotional information processing and emotional intelligence (EI). To evaluate individual differences in the speed of emotional information processing, a recognition memory task consisted of two subtests similar in design but differing in the emotionality of the stimuli. The first subtest required judgment about whether an emotional facial expression in the test face was identical to one of the four emotional expressions of the same individual previously presented. The second subtest required deciding whether the test face with a neutral emotional expression was identical to one of the four neutral faces of different individuals previously presented. Mean response latencies were calculated for “Yes” and “No” responses. All latencies were correlated with other measures of processing speed such as discrimination time and time of figure recognition. However, the emotional expression recognition subtest was hypothesized to require the processing of emotional information in addition to that of facial identity. Latencies in this subtest were longer than those in the face recognition subtest. To obtain a measure of the additional processing that was called for by the emotionality of the stimuli, a subtraction method and regression analysis were employed. In both cases, measures calculated for “No” responses were related to ability EI, as assessed via a self‐report questionnaire. According to structural equation modeling, there was a moderately negative association between latent EI and the latency of “No” responses in the subtest with emotional stimuli. These relationships were not observed for “Yes” responses in the same subtest or for responses in the subtest with neutral face stimuli. Although the differences between “Yes” and “No” responses in their associations with EI require further investigation, the results suggest that, in general, individuals with higher EI are also more efficient in the processing of emotional information.     

THE BRITISH ABILITY SCALES SPEED OF INFORMATION PROCESSING SUBTEST: WHAT DOES IT MEASURE?

Summary . Recent theory and research has focused on the relationship between speed of performing elementary cognitive operations and general intelligence. The developers of the British Ability Scales (BAS) included the Speed of Information Processing (SOIP) subtest as a measure of mental processing speed. To test the validity of the SOIP subtest, a group of 12-year-old children were given the BAS short-form, including the SOIP subtest, and a series of electronically timed reaction time (RT) tasks. Correlations between RT and SOIP were higher than correlations between RT and other BAS subtests, suggesting convergent and discriminant validity. A hierarchical factor analysis of the data shows that the BAS SOIP subtest is related more to speed of apprehension (RT) than to speed of making the motor response, or movement time (MT).     

Wechsler Digit Span subtest: diagnostic usefulness with dyslexic children

The current study supports Levinson's contention (1988) that the diagnosis of dyslexia cannot be based solely on neurophysiological evidence but must be made in conjunction with reading and educational test scores. Contrary to much current research, in the present study specific WISC-R profiles are identified within a stringently defined subgroup of 57 9- to 12-yr-old dyslexic children. Well-defined subtest scatter can differentiate dyslexics from proficient readers. In particular, the Digit Span subtest, which is not routinely administered by psychologists or included in computing Verbal IQ, can be considered an important component of a diagnostic battery. Unexpectedly, Coding emerged with Digit Span as a third factor in a principal component analysis; statistically significant sex differences appeared on the Coding task. Findings appear to confirm the phonological encoding deficiencies displayed by dyslexics on the Digit Span subtest. This study strongly supports consideration of WISC-R subtest differences, along with correlated factors, neurophysiological and perceptual evidence, when diagnosing dyslexic children. Cross-validation is planned.     

Parallel-form reliability of the Wechsler Memory Scale in a geriatric population with suspected dementia

The present study examined parallel-form reliability of the Wechsler Memory Scale (WMS) in a population where such forms are particularly useful, that is, among patients who are suspected of having some type of dementing illness. Comparative analyses were made for total raw scores. Memory Quotient (MQ) scores, and individual subtest scores. Thirty geriatric patients originally referred for neuropsychological testing because of suspected dementia were given both Forms 1 and 2 of the WMS. The results yielded high overall reliability coefficients for MQ and total raw scores. Individual subtest scores also yielded rather high coefficients, and only one subtest (Associate Learning) had a significant mean difference. Apparently, the two forms have sufficient parallel-form reliability to be considered clinically interchangeable. Thus, the WMS may be a useful tool for repeat evaluations of memory functioning in a demented population.     

The role of verbal and nonverbal memory in the Family Pictures Subtest: Data from children with specific language impairment

This study examined the contribution of verbal and visual memory to performance on the Family Pictures subtest of the Children’s Memory Scale. This subtest purports to assess declarative memory functioning in the visual/nonverbal domain. A total of 115 nine-year-old children participated in this study. Fifty-eight had specific language impairment (SLI), whilst the remaining 57 were typically developing (TD), with no history of language difficulties. Results showed that the children with SLI, who had intact declarative memory for visual but not verbal information, obtained significantly lower scores on the Family Pictures subtest when compared to the TD group. Regression analyses revealed that across the entire sample, individual differences on the Family Pictures subtest was best predicted by a measure of verbal working memory. These results question whether the Family Pictures subtest can be considered a measure of visual memory in pediatric populations. These results have implications for the interpretation of scores on this subtest regarding the nature of the types of neurocognitive difficulties children may exhibit.     

Repressive Adaptation in a Nonclinical Sample: Consistent and Inconsistent Reports of Child Psychosocial Functioning Across Informants

A method is described which utilizes the Picture Arrangement subtest of the Wechsler Scales as a technique in the investigation of fantasy material. The method can be readily adapted to the quantitative and qualitative analyses which are suitable for other thematic tools. A case history was presented illustrating the method and its integration with the results of the intelligence test itself.     

Sex Differences in Mathematics Components of the Iowa Tests of Basic Skills

The Mathematics Problem Solving (MRS) and Mathematics Concepts (MC) subtests of the Iowa Tests of Basic Skills were investigated for content and psychometric item bias at grades 3, 6, and 8. A small proportion of items were identified in each subtest which significantly favored either males or females. No skill classification, item content or location trends could be found for the mathematics subtests at each grade level. Across the grade levels, items in the MC subtest favored males for grades 3 and 6, but females were favored at grade 8. The procedure used in the study is generalizable to other groups (minority or grade levels). Test consumers have the right to know whether the test they use is fair for selected groups of students. Results from empirical investigations should appear in the Test Manual that accompanies the test battery.     

The relation between the difficulty and the differential validity of a test

Using scores of 1200 students on a long test as a criterion, each of five subtests of different difficulty has maximum correlation with the criterion when the criterion is dichotomized at a value appropriate to the difficulty of the subtest. A 50-item test element is scored on an all-or-none basis with different standards for passing, and the percentage of passes for successive points on the criterion variable is computed. The Constant Method is applied to this relationship. The limen thus computed is a measure of difficulty, the dispersion is a measure of average (or total) validity, and the slope of the curve is a measure of differential validity. The difficulty of a test element is thus directly related to the maximum differential validity.     

Factor structure of the Norwegian version of the WAIS-III in a clinical sample: the arithmetic problem

Confirmatory Factor Analyses (CFA) of the Wechsler Adult Intelligence Scale-III (WAIS-III) lend partial support to the four-factor model proposed in the test manual. However, the Arithmetic subtest has been especially difficult to allocate to one factor. Using the new Norwegian WAIS-III version, we tested factor models differing in the number of factors and in the placement of the Arithmetic subtest in a mixed clinical sample (n = 272). Only the four-factor solutions had adequate goodness-of-fit values. Allowing Arithmetic to load on both the Verbal Comprehension and Working Memory factors provided a more parsimonious solution compared to considering the subtest only as a measure of Working Memory. Effects of education were particularly high for both the Verbal Comprehension tests and Arithmetic.     

行政职业能力测验阅读理解能力考查体系研究

本研究对行政职业能力测验中的阅读理解分测验考查何种能力要素进行了深入探讨.研究一首先通过文献法初步概括出行政职业能力测验阅读理解的能力考查体系,进而使用出声思维的方法对该考查体系进行验证和补充,最后通过专家调查论证方法对其进行最终确定.结果表明,行政职业能力测验阅读理解考查了6种能力,分别是提取信息能力、理解意义能力、理解细节能力、分析结构能力、概括能力及推断能力,其中提取信息能力和理解意义能力是基础.在此基础上,研究二使用该能力框架选取合适的行政职业能力测验阅读理解题目,组成一套阅读理解测验,并借助认知诊断方法,通过专家标定测验的Q矩阵,利用实测数据对研究一构建的能力考查体系的完备性进行了验证.     

Incremental validity of WISC-III profile elevation, scatter, and shape information for predicting reading and math achievement

The use of cognitive subtest profiles to hypothesize about children's learning strengths and weaknesses implicitly assumes that subtest profiles are predictive of academic performance. To test this assumption, Wechsler Intelligence Scale for Children-Third Edition (WISC-III) subtest profiles were decomposed into elevation, scatter, and shape components and sequentially regressed onto reading and math achievement scores for 1,118 nonexceptional and 538 exceptional students. Profile elevation was statistically and practically significant for both exceptional (R = .36-.61) and nonexceptional (R = .72-.75) students. Profile scatter did not aid in the prediction of achievement. Profile shape accounted for an additional 5%-8% of the variance in achievement measures. It was concluded that using WISC-III subtest scatter and shape to predict academic performance was not supported by the accumulated scientific evidence.     

Change and stability in adult intellectual functioning assessed by Wechsler item responses

Change in adult intellectual performance was assessed with longitudinal data from the Intergenerational Studies at the Institute of Human Development. Wechsler Intelligence data from two age cohorts spanning ages 18 to 61 were analyzed at the subtest and item level. Hotelling T2 analyses on sets of equivalent items from Wechsler subtests were studied to determine if change in response occurred between pairwise combinations of occasions of test administrations. We used Bowker's test to analyze data at the item level to determine the direction of change in performance. Consistent improvement in performance occurred between the ages of 18-40 and 18-54. Between the ages of 40 and 61, results showed mostly improved performance on the Information, Comprehension, and Vocabulary subtests, mixed change on the Picture Completion subtest, and decline on the Digit Symbol and Block Design subtests. The pattern of mixed change on the Picture Completion subtest indicated improvement on the easy items and decline on the difficult items. Decline in performance on the Block Design test occurred only for the most difficult items.