Test-retest reliability and practice effects of the Wechsler Memory Scale-III

Although serial administration of cognitive tests is increasingly common, there is a paucity of research on test-retest reliabilities and practice effects, both of which are important for evaluating changes in functioning. Reliability is generally conceptualized as involving short-lasting changes in performance. However, when repeated testing occurs over a period of years, there will be some longer lasting effects. The implications of these longer lasting effects and practice effects on reliability were examined in the context of repeated administrations of the Wechsler Memory Scale-III in 339 community-dwelling women aged 40-79 years over 2 to 7 years. The results showed that Logical Memory and Verbal Paired Associates subtests were consistently the most reliable subtests across the age cohorts. The magnitude of practice effects varied as a function of subtests and age. The largest practice effects were found in the youngest age cohort, especially on the Faces, Logical Memory, and Verbal Paired Associates subtests.     

Test reliability and correction for attenuation

Evidence is cited to show that specificity, or lack of equivalence, in the comparable forms of tests has a tendency to lower the value of reliability coefficients but has no tendency to lower the value of observed trait coefficients. This implies that the greater the lack of equivalence, the higher will be coefficients corrected for attenuation. Errors of measurement are supposed to reduce the magnitude of observed trait coefficients. Since specificity does not lower the correlation between two tests and since the split-half and equivalent-form reliability coefficients treat specificity as error, it follows that these two coefficients cannot legitimately be used in Spearman's correction-for-attenuation formula.     

Test reliability and effective test length

Measures of effective test length are developed for speeded and power tests, which are independent of the number of items in the test or of the time required for administration. These measures are used in determining reliability for (1) speeded and power tests, where a separately timed short parallel form is administered in addition to the full-length test; (2) power tests, where a subset of items is imbedded within the total test, parallel to the total test; and (3) power tests, where the subset of items is correlated with the complementary parallel subset in the test.     

The reliability of the Thematic Apperception Test

Controversy over the TAT's reliability may stem largely from the mis-application of traditional psychometric measures, which are inappropriate to this test. The TAT is implicitly based on a multiple regression model, for which coefficient alpha is not appropriate. Also, test-retest correlations may be adversely affected by the standard instructions to write a "creative" story. In a test-retest study, 47 high school students retook the TAT after a year with instructions designed to break the implicit set to produce a new and different story from that previously written. The test-retest correlations were r = .48 (need for affiliation) and .56 (need for intimacy), or approximately the same as those for, e.g., the MMPI, 16PF, and CPI, It was demonstrated that this high stability over time was not due to subjects' recalling and repeating previous responses. Finally, it was shown that alpha considerably underestimated the test-retest reliability, contrary to assumptions of classical psychometrics.     

Test reliability estimated by analysis of variance

A formula for estimating the reliability of a test, based on the analysis of variance theory, is developed and illustrated. The data needed for the required computation are the number of correct responses to each item and the score for each subject. The results obtained from this formula are identical with those from one of the special cases of the Kuder-Richardson formulation. The relationships of the new procedure to other approaches to the problem are indicated.     

Comparability, reliability, and practice effects on alternate forms of the Digit Symbol Substitution and Symbol Digit Modalities tests

The present study examined the comparability of 4 alternate forms of the Digit Symbol Substitution test and the Symbol Digit Modalities (written) test, including the original versions. Male contact-sport athletes (N = 112) were assessed on 1 of the 4 forms of each test. Reasonable alternate form comparability was demonstrated through establishing normality of form distributions and conducting pairwise form comparisons of means, variability, and intraclass correlations. Nonetheless, alternate forms are likely an insufficient means of controlling practice in speeded measures at brief (1-2 weeks) retest intervals. Reliable change indices demonstrated that practice must be accounted for in individual retesting.     

Test Anxiety Scale: reliability among Ethiopian students

Sarason's Test Anxiety Scale, translated into an Ethiopian language, was administered to 391 students in Grade 8 and to 422 students in preparatory school (Grades 11 and 12). In the first sample, 32 items loaded above the 0.3 criterion of acceptable item-remainder correlations and Cronbach alpha of .84. In the second sample, Cronbach alpha was .84 for the 34 items, but only 19 items had acceptable item-remainder correlations. The internal consistency reliabilities were comparable with those reported in the literature. However, the results of confirmatory factor analyses with extraction of four factors did not confirm the item loadings on factors as reported in the literature. Younger students (Grade 8) were found to have higher mean Test Anxiety than Grades 11 and 12 students. The Amharik version of the Test Anxiety Scale as a whole could be considered reliable and useful for Ethiopian students.     

Test of reliability of perception of parental and childhood illness behavior

This report details the reliability of perceived parental and childhood illness behavior. Three versions of the Illness Behavior Inventory were created to assess perceived illness behavior of one's mother, father, and oneself as a child. The measures were administered twice to 32 students of linguistics at a major university with a 2-wk. interval between administrations. Each measure across administrations correlated highly and significantly (.98 to .99). It was concluded that perceptions of parental and childhood illness behavior are reliable over time but their sensitivity to actual historical events remains an empirical question.     

Test–retest reliability of an emotion maintenance task

Research on working memory has suggested domain-specific components for visual, verbal, and spatial information, and more recently for emotion. Affective working memory has been proposed as the set of processes involved in the maintenance of emotions to guide behaviour. The current study examined the reliability of an emotion maintenance/affective working memory task over two experimental sessions separated by one week. Subjective accuracy based on individual ratings was found to correlate over time and was highest for negatively valenced pictures. Results suggest that this paradigm is a reliable measure of emotion maintenance, underscoring the utility of this measure as an assessment tool for normative and clinical populations.     

Test-retest reliability of the Hand Test with normal subjects

The Hand Test (Wagner, 1962) was administered to 71 subjects; 14 days later these subjects were again administered the Hand Test. Results indicated the Hand Test is a highly reliable measure of an individual's behavioral action tendencies.     

Event-related potential practice effects on the Paced Auditory Serial Addition Test (PASAT)

Practice can change the nature and quality of a stimulus-response relationship. The current study observed the effects of repeated administration of the Paced Auditory Serial Addition Test (PASAT) in 12 healthy individuals, in an effort to establish distinct profiles associated with novel and practiced processing. Over four training sessions the mean number of correct responses on this demanding test of attention significantly improved and was approaching ceiling for most task conditions. Behavioural improvements were associated with significantly reduced amplitude of late Processing Negativity, a frontally distributed component of the event-related potential waveform associated with voluntary, limited-capacity activity within higher-order attentional systems. These results suggest that PASAT performance became more efficient as practice seemingly eased the strategic planning and coordination requirements the task places on frontally-mediated executive attention resources. The findings of the current study extend our understanding of the functional and behavioural mechanisms underlying the effects of practice.     

The effects of interleaved practice

Previous research shows that interleaving rather than blocking practice of different skills (e.g. abcbcacab instead of aaabbbccc) usually improves subsequent test performance. Yet interleaving, but not blocking, ensures that practice of any particular skill is distributed, or spaced, because any two opportunities to practice the same task are not consecutive. Hence, because spaced practice typically improves test performance, the previously observed test benefits of interleaving may be due to spacing rather than interleaving per se. In the experiment reported herein, children practiced four kinds of mathematics problems in an order that was interleaved or blocked, and the degree of spacing was fixed. The interleaving of practice impaired practice session performance yet doubled scores on a test given one day later. An analysis of the errors suggested that interleaving boosted test scores by improving participants' ability to pair each problem with the appropriate procedure. Copyright © 2009 John Wiley & Sons, Ltd.     

Measures of concurrent validity and alternate-form reliability of the Test of Nonverbal Intelligence

The subjects, 60 undergraduate students, were administered the Test of Nonverbal Intelligence (TONI) individually. The Shipley Institute of Living Scale was administered in small groups. A Pearson correlation of .56 was obtained for TONI Quotients, Forms A and B. TONI Quotients, Forms A and B, correlated with Shipley estimated WAIS--R IQ .50 and .46, respectively, and correlated to .71 and .64, with Shipley Total T scores, .52 and .44, respectively (corrected to .71 and .61), with Shipley Abstraction T scores, .51 and .42, respectively (corrected, .63 and .52), and with Shipley Vocabulary T scores .26 and .32, respectively (corrected to .63 and .52). TONI scores seem more closely related to Shipley Total and Abstraction scores than to Shipley Vocabulary.     

Tactual Performance Test: internal consistency reliability of the memory and location scores

Internal consistency reliabilities were computed for the Tactual Performance Test Memory and Location scores (N=602). After adjusting for unequal item difficulty, the reliabilities for Memory and Location were .69 and .79, respectively.     

Test-retest reliability in fMRI of language: group and task effects

This paper explores how the test-retest reliability is modulated by different groups of participants and experimental tasks. A group of 12 healthy participants and a group of nine stroke patients performed the same language imaging experiment twice, test and retest, on different days. The experiment consists of four conditions, one audio condition and three audiovisual conditions in which the hands are either resting, gesturing, or performing self-adaptive movements. Imaging data were analyzed using multiple linear regression and the results were further used to generate receiver operating characteristic (ROC) curves for each condition for each individual subject. By using area under the curve as a comparison index, we found that stroke patients have less reliability across time than healthy participants, and that when the participants gesture during speech, their imaging data are more reliable than when they are performing hand movements that are not speech-associated. Furthermore, inter-subject variability is less in the gesture task than in any of the other three conditions for healthy participants, but not for stroke patients.