To test or not to test: A clinical dilemma

This paper argues that clinicians are sometimes justified in not testing diagnoses or in not subjecting them to a full battery of tests. In deciding whether to conduct a test, a clinician may consider and weigh several different factors, including her confidence in her initial diagnosis, the specificity and sensitivity of the test, the consequences of making a false diagnosis, the pain, harm, and inconvenience caused by the test, and the costs of the test to the patient and society. This view suggests that diagnoses are fundamentally different from scientific hypotheses in that they are not always subjected to the same evidential standards.     

A two-sample test

In certain investigations involving an experimental group and a control group, the effect of the experimental treatment may be expected to manifest itself by increasing the scores of some subjects but by decreasing the scores of other subjects. Examples arise in the area of defensive behavior. Customary tests of differences between means are inappropriate for assessing the existence of effect of the experimental treatment in such cases. A test based on the ranks of the observations is proposed; it will be sensitive to extreme scores (either large or small or both) for experimentals.     

An automated test development of parallel tests from a seed test

Binary programming models are presented to generate parallel tests from an itembank. The parallel tests are created to match item for item an existing seed test and match user supplied taxonomic specifications. The taxonomic specifications may be either obtained from the seed test or from some other user requirement. An algorithm is presented along with computational results to indicate the overall efficiency of the process. Empirical findings based on an itembank for the Arithmetic Reasoning section of the Armed Services Vocational Aptitude Battery are given.The Office of Naval Research, Program in Cognitive Science, N00014-87-C-0696 partially supported the work of Douglas H. Jones. The Rutgers Research Resource Committee of the Graduate School of Management partially supported the work of Douglas H. Jones and Ing-Long Wu. A Thomas and Betts research fellowship partially supported the work of Ing-Long Wu. The Human Resources Laboratory, United States Air Force, partially supported the work of Ronald Armstrong. The authors benefited from conversations with Dr. Wayne Shore, Operational Technologies, San Antonio, Texas. The order of authors' names is alphabetical and denotes equal authorship.     

The multidimensionality of test anxiety: A test of different models

Abstract

Confirmatory factor analysis was employed to test the multidimensional structure of test anxiety. Starting with the traditional two-factor model, it was of interest whether additional dimensions could still be represented by a single higher-order factor of test anxiety or whether the additional dimensions represented correlates of test anxiety. The Revised Test Anxiety (RTA) scale and the German Test Anxiety Inventory (TAI-G) were combined collecting data from a binational sample of 218 American and 218 German university students. Considering the statistical and theoretical aspects of model fit, they indicated that a model consisting of three primary factors (worry, emotionality, and lack of confidence) fit the data best. While distraction and self-efficacy may be regarded as correlates of test anxiety, lack of confidence was substantiated as a component of test anxiety. Implications for the conceptualization of the content domain and hence area of test anxiety were discussed.     

The effects of immediate forewarning of test difficulty on test performance

Whereas prior research has demonstrated that warning students of an exam's difficulty well in advance of the exam enhances performance, the current research demonstrated the effect of such forewarning immediately before examination administration. Moments before taking an examination in a laboratory environment, participants were either (a) informed that the examination would be difficult, (b) informed that the examination would be easy, or (c) not given any information about the exam's ostensible difficulty. Participants of low trait anxiety performed better when told the exam would be difficult than when told that the exam would be easy. Conversely, participants of high trait anxiety performed worse when told the exam would be difficult than they did when told the exam would be easy or when provided with no such information. Implications and directions for future research are discussed.     

Expectancy of an open-book test decreases performance on a delayed closed-book test

Two experiments examined the influence of practice with, and the expectancy of, open-book tests (students viewed studied material while taking the test) versus closed-book tests (students completed the test without viewing the studied material) on delayed retention and transfer. Using GRE materials specifically designed for open-book testing, participants studied passages and then took initial open- or closed-book tests. Open-book testing led to better initial performance than closed-book testing, but on a delayed criterial (closed-book) test both types of testing produced similar retention after a two-day delay in Experiment 1. In Experiment 2 participants were informed in advance about the type of delayed criterial test to expect (open- or closed-book). Expecting an open-book test (relative to a closed-book test) decreased participants' time spent studying and their delayed test performance on closed-book comprehension and transfer tests, demonstrating that test expectancy can influence long-term learning. Expectancy of open-book tests may impair long-term retention and transfer compared to closed-book tests, despite superior initial performance on open-book tests and students' preference for open-book tests.     

Using the Bender-Gestalt test to predict graphomotor dimensions of the Draw-A-Person test

The investigators examined four correlated aspects of the Bender-Gestalt and the Draw-A-Person tests. Subjects were 41 boys and 14 girls classified as seriously emotionally disturbed or seriously behavior disordered by their school system in southwest Georgia. Each subject's Bender-Gestalt and human figure drawings were placed on a digitizing pad and encoded to provide information relative to the width, height, average point of location on the fourth quadrant abscissa and ordinate of each drawing. The widths of Bender Figures 2 and 8 correlated significantly with the widths of human drawings; the heights of Figures A, 5, and 7 were significantly correlated with the heights of the human drawings. Bender Figure 1 was significantly correlated with average points of location on the abscissa of human figure drawings (distance from the left margin of the page), but correlations between the average points of location from the top of the page were nonsignificant. Comparison of these results with data from other samples might refine diagnosis.     

The Fisher-Pitman permutation test: an attractive alternative to the F test

The Fisher-Pitman permutation test is shown to possess significant advantages over conventional alternatives when analyzing differences among independent samples with unequal variances.     

Teachers' task demands,students' test expectations,and actual test content

Background: Previous studies on instructional importance show that individual students and their teachers differ in the topics that they consider important in the context of an upcoming teacher‐made test. Aims: This study aimed to examine whether such differences between students' test expectations and teachers' intended task demands can be explained by the actual test content. Samples: Participants were history teachers (N=19) and their 11th‐grade students (N=388). Method: Teachers and students rated the importance of text sections that would be tested in the near future. By means of multilevel analysis, ratings were compared with the occurrence of sections in the tests. Results: Although teachers considered a majority of sections as important and tested only a minority of the sections, their tests still included sections rated as unimportant. The number of such discrepancies, however, was relatively small. Sections the teachers rated important had a much higher probability of being included in the test than sections rated unimportant. For students, a similar but lower degree of correspondence between ratings and test content was found. Interestingly, for sections that teachers considered important, students more often gave a higher rating when these sections appeared in the test than when they did not. The same holds for sections that teachers considered unimportant. Conclusions: For both teachers and students there is a limited correspondence between perceived task demands and test content. Furthermore, students' perceptions of task demands show a compensation for some of the differences between their teachers' intended task demands and the test demands.     

The multiple vigilance test

The multiple vigilance test is a 30-min signal detection task designed to assess performance. Sixty target and 180 nontarget stimuli are presented with random interstimulus intervals ranging from 4 to 11 sec. Subjects respond by pressing the spacebar (or microswitch in an alternate version of the program) when they see the masking pattern transformed into the target. This vigilance test is used in conjunction with measures of physiological sleepiness (Multiple Sleep Latency Test) and subjective sleepiness (Stanford Sleepiness Scale). This behavioral information concerning manifest sleepiness provides a more complete picture of a subject’s state of alertness. The assessment of vigilance during long, monotonous, nonstimulating tasks is clinically important in patients with disorders of sleep and arousal.     

The significance test controversy

Summary The pre-designationist, anti-inductivist and operationalist tenor of Neyman-Pearson theory give that theory an obvious affinity to several currently influential philosophies of science, most particularly, the Popperian. In fact, one might fairly regard Neyman-Pearson theory as the statistical embodiment of Popperian methodology. The difficulties raised in this paper have, then, wider purport, and should serve as something of a touchstone for those who would construct a theory of evidence adequate to statistics without recourse to the notion of inductive probability.     

The test validation summary

Common rates employed in classificatory testing are the true positive rate (TPR), false positive rate (FPR), positive predictive power (PPP), and negative predictive power (NPP). FPR and TPR are estimated from research samples representing populations to be distinguished by classificatory testing. PPP and NPP are used by clinicians to classify test takers into populations. PPP and NPP depend on the base rate (BR) of population members in the clinician's sample. The authors introduce the test validation summary (TVS) as a means to report within a single graph the FPR and TPR and the ranges of PPP and NPP across all potential sample BRs for any chosen cut score. The authors investigate how the TVS has other applications, including the estimation of local BR for the condition of interest and the estimation of standard errors for FPR and TPR when estimated across multiple independent validation studies of the classificatory test.