Prospect theory analysis of guessing in multiple choice tests

The guessing of answers in multiple choice tests adds random error to the variance of the test scores, lowering their reliability. Formula scoring rules that penalize for wrong guesses are frequently used to solve this problem. This paper uses prospect theory to analyze scoring rules from a decision‐making perspective and focuses on the effects of framing on the tendency to guess. In three experiments participants were presented with hypothetical test situations and were asked to indicate the degree of certainty that they thought was required for them to answer a question. In accordance with the framing hypothesis, participants tended to guess more when they anticipated a low grade and therefore considered themselves to be in the loss domain, or when the scoring rule caused the situation to be framed as entailing potential losses. The last experiment replicated these results with a task that resembles an actual test. Copyright © 2002 John Wiley & Sons, Ltd.     

Replicability and randomization test logic in behavior analysis

Randomization tests are a class of nonparametric statistics that determine the significance of treatment effects. Unlike parametric statistics, randomization tests do not assume a random sample, or make any of the distributional assumptions that often preclude statistical inferences about single‐case data. A feature that randomization tests share with parametric statistics, however, is the derivation of a p‐value. P‐values are notoriously misinterpreted and are partly responsible for the putative “replication crisis.” Behavior analysts might question the utility of adding such a controversial index of statistical significance to their methods, so it is the aim of this paper to describe the randomization test logic and its potentially beneficial consequences. In doing so, this paper will: (1) address the replication crisis as a behavior analyst views it, (2) differentiate the problematic p‐values of parametric statistics from the, arguably, more useful p‐values of randomization tests, and (3) review the logic of randomization tests and their unique fit within the behavior analytic tradition of studying behavioral processes that cut across species.     

Cerebral lateralization and general intelligence: gender differences in a transcranial Doppler study

The present study evaluated cerebral lateralization during Raven's progressive matrices (RPM) paradigm in female and male subjects. Bilateral simultaneous transcranial Doppler (TCD) ultrasound was used to measure mean blood flow velocities (MBFV) in the right and left middle cerebral arteries (MCAs) in 24 (15 females and 9 males) right-handed normal subjects. The female subjects used a left hemisphere strategy, while males used a right hemisphere strategy to successfully solve RPM tasks. This implies that general intelligence is associated with neural systems within one hemisphere that are accessible to a variety of cognitive processes.     

Analysis of Cognitive Dysfunction in Patients with Low-Grade Glioma

All living adults with histopatologically proven diagnosis of low-grade glioma in a Swedish county were identified with help of the Regional Cancer Register, half of them (n = 24) participated in a neuropsychological evaluation. A considerable variation was found in cognitive function within this group of patients, ranging from good ability to severe disturbance. Different patterns of cognitive dysfunction emerged resulting in three subgroups; patients with severe, mild, and minimal selective dysfunction. The patients with severe disturbance had a global dysfunction covering most assessed cognitive domains. Slow information-processing speed was obvious in the subgroups with both severe and mild dysfunction. Cognitive problems present in the best performing group seemed related to tumor localization. Cognitive function in the whole sample was related to histopathological diagnosis of the tumor, as well as to educational level of the patients. The nonworking patients had significantly poorer performance than the working patients.     

Neuropsychological assessment with the Visual Gestalt Test: psychometric properties and differential diagnostic probabilities

The Visual Gestalt Test is a neuropsychological instrument developed for evaluation of learning and memory of visuo-spatial material. A revised strategy of scoring has motivated the present study, where data from 153 normal persons, 99 epilepsy patients, and 24 depressed patients are presented and compared. The Visual Gestalt Test is observed to discriminate between normal and diagnosed groups in several ways. Additionally it is found to discriminate between depressed and brain damage subgroups of patients. Data are presented in order to supplement previously published ways of scoring and norms. Practical guidelines for the clinical applications of the test are suggested as perspectives.     

Classical Test Theory as a first-order Item Response Theory: Application to true-score prediction from a possibly nonparallel test

We give an account of Classical Test Theory (CTT) in terms of the more fundamental ideas of Item Response Theory (IRT). This approach views classical test theory as a very general version of IRT, and the commonly used IRT models as detailed elaborations of CTT for special purposes. We then use this approach to CTT to derive some general results regarding the prediction of the true-score of a test from an observed score on that test as well from an observed score on a different test. This leads us to a new view of linking tests that were not developed to be linked to each other. In addition we propose true-score prediction analogues of the Dorans and Holland measures of the population sensitivity of test linking functions. We illustrate the accuracy of the first-order theory using simulated data from the Rasch model, and illustrate the effect of population differences using a set of real data.This research is collaborative in every respect and the order of authorship is alphabetical. It was begun when both authors were on the faculty of the Graduate School of Education at the University of California, Berkeley.We would like to thank both Neil Dorans, Skip Livingston and two anonymous referees for many suggestions that have greatly improved this paper.     

The Ecological Validity of Neuropsychological Tests: A Review of the Literature on Everyday Cognitive Skills

Evaluating the ecological validity of neuropsychological tests has become an increasingly important topic over the past decade. In this paper, we provide a comprehensive review of the research on the ecological validity of neuropsychological tests, as it pertains to everyday cognitive skills. This review is presented in the context of several theoretical issues facing ecological validity research. Overall, the research suggests that many neuropsychological tests have a moderate level of ecological validity when predicting everyday cognitive functioning. The strongest relationships were noted when the outcome measure corresponded to the cognitive domain assessed by the neuropsychological tests. Several other factors that may moderate the degree of ecological validity established for neuropsychological tests are in need of further exploration. These factors include the effects of the population being tested, the approach utilized (verisimilitude vs. veridicality), the person completing the outcome measure (significant other vs. clinician), illness severity, and time from injury until evaluation. In addition, a standard measurement of outcome for each cognitive domain is greatly needed to allow for comparison across studies.     

A two-stage algorithm for assessing violations of additivity via axiomatic and numerical conjoint analysis

An algorithm for assessing additivity conjunctively via both axiomatic conjoint analysis and numerical conjoint scaling is described. The algorithm first assesses the degree of individual differences among sets of rankings of stimuli, and subsequently examines either individual or averaged data for violations of axioms necessary for an additive model. The axioms are examined at a more detailed level than has been previously done. Violations of the axioms are broken down into different types. Finally, a nonmetric scaling of the data can be done based on either or both of two different badness-of-fit scaling measures. The advantages of combining all of these features into one algorithm for improving the diagnostic value of axiomatic conjoint measurement in evaluating additivity are discussed.     

Normative data for eight neuropsychological tests based on a Danish sample

Normative neuropsychological data have been provided using a sample of 101 persons aged 20 to 54 years. Eighty-seven of the subjects were recruited among patients who had undergone minor surgery, and 14 subjects were volunteers from the staff at the hospital laundry. The measures consisted of seven subjects from the WAIS-R, the trail-making test, symbol digit modalities test, auditory-verbal learning test, story recall test, visual gestalt test, recurring figures test, verbal fluency, and Purdue pegboard test. These measures were selected because of suitability in regard to time taken for testing each subject and their sensitivity to subtle changes in neuropsychological functioning. The sample was arbitrarily divided into three age groups, and for each of the three samples, mean, standard deviation, and range were calculated for each test. The results of the Information and Vocabulary subtests from the WAIS-R, education, and social status were applied as independent variables in linear regression analyses where each of the neuropsychological tests, in turn, was the dependent variable. For the oldest age group, age in addition was used as an independent variable. Significant linear relationships, which accounted for an optimal part of the variance, were selected for clinical application.