Statistical Tests of Conditional Independence Between Responses and/or Response Times on Test Items

Three plausible assumptions of conditional independence in a hierarchical model for responses and response times on test items are identified. For each of the assumptions, a Lagrange multiplier test of the null hypothesis of conditional independence against a parametric alternative is derived. The tests have closed-form statistics that are easy to calculate from the standard estimates of the person parameters in the model. In addition, simple closed-form estimators of the parameters under the alternatives of conditional dependence are presented, which can be used to explore model modification. The tests were applied to a data set from a large-scale computerized exam and showed excellent power to detect even minor violations of conditional independence.     

Normative data for eight neuropsychological tests based on a Danish sample

Normative neuropsychological data have been provided using a sample of 101 persons aged 20 to 54 years. Eighty-seven of the subjects were recruited among patients who had undergone minor surgery, and 14 subjects were volunteers from the staff at the hospital laundry. The measures consisted of seven subjects from the WAIS-R, the trail-making test, symbol digit modalities test, auditory-verbal learning test, story recall test, visual gestalt test, recurring figures test, verbal fluency, and Purdue pegboard test. These measures were selected because of suitability in regard to time taken for testing each subject and their sensitivity to subtle changes in neuropsychological functioning. The sample was arbitrarily divided into three age groups, and for each of the three samples, mean, standard deviation, and range were calculated for each test. The results of the Information and Vocabulary subtests from the WAIS-R, education, and social status were applied as independent variables in linear regression analyses where each of the neuropsychological tests, in turn, was the dependent variable. For the oldest age group, age in addition was used as an independent variable. Significant linear relationships, which accounted for an optimal part of the variance, were selected for clinical application.     

Reaction time correlations with intelligence test scores obtained under either timed or untimed conditions

One hundred and thirteen high school students were randomly assigned to one of two groups that were administered an intelligence test (the Multidimensional Aptitude Battery) under either timed or untimed conditions. Subsequently, all subjects were given a battery of 8 reaction time tests. Multiple regression analyses showed that combinations of RTs were approximately equally good predictors of IQ scores in both groups. Zero-order correlations between each RT test and IQ scores were also approximately equal for timed and untimed Verbal and Full-Scale scores but correlations with Performance scores were higher in the timed condition. The extent to which the different RT tests correlated with timed scores was quite highly related to the tests' loadings on a general speed factor but these loadings were not related to the tests' correlations with untimed scores. Rather, the relative complexity of the RT tests had a stronger influence on their correlations with IQ scores in the untimed condition. It is concluded that timed and untimed intelligence tests impose different information-processing demands on subjects but that the speed with which subjects can cope with these demands is equally important in both conditions.     

《中小学生注意力测验》全国常模制定报告

《中小学生注意力测验》在全国范围对约２０００名被试试测后,本研究对所获数据进行了初步分析,得到：（１）各变量原始数据的均数和标准差;（２）对各变量进行因素分析的结果表明：可以认为该测验测出注意的稳定性、广度、转移、持续性和集中性;（３）各变量原始数据有显著年龄差异,但无性别差异。（４）信效度检验结果基本符合心理测量学要求。（５）制定了全国常模。     

The differential aptitude test (language usage and spelling): A clinical study of a computerized form

As part of the Leicester/DHSS project on microcomputer-aided assessment, 103 subjects were tested and retested on standard and computerized versions of the Differential Aptitude Tests for Language Usage and Spelling (Forms S and T). Subjects were generally faster on the computerized versions than on the standard versions. On the Language Usage test, subjects scored significantly higher on the computerized than on the standard test. The correlations found between the standard and computerized versions were modest in comparison to the original test-retest reliabilities. It is concluded that these data argue against the claim that the current computerized versions of the tests are psychometrically parallel to the standard versions. This research, which was carried out when the authors were at the Department of Psychology, University of Leicester, was supported by the United Kingdom Department of Health and Social Security.     

Determination of optimal test length to maximize the multiple correlation

If the lengths of the tests in a battery are altered, their intercorrelations and their validities or correlations with a criterion are also altered. Consequently, the multiple correlation of the battery with the criterion will also be altered. These changes are a function of the reliabilities of the tests. Suppose we have given from a set of experimental data (1) the time allowed for each test in the battery, (2) the reliability of each test, (3) the intercorrelations, and (4) the validities of all the tests. If we specify the over-all testing time we are willing to allow for the test in the future, we can determine the amount by which each test must be altered in order to give the maximum multiple correlation with the criterion. The method is presented, together with numerical examples and the mathematical proof.     

Predicting adult temperament from minor physical anomalies

It is known that in children, individual differences in temperament are linked to rates of minor physical anomalies (MPAs). These anomalies are a standard set of 17 nonobvious but measurable characteristics of the face, hands, and feet. We explored the relation between MPAs and temperament in adults. An intact sample of 88 undergraduates completed a battery of personality tests and lifestyle questions, and then were examined for MPAs. The index of MPAs for each subject was simply the total count out of 17. The results showed a pattern of correlates parallel to that found in children. Among male subjects, correlations with MPAs were significant for the Physical Activity and Clumsiness factors of the lifestyle inventory as well as for measures of emotionality, extraversion, masculinity, femininity, and Type A personality. The MPA index was also predictive of a behavioral index of temperament in male subjects. No correlations were significant among female subjects.     

Performances on Symbol Digit Modalities Test,Color Trails Test,and modified Stroop test in a healthy,elderly Danish sample

ABSTRACT

This study presents Danish data for the Symbol Digit Modalities Test (SDMT), Color Trails Test (CTT), and a modified Stroop test from 100 subjects aged 60–87 years. Among the included demographic variables, age had the highest impact on test performances. Thus, the study presents separate data for different age groups. For SDMT and CTT1, Danish Adult Reading Test (DART) score also had a significant impact on test performances. The incongruent version of the modified Stroop test was significantly correlated to education. Moderate and significant correlations were found between the three tests. Even though the three tests are commonly used, few normative data for elderly exists. SDMT and CTT performances from this study were in the same range as previously published international norms, but the validity of the result from the modified Stroop test could not be investigated.     

Is strong inference really superior to simple inference?

The method of strong inference, wherein multiple hypotheses are constructed and a crucial experiment is carried out, is said to have special status in science because it guarantees falsifying results. However, the proposition that strong inference is in any way superior to the method of constructing and testing a single hypothesis is contradicted both by close rational analysis and by the empirical evidence. An experiment is reviewed in which subjects who conduct strong tests are much less likely to discover or approximate the truth than subjects who conduct simple tests of a false hypothesis. It is concluded that a potential to falsify is necessary for a test to have corroborative value; however, arguments as to the general superiority of one type of potentially falsifying test over another have no logical basis. Any claim as to a general superiority of strong tests over simple tests would require access to information about the probability of each strategy to produce various relationships between the truth and whatever explanations are most accessible, and such information is not knowable, even in principle.     

Note on a reanalysis of Davis' reading tests

Correlation data on nine reading tests originally analyzed by Frederick B. Davis by the principal axes method are reanalyzed by Spearman's uni-dimensional method. It is concluded that a single common factor (reading ability) accounts for the correlations among the tests with residuals remarkably small in view of the fact that the tests were designed to test nine supposedly different skills. Three of the tests showed additional specific variance not attributable to the common factor.     

The development of functional and equivalence classes in high-functioning autistic children: the role of naming.

The development of functional and equivalence classes was studied in four high-functioning, preschool-aged autistic children. Initially, all subjects failed to demonstrate match-to-sample relations indicative of stimulus equivalence among two three-member classes of visual stimuli. Then, 2 subjects showed emergence of those relations after they were taught to assign the same name to all members in each class. Next, subjects were taught names for new stimuli outside the match-to-sample format. On subsequent match-to-sample tests, 2 subjects demonstrated untrained conditional relations among the stimuli given a common name. New, unnamed stimuli were then related via match-to-sample training to stimuli from sets of named stimuli. Tests for emergent conditional relations between the new unnamed stimuli and the named stimuli yielded positive results for 1 subject and somewhat mixed results for 3 subjects. Finally, without naming, 2 subjects developed stimulus equivalence among two new three-member classes of visual stimuli. These data suggest that naming may remediate failures to develop untrained conditional relations, some of which are indicative of stimulus equivalence.     

Approximate methods in calculating discriminant functions

Approximate methods of solving for discriminant functions have been tried on three sets of data. The principal illustration is the problem of finding a weighted sum of scores, on four psychological tests, so that men and women may be distinguished most clearly. The work starts from the complete solution, due to R. A. Fisher, where it is necessary to solve as many simultaneous equations, dependent on the standard deviations of the tests and their mutual correlations, as there are tests. It is proposed, by way of numerical simplification, that a set of equations be substituted where some one quantity replaces all the correlations. A solution is obtained where the weights to be assigned the tests are very simply expressed in terms of differences between the mean values of tests, the standard deviations of tests, and the said quantity. The difficulty remains of finding an estimate of the arbitrary constant that will give good discrimination. If an optimal solution is made a result is obtained which, in the three sets of data considered, is almost indistinguishable from that yielded by the complete solution. The calculation of this optimal common quantity is, however, itself so considerable that another estimate, previously suggested by R. W. B. Jackson, appears more profitable. This estimate is derived simply from the variability between the total scores for each subject and the variability of each test. Using this estimate, the discriminant functions can be rapidly calculated; the results compare very favorably, in the case of the data considered, with those from the complete solution.The present work was done while the writer was employed by the Ontario Department of Health.     

Sensitivity of MRQAP Tests to Collinearity and Autocorrelation Conditions

Multiple regression quadratic assignment procedures (MRQAP) tests are permutation tests for multiple linear regression model coefficients for data organized in square matrices of relatedness among n objects. Such a data structure is typical in social network studies, where variables indicate some type of relation between a given set of actors. We present a new permutation method (called “double semi-partialing”, or DSP) that complements the family of extant approaches to MRQAP tests. We assess the statistical bias (type I error rate) and statistical power of the set of five methods, including DSP, across a variety of conditions of network autocorrelation, of spuriousness (size of confounder effect), and of skewness in the data. These conditions are explored across three assumed data distributions: normal, gamma, and negative binomial. We find that the Freedman–Lane method and the DSP method are the most robust against a wide array of these conditions. We also find that all five methods perform better if the test statistic is pivotal. Finally, we find limitations of usefulness for MRQAP tests: All tests degrade under simultaneous conditions of extreme skewness and high spuriousness for gamma and negative binomial distributions. Special thanks go to Cajo Ter Braak, Philip Hans Franses, Patrick Houweling, Pierre Legendre, three anonymous reviewers, the associate editor, and the editor for comments.     

Performance in normal subjects on a novel battery of driving-related sensory-motor and cognitive tests

Currently, there is no international standard for the assessment of fitness to drive for cognitively or physically impaired persons. A computerized battery of driving-related sensory-motor and cognitive tests (SMCTests) has been developed, comprising tests of visuoperception, visuomotor ability, complex attention, visual search, decision making, impulse control, planning, and divided attention. Construct validity analysis was conducted in 60 normal, healthy subjects and showed that, overall, the novel cognitive tests assessed cognitive functions similar to a set of standard neuropsychological tests. The novel tests were found to have greater perceived face validity for predicting on-road driving ability than was found in the equivalent standard tests. Test—retest stability and reliability of SMCTests measures, as well as correlations between SMCTests and on-road driving, were determined in a subset of 12 subjects. The majority of test measures were stable and reliable across two sessions, and significant correlations were found between on-road driving scores and measures from ballistic movement, footbrake reaction, hand-control reaction, and complex attention. The substantial face validity, construct validity, stability, and reliability of SMCTests, together with the battery’s level of correlation with on-road driving in normal subjects, strengthen our confidence in the ability of SMCTests to detect and identify sensory-motor and cognitive deficits related to unsafe driving and increased risk of accidents.     

An Improved Correction for Range Restricted Correlations Under Extreme,Monotonic Quadratic Nonlinearity and Heteroscedasticity

Standardized tests are frequently used for selection decisions, and the validation of test scores remains an important area of research. This paper builds upon prior literature about the effect of nonlinearity and heteroscedasticity on the accuracy of standard formulas for correcting correlations in restricted samples. Existing formulas for direct range restriction require three assumptions: (1) the criterion variable is missing at random; (2) a linear relationship between independent and dependent variables; and (3) constant error variance or homoscedasticity. The results in this paper demonstrate that the standard approach for correcting restricted correlations is severely biased in cases of extreme monotone quadratic nonlinearity and heteroscedasticity. This paper offers at least three significant contributions to the existing literature. First, a method from the econometrics literature is adapted to provide more accurate estimates of unrestricted correlations. Second, derivations establish bounds on the degree of bias attributed to quadratic functions under the assumption of a monotonic relationship between test scores and criterion measurements. New results are presented on the bias associated with using the standard range restriction correction formula, and the results show that the standard correction formula yields estimates of unrestricted correlations that deviate by as much as 0.2 for high to moderate selectivity. Third, Monte Carlo simulation results demonstrate that the new procedure for correcting restricted correlations provides more accurate estimates in the presence of quadratic and heteroscedastic test score and criterion relationships.     

Within-modal and cross-modal reliabilities of two laterality tests

Sixteen right handers were given four tests of a dichotic listening recall task (concrete words) and a visual half-field recall task (letters). There was a significant right ear and right visual half-field superiority for the four test sessions, with improvement in recall scores over sessions. Pearson correlations for each ear on the four dichotic tests ranged from .74 to .90. Correlations on the visual half-field scores, however, were significant only after the first session. Analysis of stability in lateral preferences across modalities revealed stable and reliable performance, particularly in the last two testings, where 81.3% of the subjects had a right-sided asymmetry. Cross-modal correlations, however, were not significant. Implications regarding predictions of speech laterality are discussed in the light of these findings.     

The Relationship Between Performance on the Gordon Diagnostic System and Other Measures of Attention

Tests purporting to measure attention were administered to college students to examine their relationship to three relatively new tests of attention from the Gordon Diagnostic System (GDS). All subjects received the Standard Delay, Vigilance, and Distractibility Tests of the GDS. In addition, 69 of 136 subjects received the Digit Span, Arithmetic, and Digit Symbol subtests of the Wechsler Adult Intelligence Scale-Revised (WAIS-R), and Kagan's Matching Familiar Figures Test. After the GDS, the other 67 subjects received the Visual Span subtest of the Wechsler Memory Scale-Revised (WMS-R), and the Stroop. Replication (across two studies) and gender influences were not evident on GDS performance. Performance data on this test for these 136 college students, collapsed across replication and gender, are presented. Although within-test correlations were strong, correlations between tests were weak. A ceiling effect on the Vigilance task may have effected its correlations with other tests. Results suggest that the GDS tasks are not substitutes for commonly used tests of attention, but may be useful as an additional source of information in the assessment of attention.     

The effects of selection in factor analysis

Factorial results are affected by selection of subjects and by selection of tests. It is shown that the addition of one or more tests which are linear combinations of tests already in a battery causes the addition of one or more incidental factors. If the given test battery reveals a simple structure, the addition of tests which are linear combinations of the given tests leaves the structure unaffected unless the number of incidental factors is so large that the common factors become indeterminate.     

Limited information estimation and testing of discretized multivariate normal structural models

Discretized multivariate normal structural models are often estimated using multistage estimation procedures. The asymptotic properties of parameter estimates, standard errors, and tests of structural restrictions on thresholds and polychoric correlations are well known. It was not clear how to assess the overall discrepancy between the contingency table and the model for these estimators. It is shown that the overall discrepancy can be decomposed into a distributional discrepancy and a structural discrepancy. A test of the overall model specification is proposed, as well as a test of the distributional specification (i.e., discretized multivariate normality). Also, the small sample performance of overall, distributional, and structural tests, as well as of parameter estimates and standard errors is investigated under conditions of correct model specification and also under mild structural and/or distributional misspecification. It is found that relatively small samples are needed for parameter estimates, standard errors, and structural tests. Larger samples are needed for the distributional and overall tests. Furthermore, parameter estimates, standard errors, and structural tests are surprisingly robust to distributional misspecification. This research was supported by the Department of Universities, Research and Information Society (DURSI) of the Catalan Government, and by grants BSO2000-0661 and BSO2003-08507 of the Spanish Ministry of Science and Technology.