Repeated challenge studies: A comparison of union-intersection testing with linear modeling

Challenge studies are often implemented for assessing whether a subject is sensitive to a certain agent or allergen. In particular, researchers test groups of subjects to determine if there really exists a causal relationship between some agent of interest and a response. To answer such a question, we need to detect the presence of the phenomenon in just one individual. Typically, however, there are a large number of unknown risk factors associated with the response and a potentially small population prevalence. Hence, standard statistical techniques, by averaging the treatment effect across the group, may miss a significant response of a single individual and lead to inconclusive results. We develop an alternative approach based on union-intersection testing that will allow a practitioner to correctly examine observations on an individual apart from the other subjects and test the hypothesis of interest: Does the phenomenon exist in the population? More specifically, we show how this technique adjusts for the multiple number of tests encountered when analyzing data for each individual subject separately. Furthermore, we demonstrate power calculations for the determination of sample size prior to performing the study. The performance of the union-intersection approach in comparison to linear models and semiparametric techniques is considered through sample size calculations and simulations. The union-intersection testing methodology out performs the Kolmogorov tests. However, the nested linear model performs as well if not better than the union-intersection tests. To illustrate the ideas presented in the paper, we provide an application in which we analyze psychological data collected by way of a challenge study design.     

Emotional disturbances and their interaction with neuropsychological deficits

The interaction and relationships between neuropsychological tests (which are principally oriented to intellectual and cognitive abilities) and tests of personality and emotional status are complex, but nevertheless important in the clinical assessment of brain-damaged persons. Are indications of emotional disturbances to be expected as a direct consequence of brain damage? If so, how can the indications of emotional disturbances be differentiated from results obtained with psychiatrically disturbed (non-brain-damaged) subjects? Some authors have presumed that emotional disturbances, such as depression, acute anxiety, etc., in their own right cause impaired performances on neuropsychological tests, whereas other authors have proposed that brain damage predisposes the individual to demonstrate evidence of emotional disturbances. If emotional disturbances cause impairment on neuropsychological tests, why is it that so many emotionally disturbed persons without brain damage tend to perform normally on neuropsychological tests? This review of relevant publications considers (1) different general approaches to these questions and their implications for neuropsychology, (2) evidence of differential sensitivity to brain damage of neuropsychological and emotional instruments, (3) the sensitivity and specificity of self-assessments and complaints of head-injured subjects, (4) MMPI findings among head-injured subjects and in interaction with neuropsychological measurements, and (5) principles and guidelines that may be of value in clinical application of findings reported in the literature.     

An application of order restricted inference methodology to a problem in psychiatry

Using data which reflects upon the genetic makeup of patients, Cadoret, Woolson and Winokur (1977) test two theories regarding the genetic makeup of patients suffering from unipolar affective disorder. This hypothesis test is a good example of a problem in order restricted inference. Using some recently developed techniques in order restricted inference we analyze these data and compare the analyses.This research was sponsored by the Office of Naval Research under contract N00014-80-C-0321.     

On the analysis of studies of choice

In a review of 103 sets of data from 23 different studies of choice, Baum (1979) concluded that whereas undermatching was most commonly observed for responses, the time measure generally conformed to the matching relation. A reexamination of the evidence presented by Baum concludes that undermatching is the most commonly observed finding for both measures. Use of the coefficient of determination by both Baum (1979) and de Villiers (1977) for assessing when matching occurs is criticized on statistical grounds. An alternative to the loss-in-predictability criterion used by Baum (1979) is proposed. This alternative statistic has a simple operational meaning and is related to the usual F-ratio test. It can therefore be used as a formal test of the hypothesis that matching occurs. Baum (1979) also suggests that slope values of between .90 and 1.11 can be considered good approximations to matching. It is argued that the establishment of a fixed interval as a criterion for determining when matching occurs, is inappropriate. A confidence interval based on the data from any given experiment is suggested as a more useful method of assessment.     

A monte carlo study of thirty internal criterion measures for cluster analysis

A Monte Carlo evaluation of thirty internal criterion measures for cluster analysis was conducted. Artificial data sets were constructed with clusters which exhibited the properties of internal cohesion and external isolation. The data sets were analyzed by four hierarchical clustering methods. The resulting values of the internal criteria were compared with two external criterion indices which determined the degree of recovery of correct cluster structure by the algorithms. The results indicated that a subset of internal criterion measures could be identified which appear to be valid indices of correct cluster recovery. Indices from this subset could form the basis of a permutation test for the existence of cluster structure or a clustering algorithm.     

The rowwise correlation between two proximity matrices and the partial rowwise correlation

This paper discusses rowwise matrix correlation, based on the weighted sum of correlations between all pairs of corresponding rows of two proximity matrices, which may both be square (symmetric or asymmetric) or rectangular. Using the correlation coefficients usually associated with Pearson, Spearman, and Kendall, three different rowwise test statistics and their normalized coefficients are discussed, and subsequently compared with their nonrowwise alternatives like Mantel'sZ. It is shown that the rowwise matrix correlation coefficient between two matricesX andY is the partial correlation between the entries ofX andY controlled for the nominal variable that has the row objects as categories. Given this fact, partial rowwise correlations (as well as multiple regression extensions in the case of Pearson's approach) can be easily developed.The author wishes to thank the Editor, two referees, Jan van Hooff, and Ruud Derix for their useful comments, and E. J. Dietz for a copy of the algorithm of the Mantel permutation test.     

Nonunique solutions to the likelihood equation for the three-parameter logistic model

Samejima identified the possibility of multiple solutions to the likelihood equation (multiple maxima in the likelihood function) for estimating an examinee's trait value for the three-parameter logistic model. In the practical applications that Lord studied, he found that multiple solutions did not occur when the number of items was 20. In the present paper, fourteen multiple-choice achievement tests with from 20 to 50 items were examined to see if it was possible for them to produce item response vectors with multiple maxima; such vectors were found for all the tests. Examination of response vectors for large groups of real examinees found that from 0 to 3.1% of them had response vectors with multiple maxima. The implications of these results for multiple-choice tests are discussed.     

Desirable features in neuropsychological tests

Seven desirable features are presented as guidelines for the design and evaluation of neuropsychological tests. Such features are not systematically present in most commonly used testing procedures because the field's methods have been borrowed from disciplines whose theories and measurement goals are quite different from those of modern neuropsychology. In the course of the presentation, three meanings of the term deficit are differentiated.     

The comparison of interdependent correlations between optimal linear composites

A general procedure is provided for comparing correlation coefficients between optimal linear composites. The procedure allows computationally efficient significance tests on independent or dependent multiple correlations, partial correlations, and canonical correlations, with or without the assumption of multivariate normality. Evidence from some Monte Carlo studies on the effectiveness of the methods is also provided.This research was supported in part by an operating grant (#67-4640) to the first author from the National Sciences and Engineering Research Council of Canada. The authors would also like to acknowledge the helpful comments and encouragement of Alexander Shapiro, Stanley Nash, and Ingram Olkin.